지속적 학습 환경에서 효율적 경로 선택
Efficient Path Selection in Continuous Learning Environment
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 논문에서는, 지속적 학습 환경에서 효율적 경로 선택에 의한 LwF방법의 성능향상을 제안한다. 이를 위해 콘볼루션 레이어를 분리하는 방법을 사용하여 기존의 LwF와 성능 및 구조를 비교한다. 비교를 위해 복잡도가 다른 구성을 가진MNIST, EMNIST, Fashion MNIST, CIFAR10 데이터를 사용하여 성능을 실험하였다. 실험결과, 각 태스크 별 정확도가 최대 20% 향상되었으며, LwF 기반의 지속적 학습 환경에서 치명적 망각 현상이 개선되었다.
Abstract
In this paper, we propose a performance improvement of the LwF method using efficient path selection in ContinuousLearning Environment. We compare performance and structure with conventional LwF. For comparison, we experimentwith performance using MNIST, EMNIST, Fashion MNIST, and CIFAR10 data with different complexity configurations. Experiments show up to 20% improvement in accuracy for each task, which mitigating the Catastrophic Forgettingphenomenon in Continuous Learning environments.
참고문헌 (10)
- Z. Li, "Learning without forgetting", IEEE transactions on pattern analysis and machine intelligence, 40(12), pp.2935~2947 (2017)[DOI]
- IJ. Goodfellow, "An empirical investigation of catastrophic forgetting in gradient-based neural networks" (2013)
- GI. Parisi, "Continual lifelong learning with neural networks: A review", Neural Networks, 113, pp.54~71 (2019)[DOI]
- RM. French, "Catastrophic forgetting in connectionist networks", Trends in cognitive sciences, 3(4), pp.128~135 (1999)[DOI]
- YC. Hsu, "Re-evaluating continual learning scenarios: A categorization and case for strong baselines" (2018)
- K. McRae, "Catastrophic interference is eliminated in pretrained networks", Proceedings of the 15h Annual Conference of the Cognitive Science Society,, pp.723~728 (1993)
- J. Kirkpatrick, "Overcoming catastrophic forgetting in neural networks", roceedings of the national academy of sciences, 114(13), pp.3521~3526 (2017)
- G. Hinton, "Distilling the knowledge in a neural network", NIPSWorkshop (2014)
- F. Zenke, "Continual learning through synaptic intelligence", Proceedings of the 34th International Conference on Machine Learning, 70, pp.3987~3995 (2017), JMLR
- 박성현, "데이터 유사도를 이용한 지속적 학습방법", 전기전자학회논문지, 24(2), pp.514~522 (2020), 한국전기전자학회[DOI]