라벨 임베딩 분포를 사용한 효율적인 딥러닝 앙상블 방법
An Efficient Deep Learning Ensemble Using a Distribution of Label Embedding
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
본 연구에서는 레이블 임베딩의 분포를 반영하는 딥러닝 모형을 위한 새로운 스태킹 앙상블 방법론을 제안하였다. 제안된 앙상블 방법론은 기본 딥러닝 분류기를 학습하는 과정과 학습된 모형으로부터 얻어진 레이블 임베딩을 이용한 군집화 결과로부터 소분류기들을 학습하는 과정으로 이루어져있다. 본 방법론은 주어진 다중 분류 문제를 군집화 결과를 활용하여 소 문제들로 나누는 것을 기본으로 한다. 군집화에 사용되는 레이블 임베딩은 처음 학습한 기본 딥러닝 분류기의 마지막 층의 가중치로부터 얻어질 수 있다. 군집화 결과를 기반으로 군집화 내의 클래스들을 분류하는 소분류기들을 군집의 수만큼 구축하여 학습한다. 실험 결과 기본 분류기로부터의 레이블 임베딩이 클래스 간의관계를 잘 반영한다는 것을 확인하였고, 이를 기반으로 한 앙상블 방법론이 CIFAR 100 데이터에대해서 분류 성능을 향상시킬 수 있다는 것을 확인할 수 있었다.
Abstract
In this paper, we propose a new stacking ensemble framework for deep learning models which reflects the distribution of label embeddings. Our ensemble framework consists of two phases: training the baseline deep learning classifier, and training the sub-classifiers based on the clustering results of label embeddings. Our framework aims to divide a multi-class classification problem into small sub-problems based on the clustering results. The clustering is conducted on the label embeddings obtained from the weight of the last layer of the baseline classifier. After clustering, sub-classifiers are constructed to classify the sub-classes in each cluster. From the experimental results, we found that the label embeddings well reflect the relationships between classification labels, and our ensemble framework can improve the classification performance on a CIFAR 100 dataset.
참고문헌 (14)
- Alex Krizhevsky, "ImageNet classification with deep convolutional neural networks", Communications of the ACM, 60(6), pp.84~90 (2017), Association for Computing Machinery (ACM)[DOI]
- Sutskever, Ilya, "Sequence to sequence learning with neural networks", Advances in neural information processing systems, 27(1), pp.3104~3112 (2014)
- Friedman, Jerome, "The elements of statistical learning" (2001), Springer
- Malinin, Andrey, "Ensemble distribution distillation", arXiv preprint arXiv:1905.00076, 2019, ~ (2019)
- Hyungjin Ko, "Loss-Driven Adversarial Ensemble Deep Learning for On-Line Time Series Analysis", Sustainability, 11(12), pp.3489~ (2019), MDPI AG[DOI]
- M. Paz Sesmero, "CCE: An ensemble architecture based on coupled ANN for solving multiclass problems", Information Fusion, 58, pp.132~152 (2020), Elsevier BV[DOI]
- S. Park, "Inductive ensemble clustering using kernel support matching", Electronics Letters, 53(25), pp.1625~1626 (2017), Institution of Engineering and Technology (IET)[DOI]
- Sun, Xu, "Label embedding network: Learning label representation for soft training of deep networks", arXiv preprint arXiv:1710.10393, ~ (2017)
- Sellah, Smail, "Label Clustering for a Novel Problem Transformation in Multi-label Classification", J. UCS, 26(1), pp.71~88 (2020)
- Tao, Sean, "Internatio nal Conference on Machine Learning, Optimization, and Data Science" (2019), Springer
- Cheng-Yaw Low, "Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification", IEEE Transactions on Cybernetics, 50(12), pp.5021~5034 (2020), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Liu, J. Z., "Simple and principled uncertainty estimation with deterministic deep learning via distance awareness", Advances in Neural Information Processing Systems (2020)
- Cheng-Yaw Low, "Stacking-Based Deep Neural Network: Deep Analytic Network for Pattern Classification", IEEE Transactions on Cybernetics, 50(12), pp.5021~5034 (2020), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- Abdar, M, "Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges", arXiv preprint arXiv:2011.06225, ~ (2020)