회귀 최적화 기반 관련도와 중복도를 이용한 다중 레이블 특징 선별 기법
Multi-label Feature Selection Using Redundancy and Relevancy based on Regression Optimization
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
고차원 데이터는 기계학습에서 높은 시간 소모와 큰 메모리 요구로 학습에 어려움을 유발한다. 특히다중 레이블 환경에서는 레이블의 개수만큼 더 높은 복잡도를 요구한다. 본 논문에서는 다중 레이블환경에서 분류 성능 향상을 위한 특징 선별 기법을 제안한다. 중요한 특징을 선별하기 위해 특징과특징, 특징과 레이블, 레이블과 레이블, 세 가지 관계를 고려했으며 이를 위해 회귀 기반 목적 함수를설계하였다. 이 목적 함수는 특징과 레이블 사이의 선형적인 관계를 계산하고 상호 정보 기반의 특징과특징, 레이블과 레이블 사이의 관계를 계산한다. 이 목적 함수를 최소화하는 가중치를 찾아 특징을선별할 수 있다. 이 목적 함수 최적화를 위해 경사하강법 방식을 제시하여 빠르게 수렴할 수 있는알고리즘을 제안하였다. 여섯 개의 다중 레이블 데이터에 대한 실험 결과 제안된 방법이 기존 다중레이블 특징 선별 기법보다 높은 성능을 보여주었다. 여섯 개의 데이터 평균으로 본 제안 방법의 분류성능은 해밍 로스 0.1285, 랭킹 로스 0.1811, 다중 레이블 정확도 0.6416으로, 비교 대상 알고리즘인AMI(Approximating Mutual Information)에 비해 각각 0.0148, 0.0435, 0.0852 더 우수한 성능을 보였다.
Abstract
High-dimensional data causes difficulties in machine learning due to high time consumption and large memory requirements. In particular, in a multi-label environment, higher complexity is required as much as the number of labels. This paper proposes a feature selection method to improve classification performance in multi-label settings. The method considers three types of relationships: between features, between features and labels, and between labels themselves. To achieve this, a regression-based objective function is designed. This objective function calculates the linear relationships between features and labels and uses mutual information to compute relationships between features and between labels. By minimizing this objective function, the optimal weights for feature selection are found. To optimize the objective function, a gradient descent method is applied to develop a fast-converging algorithm. The experimental results on six multi-label datasets show that the proposed method outperforms existing multi-label feature selection techniques. The classification performance of the proposed method, averaged over six datasets, showed a Hamming loss of 0.1285, a ranking loss of 0.1811, and a multi-label accuracy of 0.6416. Compared to the AMI(Approximating Mutual Information) algorithm, the performance was better by 0.0148, 0.0435, and 0.0852, respectively.
참고문헌 (33)
- E. Smirni, "Workload-Aware Load Balancing for Cluster Web Servers", IEEE Trans. on Parallel and Distributed Systems, 16(3), pp.219~232 (2005)[DOI]
- E. Elhamifar, "Sparse subspace clustering : Algorithm, theory, and applications", IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(11), pp.2765~2781 (2013)[DOI]
- H. Lim, "Optimization approach for feature selection in multi-label classification", Pattern Recognition Letters, 89, pp.25~30 (2017)[DOI]
- J. Lee, "Fast multi-label feature selection based on information-theoretic feature ranking", Pattern Recognition, 48, pp.2761~2771 (2015)[DOI]
- S. Sharmin, "Simultaneous feature selection and discretization based on mutual information", Pattern Recognition, 91, pp.162~174 (2019)[DOI]
- R. B. Pereira, "Categorizing feature selection methods for multi-label classification", Artificial Intelligence Review, 49, pp.57~78 (2018)[DOI]
- G. Tsoumakas, "Random k-labelsets : an ensemble method for multilabel classification", Proceedings of the European Conference on Machine Learning, pp.406~417 (2007)
- J. Read, "A pruned problem transformation method for multi-label classification", Proceedings of the New Zealand Computer Science Research Student Conference, pp.143~150 (2008)
- M. L. Zhang, "Feature selection for multi-label naive Bayes classification", Information Sciences, 179, pp.3218~3229 (2009)[DOI]
- A. Clare, "Knowledge discovery in multi-label phenotype data", Proceedings of the European Conference on Principles of Data Mining and Knowledge Discovery, pp.42~53 (2001)
- J. Lee, "Feature selection for multi-label classification using multivariate mutual information", Pattern Recognition Letters, 34, pp.349~357 (2013)[DOI]
- Y. Fan, "Multi-label feature selection based on label correlations and feature redundancy", Knowledge-Based Systems, 241, pp.108256~ (2022)[DOI]
- Y. Li, "Multi-label feature selection via robust flexible sparse regularization", Pattern Recognition, 134, pp.109074~ (2023)[DOI]
- Y. Fan, "Learning correlation information for multi-label feature selection", Pattern Recognition, 145, pp.109899~ (2024)[DOI]
- Y. Li, "Label correlations variation for robust multi-label feature selection", Information Sciences, 609, pp.1075~1097 (2022)[DOI]
- L. Hu, "Feature-specific mutual information variation for multi-label feature selection", Information Sciences, 593, pp.449~471 (2022)[DOI]
- J. Dai, "Multi-label feature selection by strongly relevant label gain and label mutual aid", Pattern Recognition, 145, pp.109945~ (2024)[DOI]
- M. Faraji, "Multi-label feature selection with global and local label correlation", Expert Systems with Applications, 246, pp.123198~ (2024)[DOI]
- Z. He, "Multi-label feature selection based on correlation label enhancement", Information Sciences, 647, pp.119526~ (2023)[DOI]
- P. Zhang, "MFSJMI : Multi-label feature selection considering join mutual information and interaction weight", Pattern Recognition, 138, pp.109378~ (2023)[DOI]
- Y. Yang, "Multi-label feature selection based on stable label relevance and label-specific features", Information Sciences, 648, pp.119525~ (2023)[DOI]
- H. Peng, "Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy", IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, pp.1226~1238 (2005)[DOI]
- T. Klonecki, "Cost-constrained feature selection in multilabel classification using an information-theoretic approach", Pattern Recognition, 141, pp.1~18 (2023)[DOI]
- J. Lee, "Mutual information-based multi-label feature selection using interaction information", Expert Systems with Applications, 42, pp.2013~2025 (2015)[DOI]
- M. -L. Zhang, "ML-KNN : A lazy learning approach to multi-label learning", Pattern Recognition, 40, pp.2038~2048 (2007)[DOI]
- K. Trohidis, "Multilabel classification of music into emotions", International Conference on Music Information Retrieval, pp.325~330 (2008)
- S. Diplaris, "Protein Classification with Multiple Algorithms", Panhellenic Conference on Informatics, pp.448~456 (2005)
- J. P. Pestian, "A shared task involving multi-label classification of clinical free text", Proceedings of the Workshop on BioNLP 2007 : Biological, Translational, and Clinical Language Processing, pp.97~104 (2007)
- M. R. Boutell, "Learning multi-labelscene classiffication", Pattern Recognition, 37, pp.1757~1771 (2004)[DOI]
- A. Elisseeff, "A kernel method for multi-labelled classification", Advances in Neural Information Processing Systems (2001)
- J. Lee, "SCLS : Multi-label feature selection based on scalable criterion for large label set", Pattern Recognition, 66, pp.342~352 (2017)
- J. Lee, "Approximating mutual information for multi-label feature selection", Electronics Letters, 48, pp.1~2 (2012)[DOI]
- Y. Lin, "Multi-label feature selection based on max-dependency and min-redundancy", Neurocomputing, 168, pp.92~103 (2015)[DOI]