고차원 범주형 자료를 위한 비지도 연관성 기반 범주형 변수 선택 방법
Association-based Unsupervised Feature Selection for High-dimensional Categorical Data
0
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Purpose: The development of information technology makes it easy to utilize high-dimensional categorical data. In this regard, the purpose of this study is to propose a novel method to select the proper categorical variables in high-dimensional categorical data. Methods: The proposed feature selection method consists of three steps: (1) The first step defines the goodness- to-pick measure. In this paper, a categorical variable is relevant if it has relationships among other variables. According to the above definition of relevant variables, the goodness-to-pick measure calculates the normalized conditional entropy with other variables. (2) The second step finds the relevant feature subset from the original variables set. This step decides whether a variable is relevant or not. (3) The third step eliminates redundancy variables from the relevant feature subset. Results: Our experimental results showed that the proposed feature selection method generally yielded better classification performance than without feature selection in high-dimensional categorical data, especially as the number of irrelevant categorical variables increase. Besides, as the number of irrelevant categorical variables that have imbalanced categorical values is increasing, the difference in accuracy between the proposed method and the existing methods being compared increases. Conclusion: According to experimental results, we confirmed that the proposed method makes it possible to consistently produce high classification accuracy rates in high-dimensional categorical data. Therefore, the proposed method is promising to be used effectively in high-dimensional situation.
Abstract
Purpose: The development of information technology makes it easy to utilize high-dimensional categorical data. In this regard, the purpose of this study is to propose a novel method to select the proper categorical variables in high-dimensional categorical data. Methods: The proposed feature selection method consists of three steps: (1) The first step defines the goodness- to-pick measure. In this paper, a categorical variable is relevant if it has relationships among other variables. According to the above definition of relevant variables, the goodness-to-pick measure calculates the normalized conditional entropy with other variables. (2) The second step finds the relevant feature subset from the original variables set. This step decides whether a variable is relevant or not. (3) The third step eliminates redundancy variables from the relevant feature subset. Results: Our experimental results showed that the proposed feature selection method generally yielded better classification performance than without feature selection in high-dimensional categorical data, especially as the number of irrelevant categorical variables increase. Besides, as the number of irrelevant categorical variables that have imbalanced categorical values is increasing, the difference in accuracy between the proposed method and the existing methods being compared increases. Conclusion: According to experimental results, we confirmed that the proposed method makes it possible to consistently produce high classification accuracy rates in high-dimensional categorical data. Therefore, the proposed method is promising to be used effectively in high-dimensional situation.
참고문헌 (25)
- Blum, A. L., "Selection of Relevant Features and Examples in Machine Learning", Artificial intelligence, 97(1-2), pp.245~271 (1997)
- Burnaby, T. P, "On a Method for Character Weighting a Similarity Coefficient, Employing the Concept of Information", Journal of the International Association for Mathematical Geology, 2(1), pp.25~38 (1970)
- Chakraborty, D. D, "Statistical Decision Theory. Estimation, Testing and Selection", Investigación Operacional, 29(2), pp.184~185 (2008)
- Cheng, V., "Dissimilarity learning for nominal data", Pattern Recognition, 37(7), pp.1471~1477 (2004)
- 정혜란, "4차 산업혁명시대의 품질경영", 품질경영학회지, 45(4), pp.629~648 (2017), 한국품질경영학회[DOI]
- Cost, S., "A Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features", Machine learning, 10(1), pp.57~78 (1993)
- Goodall, D. W, "A New Similarity Index Based on Probability", Biometrics, pp.882~907 (1966)
- Guyon, I., "An Introduction to Variable and Feature Sselection", Journal of Machine Learning Research, 3, pp.1157~1182 (2003)
- Hamming, R. W, "Error Detecting and Error Correcting Codes", Bell System Technical Journal, 29(2), pp.147~160 (1950)
- Jia, H., "A New Distance Metric for Unsupervised Learning of Categorical Data", IEEE Transactions on Neural Networks and Learning Systems, 27(5), pp.1065~1079 (2016)
- Kullback, S., "On Information and Sufficiency", The Annals of Mathematical Statistics, 22(1), pp.79~86 (1951)
- Le, S. Q., "An Association-based Dissimilarity Measure for Categorical Data", Pattern Recognition Letters, 26(16), pp.2549~2557 (2005)
- Lin, J, "Divergence Measures Based on the Shannon Entropy", IEEE Transactions on Information Theory, 37(1), pp.145~151 (1991)
- Lin, D, "An Information-theoretic Definition of Similarity", In Icml, 98(1998), pp.296~304 (1998)
- Liu, H., "Feature Selection with Dynamic Mutual Information", Pattern Recognition, 42(7), pp.1330~1339 (2009)
- Mitra, P., "Unsupervised Feature Selection Using Feature Similarity", IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3), pp.301~312 (2002)
- Il seok Oh, "Pattern recognition" (2008), Kyobo book
- 박영준, "주성분 분석 로딩 벡터 기반 비지도 변수 선택 기법", 대한산업공학회지, 40(3), pp.275~282 (2014), 대한산업공학회[DOI]
- Quinlan, J. R., "C4. 5: Programs for Machine Learning" (2014), Elsevier
- 이상복, "4차산업혁명시대의 한국품질경영 제안", 품질경영학회지, 45(4), pp.739~760 (2017), 한국품질경영학회[DOI]
- Smirnov, E. S, "On Exact Methods in Systematics", Systematic Biology, 17(1), pp.1~13 (1968)
- Stanfill, C., "Toward Memory-based reasoning. Commun", ACM, 29(12), pp.1213~1228 (1986)
- Tibshirani, R., "Regression Shrinkage and Selection Via the Lasso", Journal of the Royal Statistical Society: Series B (Methodological), 58(1), pp.267~288 (1996)
- Vergara, J. R., "A Review of Feature Selection Methods Based on Mutual Information", Neural Computing and Applications, 24(1), pp.175~186 (2014)
- Xie, J., "Feature Selection in Data Mining", pp.97~106 (2010)