Finding an Optimal Classification Model for Analyzing Linguistic Data
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This study aims to identify an AI classification model that is optimal for the classification of linguistic data. For this purpose, three commonly used classification models (XGBoost classifier, Random Forest Classifier, and SVM classifier) are compared in terms of their performance. Specifically, the three models are trained to classify the input data into essays and dialogues based on the syntactic complexity-related characteristics that distinguish between essays and dialogues. To determine if a model performing well on balanced data also performs well on imbalanced data, the three models’ performances are measured under two conditions: when the training dataset is balanced and when it is imbalanced. The performances of the trained models on the first test dataset are evaluated using accuracy, F1-score, normalized confusion matrix, and the area under the receiver operating characteristic curve. The performances on the second test dataset are assessed in terms of accuracy, confusion matrix, precision, and recall. The results demonstrate that the Random Forest Classifier has the best performance among the three models regardless of the balance of training data.
Abstract
This study aims to identify an AI classification model that is optimal for the classification of linguistic data. For this purpose, three commonly used classification models (XGBoost classifier, Random Forest Classifier, and SVM classifier) are compared in terms of their performance. Specifically, the three models are trained to classify the input data into essays and dialogues based on the syntactic complexity-related characteristics that distinguish between essays and dialogues. To determine if a model performing well on balanced data also performs well on imbalanced data, the three models’ performances are measured under two conditions: when the training dataset is balanced and when it is imbalanced. The performances of the trained models on the first test dataset are evaluated using accuracy, F1-score, normalized confusion matrix, and the area under the receiver operating characteristic curve. The performances on the second test dataset are assessed in terms of accuracy, confusion matrix, precision, and recall. The results demonstrate that the Random Forest Classifier has the best performance among the three models regardless of the balance of training data.
참고문헌 (16)
- Breiman, Leo, "Random Forests", Machine Learning, 45, pp.5~32 (2001)
- Chakraborty, Shankar, "Application of XGBoost Algorithm as a Predictive Tool in a CNC Turning Process", Reports in Mechanical Engineering, 2(1), pp.190~201 (2021)
- Chen, Tianqi, "Xgboost: A Scalable Tree Boosting System", Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.785~794 (2016), Association for Computing Machinery
- Hwang, Haerim, "Effects of Written versus Spoken Production Modalities on Syntactic Complexity Measures in Beginning Level Child EFL Learners", The Modern Language Journal, 104(1), pp.267~283 (2020)
- Ishikawa, Shin'ichiro, "The ICNALE and Sophisticated Contrastive Interlanguage Analysis of Asian Learners of English", Learner Corpus Studies in Asia and the World, 1, pp.91~118 (2013)
- Kim, Hyunwoo, "Additive Extensive Reading and Syntactic Development in Second Language Writing: Analyses of Syntactic Complexity and Sophistication in Young EFL Learners’ Book Reports", Journal of Second Language Writing, 61, pp.101040~ (2023)
- Kim, Wonbin, "How can Artificial Intelligence be Employed for Semantic Prosody Analysis?", Korean Journal of Applied Linguistics, 39(2), pp.3~34 (2023)
- Kim, Wonbin, "Let’s Make an Artificial Learner to Analyze Learners’ Language!", Studies in Linguistics, 70, pp.167~193 (2024)
- Kyle, Kristopher, "Measuring Syntactic Development in L2 Writing: Fine Grained Indices of Syntactic Complexity and Usage-based Indices of Syntactic Sophistication", PhD, pp.2016
- Li, Xiao, "A Debiased MDI Feature Importance Measure for Random Forests", Advances in Neural Information Processing Systems, 32, pp.8049~8059 (2019)
- Lu, Xiaofei, "Automatic Analysis of Syntactic Complexity in Second Language Writing", International Journal of Corpus Linguistics, 15(4), pp.474~496 (2010)
- Lyashevskaya, Olga, "Automated Assessment of Learner Text Complexity", Assessing Writing, 49, pp.100529~ (2021)
- Oh, Yoo Rhee, "Automatic Proficiency Assessment of Korean Speech Read Aloud by Non-Natives Using Bidirectional LSTM Based Speech Recognition", Etri Journal, 42(5), pp.761~772 (2020)
- Ryu, Jiseon, "Predicting Second Language Writing Proficiency in the Different Genres of Writing Using Computational Tools", Korean Journal of Applied Linguistics, 36(1), pp.141~170 (2020)
- Vapnik, Vladimir, "The Nature of Statistical Learning Theory" (2013), Springer
- Wang, Cong, "Automated Scoring of Chinese Grades 7 9 Students’ Competence in Interpreting and Arguing From Evidence", Journal of Science Education and Technology, 30, pp.269~282 (2021)