음소배열정보 기반 한국어 고유어, 한자어, 차용어의 머신러닝 분류
Machine learning of phonotactics for classifying native Korean, Sino-Korean and Loanwords in Korea
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The purpose of this study is to test models that automatically classify Korean nouns into native Korean, Sino-Korean, and loanwords by applying a machine learning model, naïve Bayes classification. In this study, 500 native Korean words, Sino- Korean words, and loanwords were collected, and after romanizing and decomposing them into bigram and trigram lists, the bigrams and trigrams were entered into the naïve Bayes classifier. We tested models with and without syllable boundaries, and found that both the bigram and trigram models were over 80% accurate. Contrary to the expectation that the performance of the models would improve as more information about Korean phonotactics was included in the training and validation data, the difference in performance between the bigram and trigram models was not significant. The model that included syllable boundaries in the phoneme sequence information had slightly higher accuracy than the model without syllable boundary information. When comparing the classification results of all five models, the accuracy of the bigram model with syllable boundaries was 83.55%, which was the best. For now, we have modified the model to consider only phoneme sequence information and syllable boundaries, but it is expected that the accuracy of the model can be improved by training the model while excluding bigrams and trigrams, which occur in similar proportions in all categories, and by increasing the size of the data.
Abstract
The purpose of this study is to test models that automatically classify Korean nouns into native Korean, Sino-Korean, and loanwords by applying a machine learning model, naïve Bayes classification. In this study, 500 native Korean words, Sino- Korean words, and loanwords were collected, and after romanizing and decomposing them into bigram and trigram lists, the bigrams and trigrams were entered into the naïve Bayes classifier. We tested models with and without syllable boundaries, and found that both the bigram and trigram models were over 80% accurate. Contrary to the expectation that the performance of the models would improve as more information about Korean phonotactics was included in the training and validation data, the difference in performance between the bigram and trigram models was not significant. The model that included syllable boundaries in the phoneme sequence information had slightly higher accuracy than the model without syllable boundary information. When comparing the classification results of all five models, the accuracy of the bigram model with syllable boundaries was 83.55%, which was the best. For now, we have modified the model to consider only phoneme sequence information and syllable boundaries, but it is expected that the accuracy of the model can be improved by training the model while excluding bigrams and trigrams, which occur in similar proportions in all categories, and by increasing the size of the data.
참고문헌 (14)
- CHAE, SEO-YOUNG, "The core-periphery structure in Korean lexicon reflected in a phonological variation and change", Studies in Phonetics, Phonology and Morphology, 5, pp.217~236 (1999)[DOI]
- 조혜선, "A maximum-entropy model of phonotactics for Korean male and female names", 음성음운형태론연구, 27(1), pp.99~129 (2021), 한국음운론학회[DOI]
- GOLDSMITH, JOHN, "Probabilistic models of grammar : Phonology as information minimization", Phonological Studies, 5, pp.21~46 (2002)
- HAYES, BRUCE, "A maximum entropy model of phonotactics and phonotactic learning", Linguistic Inquiry, 39, pp.379~440 (2008)
- "MaxEnt Grammar Tool" (2009)[URL]
- ITÔ, JUNKO, "University of Massachusetts Occasional Papers in Linguistics 18, Papers in Optimality Theory", pp.181~209 (1995), GLSA
- Frequency analysis of Korean morpheme and word usage 2, "KANG, BEOM-MO", Institute of Korean Culture, Korea University, ~ (2004)
- KANG, YONGSOON, "The organization of lexicon in Korean", Studies in Phonetics, Phonology and Morphology, 4, pp.55~67 (1998), The Phonology-Morphology Circle of Korea[DOI]
- KWAHK, KEE-YOUNG, "Machine Learning and Text Mining with R" (2023), Chung Ram Publishing
- 박나영, "한국어 명사의 음소배열제약에 대한 기계학습", 음성음운형태론연구, 20(3), pp.297~322 (2014), 한국음운론학회[DOI]
- 박나영, "고유어와 한자어의 비교 음소배열제약", 한국어학, 67, pp.147~192 (2015), 한국어학회
- 박나영, "한국어의 음운변동과 음소배열제약의 관계 탐구", 한말연구, 64(35), pp.1~32 (2023), 한말연구학회[DOI]
- 박선우, "한국어의 어휘계층과 음운론적 복잡성", 음성음운형태론연구, 19(2), pp.255~274 (2013), 한국음운론학회[DOI]
- SASAKI, JUN, "Ichiban Yasashi Bayes Toukei Nyumon" (2021), SB Creative Corp