딥러닝을 이용한 한국어 Head-Tail 토큰화 기법과 품사 태깅
Korean Head-Tail Tokenization and Part-of-Speech Tagging by using Deep Learning
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Korean is an agglutinative language, and one or more morphemes are combined to form a single word. Part-of-speech tagging method separates each morpheme from a word and attaches a part-of-speech tag. In this study, we propose a new Korean part-of-speech tagging method based on the Head-Tail tokenization technique that divides a word into a lexical morpheme part and a grammatical morpheme part without decomposing compound words. In this method, the Head-Tail is divided by the syllable boundary without restoring irregular deformation or abbreviated syllables. Korean part-of-speech tagger was implemented using the Head-Tail tokenization and deep learning technique. In order to solve the problem that a large number of complex tags are generated due to the segmented tags and the tagging accuracy is low, we reduced the number of tags to a complex tag composed of large classification tags, and as a result, we improved the tagging accuracy. The performance of the Head-Tail part-of-speech tagger was experimented by using BERT, syllable bigram, and subword bigram embedding, and both syllable bigram and subword bigram embedding showed improvement in performance compared to general BERT. Part-of-speech tagging was performed by integrating the Head-Tail tokenization model and the simplified part-of-speech tagging model, achieving 98.99% word unit accuracy and 99.08% token unit accuracy. As a result of the experiment, it was found that the performance of part-of-speech tagging improved when the maximum token length was limited to twice the number of words.
Abstract
Korean is an agglutinative language, and one or more morphemes are combined to form a single word. Part-of-speech tagging method separates each morpheme from a word and attaches a part-of-speech tag. In this study, we propose a new Korean part-of-speech tagging method based on the Head-Tail tokenization technique that divides a word into a lexical morpheme part and a grammatical morpheme part without decomposing compound words. In this method, the Head-Tail is divided by the syllable boundary without restoring irregular deformation or abbreviated syllables. Korean part-of-speech tagger was implemented using the Head-Tail tokenization and deep learning technique. In order to solve the problem that a large number of complex tags are generated due to the segmented tags and the tagging accuracy is low, we reduced the number of tags to a complex tag composed of large classification tags, and as a result, we improved the tagging accuracy. The performance of the Head-Tail part-of-speech tagger was experimented by using BERT, syllable bigram, and subword bigram embedding, and both syllable bigram and subword bigram embedding showed improvement in performance compared to general BERT. Part-of-speech tagging was performed by integrating the Head-Tail tokenization model and the simplified part-of-speech tagging model, achieving 98.99% word unit accuracy and 99.08% token unit accuracy. As a result of the experiment, it was found that the performance of part-of-speech tagging improved when the maximum token length was limited to twice the number of words.
참고문헌 (28)
- 박단호, "한글 형태소 및 키워드 분석에 기반한 웹 문서 분류", 정보처리학회논문지D, 19(4), pp.263~270 (2012), 한국정보처리학회
- W. Cho, "KoELECTRA-Based Named Entity Recognition Using Korean Morphological Analyzers", Proceedings of 2021 Korea Computer Congress, pp.1897~1899 (2021)
- 하예람, "형태소 분석을 통한 악성 댓글 필터링 방안", 한국콘텐츠학회 논문지, 21(9), pp.750~761 (2021), 한국콘텐츠학회[DOI]
- S. S. Kang, "Analysis of Korean Irregular Verbs by using Syllable Characteristics", Proceedings of 5th Hangul and Korean Information Processing, pp.385~394 (1993)
- S. S. Kang, "Multi-level Morphology and Morphological Analysis Model for Korean", Proceedings of 6th Hangul and Korean Information Processing, pp.140~145 (1994)
- 이재성, "한국어 형태소 분석을 위한 3단계 확률 모델", 정보과학회논문지 : 소프트웨어 및 응용, 38(5), pp.257~268 (2011), 한국정보과학회
- D. Bahdanau, "Neural Machine Translation by Jointly Learning to Align and Translate", ICLR 2015 (2015)
- A. Vaswan, "Attention Is All You Need", Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS) (2017)
- J. Devlin., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", preprint arXiv:1810.04805, ~ (2018)
- H. Y. Lee, "Bi-LSTM-CRF and Syllable Embedding for Automatic Spacing of Korean Sentences", Proceedings of 30th Hangul and Korean Information Processing, pp.605~607 (2018)
- 심광섭, "CRF를 이용한 한국어 자동 띄어쓰기", 인지과학, 22(2), pp.217~233 (2011), 한국인지과학회[DOI]
- 이현영, "음절 단위 임베딩과 딥러닝 기법을 이용한 복합명사 분해", 스마트미디어저널, 8(2), pp.74~79 (2019), (사)한국스마트미디어학회[DOI]
- J. H. Shin, "A Statistical Model for Korean Text Segmentation by using Syllable-level Bigrams", Proceedings of 9th Hangul and Korean Information Processing, pp.255~260 (1997)
- S. S. Kang, "Automatic Correction of Word-spacing Errors by using Syllable Bigram", Journal of Speech Science, 8(2), pp.83~90 (2001)
- D. H. Lim, "Word Segmentation by using Extended Syllable Bigram", Proceedings of 17th Hangul and Korean Information Processin, pp.189~193 (2005)
- T. Kudo, "Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp.66~75 (2018)
- T. Kudo, "SentencePiece: A Simple and Language Independent Subword Tokenizer nd Detokenizer for Neural Text Processing", Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System, pp.66~71 (2018)
- J. Lafferty, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data", Proceedings of the 18th International Conference on Machine Learning 2001, pp.282~289 (2001)
- T. Brants, "TnT - A Statistical Part-of-Speech Tagger", Proceedings of the 6th Applied Natural Language Processing Conference, pp.224~231 (2000)
- 윤준영, "한국어 형태소 분석 및 품사 태깅을 위한 딥 러닝 기반 2단계 파이프라인 모델", 정보과학회논문지, 48(4), pp.444~452 (2021), 한국정보과학회[DOI]
- 이건일, "Sequence-to-sequence 기반 한국어 형태소 분석 및 품사 태깅", 정보과학회논문지, 44(1), pp.57~62 (2017), 한국정보과학회
- 최용석, "트랜스포머와 BERT로 구현한 한국어 형태소 분석기의 성능 분석", 정보과학회논문지, 47(8), pp.730~741 (2020), 한국정보과학회[DOI]
- J. M. Kim, "Korean Head-Tail POS-Tagger by using Transformer", Proceedings of 33th Hangul and Korean Information Processing, pp.544~547 (2021)
- D. Kim, "Korean POS and Homonym Tagging System using HMM", Proceedings of 20th Hangul and Korean Information Processing, pp.12~16 (2008)
- J. H. Kim, "Korean Part-of-Speech Tagging using a Weighted Network", Journal of KIISE, 25(6), pp.951~959 (1998)
- 심광섭, "형태소 분석기 사용을 배제한 음절 단위의 한국어 품사 태깅", 인지과학, 22(3), pp.327~345 (2011), 한국인지과학회[DOI]
- J. M. Kim, "Korean Head-Tail POS-Tagger by using Transformer", Journal of Smart Media, 11(5), pp.17~25 (2022)
- D. Cho, "Hate Speech Detection by using Word Embedding and Deep Learning Model", M.S, pp.2021