한국어 단어 및 문장 분류 태스크를 위한 분절 전략의 효과성 연구
A Comparative study on the Effectiveness of Segmentation Strategies for Korean Word and Sentence Classification tasks
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
효과적인 분절을 통한 양질의 입력 자질 구성은 언어모델의 문장 이해력을 향상하기 위한 필수적인 단계이다. 입력 자질의 품질 제고는 세부 태스크의 성능과 직결된다. 본 논문은 단어와 문장 분류 관점에서 한국어의 언어적 특징을 효과적으로 반영하는 분절 전략을 비교 연구한다. 분절 유형은 언어학적 단위에 따라 어절, 형태소, 음절, 자모 네 가지로 분류하며, RoBERTa 모델 구조를 활용하여 사전학습을 진행한다. 각 세부 태스크를 분류 단위에 따라 문장 분류 그룹과 단어 분류 그룹으로 구분 지어 실험함으로써, 그룹 내 경향성 및 그룹 간 차이에 대한 분석을 진행한다. 실험 결과에 따르면, 문장 분류에서는 자모 단위의 언어학적 분절 전략을 적용한 모델이 타 분절 전략 대비 최대 NSMC: +0.62%, KorNLI: +2.38%, KorSTS: +2.41% 높은 성능을, 단어 분류에서는 음절 단위의 분절 전략이 최대 NER: +0.7%, SRL: +0.61% 높은 성능을 보임으로써, 각 분류 그룹에서의 효과성을 보여준다.
Abstract
The construction of high-quality input features through effective segmentation is essential for increasing the sentence comprehension of a language model. Improving the quality of them directly affects the performance of the downstream task. This paper comparatively studies the segmentation that effectively reflects the linguistic characteristics of Korean regarding word and sentence classification. The segmentation types are defined in four categories: eojeol, morpheme, syllable and subchar, and pre-training is carried out using the RoBERTa model structure. By dividing tasks into a sentence group and a word group, we analyze the tendency within a group and the difference between the groups. By the model with subchar-level segmentation showing higher performance than other strategies by maximal NSMC: +0.62%, KorNLI: +2.38%, KorSTS: +2.41% in sentence classification, and the model with syllable-level showing higher performance at maximum NER: +0.7%, SRL: +0.61% in word classification, the experimental results confirm the effectiveness of those schemes.
참고문헌 (18)
- Y. Liu., "Roberta : A robustly optimized bert pretraining approach", arXiv preprint arXiv:1907.11692., ~ (2019)
- J. Devlin, "Bert : Pre-training of deep bidirectional transformers for language understanding", arXiv preprint arXiv:1810.04805., ~ (2018)
- R. Sennrich, "Neural machine translation of rare words with subword units", arXiv preprint arXiv:1508.07909, ~ (2015)
- Y. Wu., "Google's neural machine translation system : Bridging the gap between human and machine translation", arXiv preprint arXiv:1609.08144., ~ (2016)
- T. Kudo., "Sentencepiece : A simple and language independent subword tokenizer and detokenizer for neural text processing", arXiv preprint arXiv:1808.06226., ~ (2018)
- M. Kim, "Advanced subword segmentation and interdependent regularization mechanisms for korean language understanding", 2019 Third World Conference on Smart Trends in Systems Security and Sustainablity (WorldS4), pp.221~227 (2019)
- O. Kwon, "Handling Out-Of-Vocabulary Problem in Hangeul Word Embeddings", Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pp.3213~3221 (2021)
- S. Park, "Subword-level word vector representations for Korean", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 1, pp.2429~2438 (2018)
- S. Lee, "Kr-bert: A small-scale korean-specific language model", arXiv preprint arXiv:2008.03979, ~ (2020)
- A. Matteson, "Rich character-level information for Korean morphological analysis and part-of-speech tagging", Proceedings of the 27th International Conference on Computational Linguistics, pp.2482~2492 (2018)
- S. Moon, "Jamo Pair Encoding: Subcharacter Representation-based Extreme Korean Vocabulary Compression for Efficient Subword Tokenization", Proceedings of the 12th Language Resources and Evaluation Conference, pp.3490~3497 (2020)
- Danbi Cho, "An Empirical Study of Korean Sentence Representation with Various Tokenizations", Electronics, 10(7), pp.845~ (2021), MDPI AG[DOI]
- K. Park, "An Empirical Study of Tokenization Strategies for Various Korean NLP Tasks", arXiv preprint arXiv:2010.02534., ~ (2020)
- T. Kudo., "Subword regularization : Improving neural network translation models with multiple subword candidates", arXiv preprint arXiv:1804.10959, ~ (2018)
- E. F. Sang, "Introduction to the CoNLL-2003 shared task : Language-independent named entity recognition", arXiv preprint cs, 2003
- S. Park., "KLUE : Korean Language Understanding Evaluation", arXiv preprint arXiv:2105.09680., ~ (2021)
- J. Ham, "Kornli and korsts : New benchmark datasets for korean natural language understanding", arXiv preprint arXiv:2004.03289, ~ (2020)
- M. Ott, "fairseq : A fast, extensible toolkit for sequence modeling", arXiv preprint arXiv:1904.01038, ~ (2019)