트랜스포머 인코더를 활용한 음절 단위 경량화 형태소 분석기
Syllable-Level Lightweight Korean POS Tagger using Transformer Encoder
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
형태소 분석은 의미를 가지거나 문법적 기능을 하는 언어의 최소 단위인 형태소를 분리하고, 각 형태소의 품사를 결정하는 작업으로, 개체명인식, 의존구문 분석 등과 같은 자연어 처리 작업에서 중요한 역할을 한다. 현대 자연어처리의 많은 부분은 딥러닝 기반 언어 모델에 의존하고있으며, 딥러닝 기반 한국어 형태소 분석은 크게 시퀀스-투-시퀀스 방식과 순차적 레이블링 방식으로 나뉜다. 본 연구에서는 트랜스포머 인코더를활용한 순차적 레이블링 방식으로 음절 단위 품사 태깅을 수행한 후, 기분석 사전을 통해 형태소 복원 및 품사 태깅을 진행하는 형태소 분석 방식을제안한다. 또한, CBOW 방식을 사용하여 음절 단위 임베딩을 낮은 차원으로 추출함으로써 파라미터 수를 줄인 경량화 형태소 분석기 모델을 설계하였다. 제안된 모델은 낮은 파라미터 수와 빠른 추론 속도를 통해 자원이 제한된 환경에서도 효율적으로 활용될 수 있다.
Abstract
Morphological analysis involves segmenting morphemes, the smallest units of meaning or grammatical function in a language, andassigning part-of-speech tags to each morpheme. It plays a critical role in various natural language processing tasks, such as namedentity recognition and dependency parsing. Much of modern natural language processing relies on deep learning-based language models,and Korean morphological analysis can be broadly categorized into sequence-to-sequence methods and sequential labeling methods. Thisstudy proposes a morphological analysis approach using the transformer encoder for sequential labeling to perform syllable-levelpart-of-speech tagging, followed by morpheme restoration and tagging through a pre-analyzed dictionary. Additionally, the CBOW methodwas used to extract syllable-level embeddings in lower dimensions, designing a lightweight morphological analyzer model with reducedparameters. The proposed model achieves fast inference speed and low parameter usage, making it efficient for use in resource-constrainedenvironments.
참고문헌 (14)
- H. M. Kim, "Syllable-based Korean POS Tagging using POS Distribution and Bidirectional LSTM CRFs", Proceedings of the 28th Human & Cognitive Language Technology, pp.3~8 (2016)
- X. Ma, "End-to-end Sequence Labeling via Bi-directional LSTM-CNNs–CRF", Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 1, pp.1064~1074 (2016)
- S. W. Kim, "Research on Joint Models for Korean Word Spacing and POS(Part-Of-Speech)Tagging based on Bidirectional LSTM-CRF", Journal of Korean Institute of Information Scientists and Engineers, 45(8), pp.792~800 (2018)
- E. I. Chung, "Word Segmentation and POS tagging using Seq2seq Attention Model", Proceddings of the 28th Human & Cognitive Language Technology, pp.217~219 (2016)
- J. Li, "Sequence-to-sequence based Morphological Analysis and Part-Of-Speech Tagging for Korean Language with Convolutional Features", Journal of Korean Institute of Information Scientists and Engineers, 44(1), pp.57~62 (2017)
- J. W. Min, "End-to-End Neural Transition-based Morpheme Segmentation and POS Tagging of Korean", Proceedings of the Korean Information Science Society Conference, pp.566~568 (2019)
- Y. S. Choi, "Performance Analysis of Korean Morphological Analyzer based on Transformer and BERT", Journal of Korean Institute of Information Scientists and Engineers, 47(8), pp.730~741 (2020)
- B. S. Choe, "Korean Morphological Analyzer for Neologism and Spacing Error based on Sequence-to-Sequence", Journal of Korean Institute of Information Scientists and Engineers, 47(1), pp.70~77 (2020)
- J. Y. Youn, "A Deep Learning-based Two Steps Pipeline Model for Korean Morphological Analysis and Part-of-Speech Tagging", Journal of Korean Institute of Information Scientists and Engineers, 48(4), pp.444~452 (2021)
- A. Vaswani, "Attention is all you need", Advances in Neural Information Processing Systems, 30 (2017)
- C. J. Lee, "Korean Morphological Analysis Method Based on BERT-Fused Transformer Model", THE KIPS Transactions on Software and Data Engineering, 11(4), pp.169~178 (2022)
- "KorBERT"[URL]