어휘 및 형태 정보를 이용한한국어 Two-level 어휘사전 자동 구축
Automatic Construction of Korean Two-level Lexicon using Lexical and Morphological Information
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Two-level 형태소 분석 방법은 규칙 기반 방법 중 하나로 형태소의 변화 현상을 규칙으로 처리하고, 기본 어휘 사전을 기반으로 형태소 결합관계를 분석한다. 이는 언어에 독립적인 방법으로 한국어에 대해서도 일부 구축되어 적용됨이 증명되었다. 그러나 기존 한국어에 대한Two-level 형태소 분석기는 사전을 수동으로 구축하여 규모가 매우 작고 실제 사용에 제한적이었으며, 과분석이 많아 효율성이 매우 떨어졌다. 본 논문은 세종 품사부착 말뭉치에서 대규모의 Two-level 어휘 사전을 자동으로 구축하여 형태소 분석기의 적용 범위를 넓히고, 형태소간의결합관계를 어휘 정보와 어휘 형태에 따른 하위품사 정보를 이용하여 분석함으로써 형태소 분석기의 성능을 향상시킬 수 있는 방법을 제시한다. 실험 결과, 기존의 방법보다 형태소 분석기의 과분석을 68% 이상 줄여 f-measure를 25.5% point 이상 향상시킬 수 있었다
Abstract
Two-level morphology analysis method is one of rule-based morphological analysis method. This approach handles morphological transformation using rules and analyzes words with morpheme connection information in a lexicon. It is independent of language and Korean Two-level system was also developed. But, it was limited in practical use, because of using very small set of lexicon built manually. And it has also a over-generation problem. In this paper, we propose an automatic construction method of Korean Two-level lexicon for PC-KIMMO from morpheme tagged corpus. We also propose a method to solve over-generation problem using lexical information and sub-tags. The experiment showed that the proposed method reduced over-generation by 68% compared with the previous method, and the performance increased from 39% to 65% in f-measure.
참고문헌 (12)
- Koskenniemi, Kimmo, "Two-level Model for Morphological Analysis", IJCAI'83, International Joint Conference on Artificial Intelligence, pp.683~685 (1983)
- Koskenniemi, Kimmo, "A general computational model for word-form recognition and production", Proceedings for COLING-84:Association for Computational Linguistics, pp.178~181 (1984)
- PC-KIMMO :A Two-level Processor for Morphological Analyzis, "Antworth", Summer Institute of Linguistics, ~, pp.Occasional Publications in Academic Computing No.16 (1990)
- S. Lee, "A Two-level Morphological Analysis of Korean", 석사, Dept. of Computer Science, pp.1992
- S. Lee, "A Two-level Approach to Korean Verb Morphology", Proceedings of Fall Korea Information Science Society Conference, 19(2), pp.993~996 (1992)
- Barton. G. Edward, "Computational and Natural Language" (1987), The MIT Press
- Part- Of-Speech Tagged Corpus For Korean, 21C Sejong Project, "The national institute of the Korean Language", ~ (2011)
- A. Arppe, "Inquiries Into Words; a Festschrift for Kimmo Koskenniemi on his 60th Birthday", pp.71~83 (2005), CSLI Publications, Stanford University
- W. A. Gale, "A Program for Aligning Sentences in Bilingual Corpora, In Using Large Corpora", pp.75~102 (1994), The MIT Press
- S. Y. Kim, "A morphological analyzer for korean language with tabular parsing method and connectivity information", 석사, Dept. of Computer Science, pp.1987
- J. W. Kang, "A design and implementation of hangul spelling and word-spacing checker using connectivity information", 석사, Dept. of Computer Science, pp.1990
- J. S. Lee, "Automatic Construction of Korean Morphotactic for Two-level Lexicon", LaRC2011, International Conference on Terminology, Language and Content Resources (2011)