한국어 어절 단위 중의성 해소 평가 데이터의 구축과 활용

Abstract

This study discusses a construction methodology of an evaluation dataset for Korean eojeol-level ambiguity-resolution and a framework for measuring model performance objectively and consistently. Using the NIKL part-of-speech tagged corpus and the 21st Century Sejong Corpus, we define ambiguity at the eojeol level as “the same surface form within a single eojeol which has more than one distinct analysis.” This definition is implemented in SQL to systematically harvest ambiguous eojeols and to package sentence-level items with gold analyses in a standardized JSON format. The resulting dataset comprises 8,075 ambiguous eojeols and 35,396 sentences and covers 506 fine-grained POS-combination types, with ambiguity strongly concentrated in Noun+Particle and Verb+Ending patterns. For a long time, however, Korean disambiguation has typically been evaluated on relatively easy cases or on small lists of hand-picked ambiguous items, which tend to make performance look higher than it actually is. On our dataset, eight widely used morphological analyzers achieve an average accuracy of 55.71% (SD=17.62), with the best model showing 83.1%, indicating that realistic eojeol-level ambiguity resolution is substantially harder than suggested by existing evaluations. The dataset supports type- and difficulty-stratified evaluation across segmentation, POS selection, lemma restoration, and major POS-combination patterns, exposing where systems actually fail and guiding targeted improvement. We release it as an open, living benchmark via a public repository with documentation and a leaderboard. By grounding evaluation in real usage and in the morphological realities of Korean, the proposed dataset and protocol provide a more reliable and conservative assessment of eojeol-level disambiguation and offer a transferable template for languages with similar patterns of morphological ambiguity.