딥러닝 기반 한국어 개체명 인식의 평가와 오류 분석 연구
Error Analysis and Evaluation of Deep-learning Based Korean Named Entity Recognition
1
조회
0
다운로드
9
인용
0
참고문헌
키워드
초록
Named entity recognition is a natural language processing task that recognizes and classifies named entities in an unstructured text. The targets of NER are not limited to typical proper names for persons, locations and organizations, but also date, time and quantity expressions and can be further expanded to names of events, animals, plants, materials and other encyclopedic entities. A real-world NER system is also expected to be tuned to process domain-specific terminologies. In this study, the researchers built and tested a BERT based Korean NER system and proposed methods for evaluation and error analysis. The study trained the system with 140K word NER corpus and evaluated with 60K test. Error types are proposed to be categorized into four classes: detection, boundary, segmentation, and labelling. Error rates are found to vary greatly from 1% to 30% between entity labels, which are grouped into the most accurate time and quantity expressions, relatively accurate proper names, and highly erroneous terminologies. We expect that the error analysis will provide insights for finding a better way of data collection and post-processing correction.
Abstract
Named entity recognition is a natural language processing task that recognizes and classifies named entities in an unstructured text. The targets of NER are not limited to typical proper names for persons, locations and organizations, but also date, time and quantity expressions and can be further expanded to names of events, animals, plants, materials and other encyclopedic entities. A real-world NER system is also expected to be tuned to process domain-specific terminologies. In this study, the researchers built and tested a BERT based Korean NER system and proposed methods for evaluation and error analysis. The study trained the system with 140K word NER corpus and evaluated with 60K test. Error types are proposed to be categorized into four classes: detection, boundary, segmentation, and labelling. Error rates are found to vary greatly from 1% to 30% between entity labels, which are grouped into the most accurate time and quantity expressions, relatively accurate proper names, and highly erroneous terminologies. We expect that the error analysis will provide insights for finding a better way of data collection and post-processing correction.
참고문헌 (12)
- Chinchor, Nancy, "MUC-7 Named Entity Task Definition", Seventh Message Understanding Conference (MUC-7): Proceedings of a Conference Held in Fairfax (1997)
- Devlin, Jacob, "BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1 (2019)
- 세부분류개체명 가이드라인, "한국전자통신연구원", 언어지능연구그룹, SW 콘텐츠연구소, ETRI, ~ (2018)
- Hwang, Seokhyun, "Korean Named Entity Recognition using BERT", 2019 Proceedings of the Korea Information Processing Society Conference, pp.820~822 (2019)
- Jurafsky, Dan, "Speech and Language Processing" (2020)
- 2020 개체명 말뭉치 연구 분석, "국립국어원", 국립국어원, ~ (2020)
- Nouvel, D. Maud Ehrmann, "Named Entities for Computational Linguistics" (2016), ISTE Ltd and John Wiley & Sons
- 박서연, "한국어 어휘 의미망을 활용한 CRF 모델 기반 개체명 인식", 정보과학회논문지, 48(5), pp.556~567 (2021), 한국정보과학회[DOI]
- "2021", ~
- Sundheim, Beth M., "Overview of Results of the MUC-6 Evaluation", Sixth Message Understanding Conference (MUC-6): Proceedings of a Conference Held in Columbia (1995)
- Tjong Kim Sang, "Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition", Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pp.142~147 (2003)
- "2019", ~