생성형 대규모 언어 모델과 프롬프트 엔지니어링을 통한 한국어 텍스트 기반 정보 추출 데이터셋 구축 방법
A Study on Dataset Generation Method for Korean Language Information Extraction from Generative Large Language Model and Prompt Engineering
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
본 연구는 생성형 대규모 언어 모델을 활용하여 텍스트에서 정보를 추출하기 위한 한글 데이터셋 구축 방법을 탐구한다. 현대 사회에서는 혼합된정보가 빠르게 유포되며, 이를 효과적으로 분류하고 추출하는 것은 의사결정 과정에 중요하다. 그러나 이에 대한 학습용 한국어 데이터셋은 아직부족하다. 이를 극복하기 위해, 본 연구는 생성형 대규모 언어 모델을 사용하여 텍스트 기반 제로샷 학습(zero-shot learning)을 이용한 정보 추출을시도하며, 이를 통해 목적에 맞는 한국어 데이터셋을 구축한다. 본 연구에서는 시스템-지침-소스입력-출력형식의 프롬프트 엔지니어링을 통해언어 모델이 원하는 결과를 출력하도록 지시하며, 입력 문장을 통해 언어 모델의 In-Context Learning 특성을 활용하여 데이터셋을 구축한다. 생성된 데이터셋을 기존 데이터셋과 비교하여 본 연구 방법론을 검증하며, 관계 정보 추출 작업의 경우 KLUE-RoBERTa-large 모델 대비 25.47%더 높은 성능을 달성했다. 이 연구 결과는 한국어 텍스트에서 지식 요소를 추출하는 가능성을 제시함으로써 인공지능 연구에 도움을 줄 것으로기대된다. 더욱이, 이 방법론은 다양한 분야나 목적에 맞게 활용될 수 있어, 다양한 한국어 데이터셋 구축에 잠재력을 가진다고 볼 수 있다.
Abstract
This study explores how to build a Korean dataset to extract information from text using generative large language models. In modernsociety, mixed information circulates rapidly, and effectively categorizing and extracting it is crucial to the decision-making process. However, there is still a lack of Korean datasets for training. To overcome this, this study attempts to extract information using text-basedzero-shot learning using a generative large language model to build a purposeful Korean dataset. In this study, the language model isinstructed to output the desired result through prompt engineering in the form of “system”-“instruction”-“source input”-“output format”,and the dataset is built by utilizing the in-context learning characteristics of the language model through input sentences. We validateour approach by comparing the generated dataset with the existing benchmark dataset, and achieve 25.47% higher performance comparedto the KLUE-RoBERTa-large model for the relation information extraction task. The results of this study are expected to contribute toAI research by showing the feasibility of extracting knowledge elements from Korean text. Furthermore, this methodology can be utilizedfor various fields and purposes, and has potential for building various Korean datasets.
참고문헌 (26)
- R. Grishman, "Information extraction", IEEE Intelligent Systems, 30(5), pp.8~15 (2015)[DOI]
- W. Xiang, "A survey of event extraction from text", IEEE Access, 7, pp.173111~173137 (2019)[DOI]
- W. Liao, "A simple semi-supervised algorithm for named entity recognition", Pro-ceedings of the NAACL HLT 2009 Workshop on SemiSupervised Learning for Natural Language Processing, pp.58~65 (2009)
- D. Feng, "A small samples training framework for deep Learning-based automatic information extraction : Case study of construction accident news reports analysis", Advanced Engineering Informatics, 47, pp.101256~ (2021)
- Y. Chang, "A survey on evaluation of large language models", arXiv preprint arXiv : 2307. 03109, ~ (2023)
- J. Wei, "Emergent abilities of large language models", arXiv preprint arXiv : 2206. 07682, ~ (2022)
- M. T. R. Laskar, "A systematic study and comprehensive evaluation of ChatGPT on benchmark datasets", arXiv preprint arXiv : 2305. 18486, ~ (2023)
- J. Gou, "Knowledge distillation : A survey", International Journal of Computer Vision, 129(6), pp.1789~1819 (2021)
- Y. Sari, "Rule-based pattern extractor and named entity recognition : A hybrid approach", 2010 International Symposium on Information Technology, pp.563~568 (2010)
- C. Bizer, "DBpedia – A crystallization point for the web of data", Journal of Web Semantics : Science, Services and Agents on the WWW, 7(3), pp.154~165 (2009)
- Q. C. Bui, "A fast rule-based approach for biomedical event extraction", Proceedings of the BioNLP Shared Task 2013 Workshop, pp.104~108 (2013)
- S. Rao, "Biomedical event extraction using abstract meaning representation", BioNLP 2017, pp.126~135 (2017)
- A. Vaswani, "Attention is all you need", Advances in Neural Information Processing Systems, 30 (2017)
- J. Devlin, "Bert : Pre-training of deep bidirectional transformers for language understanding", arXiv preprint arXiv : 1810. 04805, ~ (2018)
- S. Liu, "Exploiting argument information to improve event detection via supervised attention mechanisms", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 1, pp.1789~1798 (2017)
- L. Zhao, "A BERT based Sentiment Analysis and Key Entity Detection Approach for Online Financial Texts", 2021 IEEE 24th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp.1233~1238 (2021)
- S. Park, "Klue : Korean language understanding evaluation", arXiv preprint arXiv : 2105. 09680, ~ (2021)
- T. Brown, "Language models are few-shot learners", Advances in Neural Information Processing Systems, 33, pp.1877~1901 (2020)
- J. Wei, "Chain-of-thought prompting elicits reasoning in large language models", Advances in Neural Information Processing Systems, 35, pp.24824~24837 (2022)
- S. M. Xie, "An explanation of in-context learning as implicit bayesian inference", arXiv preprint arXiv:2111.02080, ~ (2021)
- O. Sainz, "ZS4IE : A toolkit for zero-shot information extraction with simple verbalizations", arXiv preprint arXiv : 2203. 13602, ~ (2022)
- B. Sharma, "Multi-Task Training with In-Domain Language Models for Diagnostic Reasoning", Proceedings of the 5th Clinical Natural Language Processing Workshop, pp.78~85 (2023), Association for Computational Linguistics
- L. Ouyang, "Training language models to follow instructions with human feedback", Advances in Neural Information Processing Systems, 35, pp.27730~27744 (2022)
- X. Wei, "Zero-shot information extraction via chatting with chatgpt", arXiv preprint arXiv : 2302. 10205, ~ (2023)
- Christopher Walker, "ACE 2005 multilingual training corpus", Linguistic Data Consortium, 57~ (2006)
- M. Popović, "chrF: character n-gram F-score for automatic MT evaluation", Proceedings of the tenth workshop on statistical machine translation, pp.392~395 (2015)