본문 바로가기
KERD
논문KCI 등재

챗봇의 자연어 이해 학습데이터 증강을 위한 의도 분석/생성용 언어자원의 구축

Construction of Language Resources for Augmenting Intent-annotated Datasets Required for Training Chatbot NLU Models

남지순|황창회|유광훈
언어정보연구소 (Institute for the Study of Language and Information)|2021.11|언어와 정보 사회 (Language & Information Society)|제44권 |pp. 89-125|DOI 10.29211/soli.2021.44..004

1

조회

0

다운로드

1

인용

0

참고문헌

키워드

챗봇ChatbotDECO 전자사전Semi-automatic Symbolic PropagationDECO Dictionary토픽-의도 주석 데이터셋반자동 언어 데이터 증강LGG 패턴문법Topic-Intent-Annotated DatasetsLGG local patterns

초록

This study aims to propose language resources named Deco-TIA/TIG, meticulously developed to annotate and generate TIAD (Topic-Intent-Annotated Datasets) as a NLU (natural language understanding) training dataset for customer service chatbots in the FinTech field. Deco-TIA/TIG are based on Semi-automatic Symbolic Propagation methodology with the DECO (Dictionnaire Electronique du COréen) Korean dictionary and Local-Grammar Graphs. By analyzing actual data web-scraped from fintech services, we found canonical linguistic constructs consisting of topic, event, discourse components with diverse patterns in Korean queries, complaints, and request texts. Deco-TIA/TIG are built to effectively provide high-quality annotated training data for Fintech CS chatbots by covering 32 topics, 38 events, and 15 discourse types extensively. For performance evaluation, we trained DIET (dual intent and entity transformer) and KoBERT framework with TIAD datasets. DIET-only shows F1-score 0.936 (Intent) / 0.854 (Slot/Entity), and DIET+KoBERT shows F1-score 0.95 (Intent) / 0.932 (Slot/Entity).

Abstract

This study aims to propose language resources named Deco-TIA/TIG, meticulously developed to annotate and generate TIAD (Topic-Intent-Annotated Datasets) as a NLU (natural language understanding) training dataset for customer service chatbots in the FinTech field. Deco-TIA/TIG are based on Semi-automatic Symbolic Propagation methodology with the DECO (Dictionnaire Electronique du COréen) Korean dictionary and Local-Grammar Graphs. By analyzing actual data web-scraped from fintech services, we found canonical linguistic constructs consisting of topic, event, discourse components with diverse patterns in Korean queries, complaints, and request texts. Deco-TIA/TIG are built to effectively provide high-quality annotated training data for Fintech CS chatbots by covering 32 topics, 38 events, and 15 discourse types extensively. For performance evaluation, we trained DIET (dual intent and entity transformer) and KoBERT framework with TIAD datasets. DIET-only shows F1-score 0.936 (Intent) / 0.854 (Slot/Entity), and DIET+KoBERT shows F1-score 0.95 (Intent) / 0.932 (Slot/Entity).

참고문헌 (22)

  1. 심층 신경망 기반 대화처리 기술 동향, "권오욱", Electronics and Telecommunications Trends, 55~64 (2019)
  2. 남지순, "질의응답시스템을 위한 속성질의문의 의문사 및 속성명사에 대한 연구", 언어학, 18(4), pp.167~197 (2010), 대한언어학회[DOI]
  3. 남지순, "코퍼스 분석을 위한 한국어 전자사전 구축방법론" (2018), 도서출판 역락
  4. 금융앱 챗봇의 사용자 의도분석 모델 개발을 위한 학습데이터이중 증강 방법, "남지순", DICORA 연구센터, 한국외국어대학교, ~, pp.DICORA-TR-2021-12 (2021)
  5. 류연준, "인공지능 챗봇을 기반으로 한환자-의사 소통 증진 소프트웨어", 한국정보처리학회 학술대회논문집, 27(2), pp.501~504 (2020)
  6. "쓸수록 진화하는 ‘챗봇’, 3년 후 10兆 시장 간다" (2021)[URL]
  7. 정천수, "포스트 코로나19 언택트 시대 대응을 위한 AI 챗봇 구축방법에 관한 연구", 한국IT서비스학회지, 19(4), pp.31~47 (2020), 한국IT서비스학회[DOI]
  8. 최수정, "룰 기반 챗봇을 활용한 교통정보 서비스 개발", 한국정보과학회 학술발표논문집, pp.1868~1869 (2017)
  9. 유광훈, "DECO-LGG 반자동 증강 학습데이터 활용 멀티태스크 트랜스포머 모델 기반 핀테크 CS 챗봇 NLU 시스템", 한글 및 한국어 정보처리 학술대회 프로시딩 (2021)
  10. Braun, D., "Evaluating Natural Language Understanding Services for Conversational Question Answering Systems", Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue (2017)
  11. Budzianowski, P., "Multiwoz-a Largescale Multi-domain Wizard-of-Oz Dataset for Taskoriented Dialogue Modelling", Proceedings of Conf. Empirical Methods Natural Language Process, pp.5016~5026 (2018)
  12. Bunk, T., "Diet: Lightweight language understanding for dialogue systems", arXiv preprint arXiv:2004.09936, ~ (2020)
  13. Chen, Q., "BERT for joint intent classification and slot filling", CoRR, abs, 2019 (1902)
  14. Gross, M., "The Construction of Local Grammars", pp.329~354 (1997), The MIT Press
  15. Gross, M., "Nouvelles applications des graphes d’automates finis a la description linguistique", Lingvisticae Investigationes, 22(1-2), pp.249~262 (1999)
  16. Harris, Z., "Distributional structure", Word, 10(23), pp.146~162 (1954)
  17. Hwang, M., "KoRasa : Pipeline Optimization for Open-Source Korean Natural Language Understanding Framework Based on Deep Learning", Mobile Information Systems, 2021, pp.9~ (2021)
  18. Lowe, R., "The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems", SIGDIAL, pp.285~294 (2015)
  19. Paumier, S., "De la Reconnaissance de Formes Linguistiques à l’analyse Syntaxique", Ph.D., pp.2003
  20. Ravuri, S., "Recurrent Neural Network and LSTM Models for Lexical Utterance Classification", Proceedings of Interspeech, pp.135~139 (2015)
  21. Shawar, B. A., "Using corpora in machine-learning chatbot systems", International journal of corpus linguistics, 10(3), pp.489~516 (2014)
  22. Williams, J., "The dialog state tracking challenge", Proceedings of the SIGDIAL 2013 Conference, pp.404~413 (2013)

이 간행물의 다른 논문 (9건)

[X+(을)+하다] 형식의 분리성 용언과 어휘고도제약이호승pp. 1-26중국어 초급 단계 회화교재의 기능 -어법 결합 현황 분석 및 고찰최현미pp. 27-64중국어 성경에 반영된 차용어 표기법 연구: ‘以’자를 중심으로신원철, 송지현pp. 65-87한국어 자유 선택어의 양화적 특성과 명령문에서의 인허 조건허세문pp. 127-164언어 학습자로서의 신경망 언어모델: 영어 주어-동사 일치를 중심으로정원일, 박명관, 김유희pp. 165-191
원문 보기