목적지향 대화시스템에서 LSTM 언어모델 기반의 한국어 자연어 생성
Korean Natural Language Generation Using LSTM-based Language Model forTask-Oriented Spoken Dialogue System
0
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
대화시스템에서 자연어 생성은 대화관리 단계에서 결정한 시스템 발화의 의미표현을 사람이 이해할 수 있는 자연어로 생성하는 것이다. 기존의 자연어 생성 연구는 의미표현에 대하여 매우 제한된 종류의 발화만을 생성하거나 문법적으로 불완전한 발화를 생성한다는 문제점이 있다. 그래서 본 논문에서는 문제점들을 동시에 처리하기 위하여 Long Short Term Memory 기반의 언어모델을 이용한 한국어 자연어 생성 모델을 제안한다. 특히 우리는 시스템 발화의 다양성과 문법적 정확성을 높이기 위하여 빔서치 디코딩을 적용한다. 실험은 어절, 형태소, 음절단위에 따라 개별적으로 진행하였으며, 생성한 문장들은 정량적, 정성적 평가를 모두 진행하였다. 그 결과 형태소 단위로 학습한 제안모델에 빔서치 디코딩을 적용한 방법은 가장 좋은 성능을 보였다. 실제로 해당 생성 문장은 정량평가 결과에서 BLEU 지표는 0.86, Slot Error Rate 지표는 0.03을 기록하였으며 정성평가 역시 문법적으로 정확하고 문맥적으로 충분히 자연스러운 결과임을 확인하였다.
Abstract
Natural language generation in the dialogue system is a task that transforms the semantic frame of the system utterance determined in the dialogue management phase into a natural language that can be understood by humans. Existing studies have still faced some obstacles in that only very limited types of utterances or grammatically incomplete ones are generated from the semantic frames. In order to address these issues simultaneously, we propose a Korean natural language generation model using a long short term memory based language model. In particular, we exploit the beam search decoding method to obtain system utterances with diverse structures and grammatical correctness. The experiments were conducted individually with respect to the word, morpheme, and syllable units, and the generated utterances were evaluated in both quantitative and qualitative ways. As a result, the morpheme-based model with the beam search decoding has achieved the most robust result of all. In fact, in the quantitative evaluation result of the generated sentence, the BLEU-4 score was 0.86 and the SER was 0.03, and the qualitative evaluation was also confirmed to be grammatically correct and contextually natural.
참고문헌 (30)
- 강상우, "대화 시스템을 위한 사용자 발화 문장의 감정 분류", 인지과학, 21(4), pp.459~480 (2010), 한국인지과학회[DOI]
- Dan Jurafsky, "Speech and language Processing. Vol 3" (2014), Pearson
- Yuxuan Wang, "Tactron : Towards End-to-End Speech Synthesis", proceedings of Interspeech, pp.4006~4010 (2017)
- Tsubasa Ochiai, "Multichannel End-to-end Speech Recognition", proceedings of the 34th International Conference on Machine Learning, 70, pp.2632~2641 (2017)
- 정민우, "Statistical Spoken Language Understanding for Human-Computer Dialog System", 한국차세대컴퓨팅학회 논문지, 4(1), pp.40~48 (2008), 한국차세대컴퓨팅학회
- 이형직, "효과적인 정답 추출을 위한 언어 분석 및 질문 패턴 기반의 질문 분할", 한국차세대컴퓨팅학회 논문지, 10(2), pp.31~38 (2014), 한국차세대컴퓨팅학회
- 이선정, "대용량 고립단어 언어처리를 위한 통계적 방안 연구", 한국차세대컴퓨팅학회 논문지, 9(1), pp.17~24 (2013), 한국차세대컴퓨팅학회
- Altinok Duygu, "An Ontology-Based Dialogue Management System for Banking and Finance Dialogue Systems", arXiv preprint arXiv : 1804. 04838, ~ (2018)
- Mrkšić, Nikola, "Fully statistical neural belief tracking", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2, pp.108~113 (2018)
- Xiang Gao, "Jointly Optimizing Diversity and Relevance in Neural Response Generation", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.1229~1238 (2019)
- Dongling Xiao, "ERNIE-GEN : An Enhanced Multi-Flow Pretraining and Fine-tuning Framework for Natural Language Generation", arXiv preprint arXiv : 2001. 11314, ~ (2020)
- Ye Jia, "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis", Advances in Neural Information Processing Systems, 31, pp.4480~4490 (2018)
- Yi Ren, "FastSpeech : Fast, Robust and Controllable Text to Speech", Advances in Neural Information Processing Systems, 32, pp.3171~3180 (2019)
- Sepp Hochreiter, "Long short-term memory", Neural Computation, 9(8), pp.1735~1780 (1997)
- Martin Sundermeyer, "LSTM neural networks for language modeling", Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, pp.194~197 (2012)
- Alex Graves, "Sequence transduction with recurrent neural networks", ICML Representation Learning Workshop (2012)
- Adam Cheyer, "Method and apparatus for building an intelligent automated assistant", U. S. Patent Application No 11, 2006
- Danilo Mirkovic, "Dialogue management using scripts", U.S. Patent No. 8,041,570, ~ (2011)
- Alice H. Oh, "Stochastic language generation for spoken dialogue systems", Proceedings of the 2000ANLP, 2000, pp.3, 27~32[DOI]
- Francois Mairesse, "Stochastic language generation in dialogue using factored language models", Computational Linguistics, 40(4), pp.763~799 (2014)
- Tsung-Hsien Wen, "Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking", Proceedings of SIGdial. Association for Computational Linguistics, pp.275~284 (2015)
- Tianyu Liu, "Towards Comprehensive Description Generation from Factual Attribute-value Tables", Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.5985~5996 (2019)
- Jason Williams, "The dialog state tracking challenge", Proceedings of the SIGdial, pp.404~413 (2013)
- Tomas Mikolov, "Recurrent neural network based language model", Proceedings of the 11th Annual Conference of the International Speech Communication Association, 2, pp.1045~1048 (2010)
- Sepp Hochreiter, "Gradient flow in recurrent nets: the difficulty of learning long-term dependencies", pp.237~244 (2000), IEEE Press
- Tsung-Hsien Wen, "Semantically conditioned lstm-based natural language generation for spoken dialogue systems", Proceedings of EMNLP, Association for Computational Linguistics, pp.1711~1721 (2015)
- Michael C. Mozer, "A focused backpropagation algorithm for temporal pattern recognition", Complex Systems, 3, pp.137~169 (1995)
- Kishore Papineni, "BLEU : a method for automatic evaluation of machine translation", Proceedings of the 40th annual meeting on association for computational linguistics, pp.311~318 (2002), Association for Computational Linguistics
- Amanda Stent, "Evaluating evaluation methods for generation in the presence of variation", Proceedings of the 6th International Conference on Computational Linguistics and Intelligent Text Processing, pp.341~351 (2005)
- Van-Khanh Tran, "Natural Language Generation for Spoken Dialogue System using RNN Encoder-Decoder Networks", Proceedings of CoNLL, pp.442~451 (2017)