계층적 디리클레 프로세스를 활용한미지원 도메인 검출
Out-Of-Domain Detection Using Hierarchical Dirichlet Process
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
With improvement of speech recognition and natural language processing, dialog systems are recently adapted to various service domains. It became possible to get desirable services by conversation through the dialog system, but it is still necessary to improve separate modules, such as domain detection, intention detection, named entity recognition, and out-of-domain detection, in order to achieve stable service offer. When it misclassifies an in-domain sentence of conversation as out-of-domain, it will result in poor customer satisfaction and finally lost business. As there have been relatively small number of studies related to the out-of-domain detection, in this paper, we introduce a new method using a hierarchical Dirichlet process and demonstrate the effectiveness of it by experimental results on Korean dataset.
Abstract
With improvement of speech recognition and natural language processing, dialog systems are recently adapted to various service domains. It became possible to get desirable services by conversation through the dialog system, but it is still necessary to improve separate modules, such as domain detection, intention detection, named entity recognition, and out-of-domain detection, in order to achieve stable service offer. When it misclassifies an in-domain sentence of conversation as out-of-domain, it will result in poor customer satisfaction and finally lost business. As there have been relatively small number of studies related to the out-of-domain detection, in this paper, we introduce a new method using a hierarchical Dirichlet process and demonstrate the effectiveness of it by experimental results on Korean dataset.
참고문헌 (31)
- "Amazon Echo"[URL]
- "SK Telecom Nugu"[URL]
- "Naver WAVE"[URL]
- "Naver AWAY IVI platform"[URL]
- "Naver search"[URL]
- "Google search"[URL]
- Lidia Mangu., "Lattice Compression in the Consensual Post-Processing Framework", Proceedings of the Third World Multiconference on Systemics, Cybernetics and Informatics, pp.246~252 (2000)
- Ian R. Lane, "Out-Of-Domain Detection Based on Confidence Measures From Multiple Topic Classificati on", Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pp.757~760 (2004)
- Seonghan Ryu., "Exploiting Out-Of-Vocabulary Words For Out-Of-Domain Detection in Dialog Systems", Proceedings of the International Conference on Big Data and Smart Computing, pp.165~168 (2014)
- Deirdre Hogan, "SMS Normalisation, Retrieval and Out-of-Domain Detection Approaches for SMS-Based FAQ Retrieval", Multilingual Information Access in South Asian Languages, 184~196 (2013)
- Mikio Nakano, "A Two-Stage Domain Selection Framework for Extensible Multi-Domain Spoken Dialogue Systems", Proceedings of the 12th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp.18~29 (2011)
- Yoko Fujita, "Out-of-Task Utterance Detection Based on Bag-of-Words Using Automatic Speech Recognition Results", Proceedings of the third Annual Summit and Conference of Asia-Pacific Signal and Information Processing Association (2011)
- "Wordnet"[URL]
- Young-Seob Jeong, "Experimental Analysis for Out-Of-Domain Detection Using Features of Word Positions in Sentences", Proceedings of the Spring Conference of Korean Society for Internet Information (2017)
- Trevor Hastie, "The elements of statistical learning" (2003), Springer
- Scott Deerwester, "Indexing by Latent Semantic Analysis", Journal of the American Society for Information Science (JASIS), 41(6), pp.391~407 (1990)[DOI]
- Bernhard E. Boser, "A Training Algorithm For Optimal Margin Classifiers", Proceedings of the fifth Annual Workshop on Computational Learning Theory, pp.144~152 (1992)
- David A. Freedman, "Statistical Models: Theory and Practice" (2009)
- Ian R. Lane, "Example-Based Training of Dialogue Planning Incorporating User and Situation Models", Proceedings of the 8th International Conference on Spoken Language Processing (2004)
- Adam L. Berger, "A Maximum Entropy Approach to Natural Language Processing", Computational Linguistics, 22(1), pp.39~71 (1996)
- Seonghan Ryu, "Neural Sentence Embedding Using Only In-Domain Sentences For Out-Of-Domain Sentence Detection in Dialog Systems", Pattern Recognition Letters, 88, pp.26~32 (2017)[DOI]
- Yee Whye Teh, "Hierarchical Dirichlet Processes", Journal of the American Statistical Association, 101(476), pp.1566~1581 (2006)[DOI]
- David M. Blei, "Latent Dirichlet Allocation", Journal of Machine Learning Research, 3, pp.993~1022 (2003)
- Machal Rosen-Zvi, "Learning Author-Topic Models", ACM Transactions on Information Systems, 28(1), pp.1~38 (2010)
- Young-Seob Jeong., "Sequential Entity Group Topic Model for Getting Topic Flows of Entity Groups Within One Document", Proceedings of the 16th Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp.366~378 (2012)
- Thomas L. Griffiths., "Finding Scientific Topics", Proceedings of the National Academy of Sciences of the United States of America, pp.5228~5235 (2004)
- Leo Breiman, "Random Forests", Machine Learning, 45(1), pp.5~32 (2001)[DOI]
- Ian R. Lane., "Incorporating Dialogue Context and Topic Clustering in Out-of-Domain Detection", Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp.1045~1048 (2005)
- John C. Platt, "Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines;Advances in Kernel Methods – Support Vector Learning" (1998), MIT Press
- Sepp Hochreiter., "Long Short-term Memory", Neural Computation, 9(8), pp.1735~1780 (1997)[DOI]