Automatic detection of similar questions from QA database using predicate-argument structures and answer sentences
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This work deals with duplicated questions in a Question and Answer (QA) database. A large volume of QA databases can be easily built from Community Question Answering (CQA) services. Questions and their corresponding answers collected from the Internet tend to be duplicated. In this work, we propose two approaches for detecting similar questions in a QA database. One is by using predicate-argument structures. Here, questions are converted to predicate-argument structures modified for an interrogative sentence. We can detect similar questions by comparing their predicate-argument structures. The other approach is to exploit answer sentences. When two questions with different predicate-argument structures have duplicated answers, these two predicate-argument structures are interchangeable in terms of detecting similar questions. We build the QA database on a computer technology domain, which consists of 6,319 QA pairs. We evaluate our module on this QA database, and obtain a 91.69 F1 accuracy in detecting sets of similar questions.
Abstract
This work deals with duplicated questions in a Question and Answer (QA) database. A large volume of QA databases can be easily built from Community Question Answering (CQA) services. Questions and their corresponding answers collected from the Internet tend to be duplicated. In this work, we propose two approaches for detecting similar questions in a QA database. One is by using predicate-argument structures. Here, questions are converted to predicate-argument structures modified for an interrogative sentence. We can detect similar questions by comparing their predicate-argument structures. The other approach is to exploit answer sentences. When two questions with different predicate-argument structures have duplicated answers, these two predicate-argument structures are interchangeable in terms of detecting similar questions. We build the QA database on a computer technology domain, which consists of 6,319 QA pairs. We evaluate our module on this QA database, and obtain a 91.69 F1 accuracy in detecting sets of similar questions.
참고문헌 (13)
- L. Nguyen-Thinh, "Advanced Computational Methods for Knowledge Engineering", pp.325~338 (2014), Springer
- J. F. Aquino, "Text2Test: Question generator utilizing information abstraction techniques and question generation methods for narrative and declarative text", Proceedings of the 8th National Natural Language Processing Research Symposium, pp.29~34 (2011)
- Z. Yan, "DocChat: An Information Retrieval Approach for Chatbot Engines Using Unstructured Documents", Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp.516~525 (2016)
- J. Zongcheng, "An information retrieval approach to short text conversation", ~ (2014)
- J. Allan, "Retrieval and novelty detection at the sentence level", Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, pp.314~321 (2003)
- T. C. Hoad, "Methods for identifying versioned and plagiarized documents", Journal of the American Society for Information Science and Technology, 54(3), pp.203~215 (2003)[DOI]
- D. Jurafsky, "Speech & Language Processing" (2000), Pearson Education India
- W. N. Zhang, "A topic clustering approach to finding similar questions from large question and answer archives", PloS one, 9(3) (2014)
- K. Wang, "A syntactic tree matching approach to finding similar questions in community-based qa services", Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pp.187~194 (2009), ACM
- H. Duan, "Searching questions by identifying question topic and question focus", Proceedings of ACL-08: HLT, pp.156~164 (2008)
- J. Jeon, "Finding similar questions in large question and answer archives", Proceedings of the 14th ACM international conference on Information and knowledge management, pp.84~90 (2005), ACM
- D. Klein, "Accurate unlexicalized parsing", Proceedings of the 41st annual meeting of the association for computational linguistics, pp.423~430 (2003)
- R. Krestel, "Predicate-argument extractor (pax)", New Challenges For NLP Frameworks Programme, pp.51~54 (2010)