SSF: Sentence Similar Function Based on word2vector Similar Elements
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
In this paper, to improve the accuracy of long sentence similarity calculation, we proposed a sentence similaritycalculation method based on a system similarity function. The algorithm uses word2vector as the systemelements to calculate the sentence similarity. The higher accuracy of our algorithm is derived from twocharacteristics: one is the negative effect of penalty item, and the other is that sentence similar function (SSF)based on word2vector similar elements doesn’t satisfy the exchange rule. In later studies, we found the timecomplexity of our algorithm depends on the process of calculating similar elements, so we build an index ofpotentially similar elements when training the word vector process. Finally, the experimental results show thatour algorithm has higher accuracy than the word mover’s distance (WMD), and has the least query time ofthree calculation methods of SSF.
Abstract
In this paper, to improve the accuracy of long sentence similarity calculation, we proposed a sentence similaritycalculation method based on a system similarity function. The algorithm uses word2vector as the systemelements to calculate the sentence similarity. The higher accuracy of our algorithm is derived from twocharacteristics: one is the negative effect of penalty item, and the other is that sentence similar function (SSF)based on word2vector similar elements doesn’t satisfy the exchange rule. In later studies, we found the timecomplexity of our algorithm depends on the process of calculating similar elements, so we build an index ofpotentially similar elements when training the word vector process. Finally, the experimental results show thatour algorithm has higher accuracy than the word mover’s distance (WMD), and has the least query time ofthree calculation methods of SSF.
참고문헌 (24)
- M. U. Devi, "Query expansion on the role of word and sentence similarity for domain ontology driven fuzzy retrieval systems", Journal of Computational and Theoretical Nanoscience, 14(6), pp.2612~2619 (2017)
- "Comparative study of CNN and RNN for natural language processing" (2017)[URL]
- D. Zhang, "A carpooling recommendation system for taxicab services", IEEE Transactions on Emerging Topics in Computing, 2(3), pp.254~266 (2014)
- J. R. Lin, "A natural‐language‐based approach to intelligent data retrieval and representation for cloud BIM", Computer‐Aided Civil and Infrastructure Engineering, 31(1), pp.18~33 (2016)
- "Neural paraphrase generation with stacked residual LSTM networks" (2016)[URL]
- M. A. Boudia, "Hybridization between scoring technique and similarity technique for automatic summarization by extraction", International Journal of Organizational and Collective Intelligence, 6(1), pp.1~14 (2016)
- P. W. McBurney, "Automatic source code summarization of context for Java methods", IEEE Transactions on Software Engineering, 42(2), pp.103~119 (2015)
- "Automatic keyword extraction for text summarization: a survey" (2017)[URL]
- Y. C. Lee, "An ontology-based approach for developing data exchange requirements and model views of building information modeling", Advanced Engineering Informatics, 30(3), pp.354~367 (2016)
- J. Muralikumar, "A statistical approach for modeling inter-document semantic relationships in digital libraries", Journal of Intelligent Information Systems, 48(3), pp.477~498 (2017)
- G. Zhou, "An empirical study of topic-sensitive probabilistic model for expert finding in question answer communities", Knowledge-Based Systems, 66, pp.136~145 (2014)
- S. Guo, "Sentence similarity calculation based on word vector and its application research", Modern Electronics Technique, 39(13), pp.99~102 (2016)
- F. Li, "Research on multi-feature sentence similarity computing method with word embedding", Journal of Frontiers of Computer Science and Technology, 11(4), pp.608~618 (2017)
- S. Arora, "A simple but tough-to-beat baseline for sentence embeddings", Proceedings of the 5th International Conference on Learning Representations (ICLR) (2017)
- Y. Mrabet, "TextFlow: a text similarity measure based on continuous sequences", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp.763~772 (2017)
- Research and implementation of paraphrasing recognition technology for question-and-answer system, "S. Xu", Harbin Institute of Technology, ~ (2009)
- M. Kusner, "From word embeddings to document distances", Proceedings of the 32nd International Conference on Machine Learning, pp.957~966 (2015)
- Y. Guan, "A new measurement of systematic similarity", IEEE Transactions on Systems, Man, and Cybernetics-Part A : Systems and Humans, 38(4), pp.743~758 (2008)
- Y. Guan, "Measurement of system similarity", Proceedings of China Computational Linguistics Conference (CCL), pp.341~347 (2005)
- "Wikimedia Chinese corpus"[URL]
- "Word2VEC_java"[URL]
- "wmd4j is a Java library for calculating Word Mover's Distance (WMD)"[URL]
- "Word Sentence Similarity Code"[URL]
- "Word2VEC"[URL]