Research on Keyword-Overlap Similarity Algorithm Optimization in Short English Text Based on Lexical Chunk Theory
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Short-text similarity calculation is one of the hot issues in natural language processing research. The conventionalkeyword-overlap similarity algorithms merely consider the lexical item information and neglect theeffect of the word order. And some of its optimized algorithms combine the word order, but the weights arehard to be determined. In the paper, viewing the keyword-overlap similarity algorithm, the short English textsimilarity algorithm based on lexical chunk theory (LC-SETSA) is proposed, which introduces the lexicalchunk theory existing in cognitive psychology category into the short English text similarity calculation for thefirst time. The lexical chunks are applied to segment short English texts, and the segmentation results demonstratethe semantic connotation and the fixed word order of the lexical chunks, and then the overlap similarityof the lexical chunks is calculated accordingly. Finally, the comparative experiments are carried out, and theexperimental results prove that the proposed algorithm of the paper is feasible, stable, and effective to a largeextent.
Abstract
Short-text similarity calculation is one of the hot issues in natural language processing research. The conventionalkeyword-overlap similarity algorithms merely consider the lexical item information and neglect theeffect of the word order. And some of its optimized algorithms combine the word order, but the weights arehard to be determined. In the paper, viewing the keyword-overlap similarity algorithm, the short English textsimilarity algorithm based on lexical chunk theory (LC-SETSA) is proposed, which introduces the lexicalchunk theory existing in cognitive psychology category into the short English text similarity calculation for thefirst time. The lexical chunks are applied to segment short English texts, and the segmentation results demonstratethe semantic connotation and the fixed word order of the lexical chunks, and then the overlap similarityof the lexical chunks is calculated accordingly. Finally, the comparative experiments are carried out, and theexperimental results prove that the proposed algorithm of the paper is feasible, stable, and effective to a largeextent.
참고문헌 (18)
- C. Banea, "UNT: a supervised synergistic approach to semantic text similarity", Proceedings of the 6th International Workshop on Semantic Evaluation (SemEval), pp.635~642 (2012)
- Huaizhong Liang, "Short Text Similarity Hybrid Algorithm for a Chinese Medical Intelligent Question Answering System", Technology-Inspired Smart Learning for Future Education, pp.129~142 (2020), Springer Singapore[DOI]
- Sneha Banerjee, "Quote examiner: verifying quoted images using web-based text similarity", Multimedia Tools and Applications, 80(8), pp.12135~12154 (2021), Springer Science and Business Media LLC[DOI]
- Yunmei Liu, "Applying text similarity algorithm to analyze the triangular citation behavior of scientists", Applied Soft Computing, 107, pp.107362~ (2021), Elsevier BV[DOI]
- X. Lin, "Short-text Classification Method Based on Concept Network", Computer Engineering, 36(21), pp.4~6 (2010)[DOI]
- C. Jin, "Chinese short text clustering based on dynamic vector", Computer Engineering and Applications, 47(33), pp.156~158 (2011)
- X. Q. Zhao, "Application of concept tree in semantic similarity of short texts", Computer Technology and Development, 22(6), pp.159~162 (2012)
- J. Yin, "Model-based clustering of short text streams", Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.2634~2642 (2018)
- T. Schick, "Automatically identifying words that can serve as labels for few-shot text classification", Proceedings of the 28th International Conference on Computational Linguistics, pp.5569~5578 (2020)
- J. W. Sun, "Short text classification based on semantics and maximum matching degree", Computer Engineering and Designing, 34(10), pp.3613~3618 (2013)
- Hien T. Nguyen, "Learning short-text semantic similarity with word embeddings and external knowledge sources", Knowledge-Based Systems, 182, pp.104842~ (2019), Elsevier BV[DOI]
- Goutam Majumder, "Interpretable semantic textual similarity of sentences using alignment of chunks with classification and regression", Applied Intelligence, 51(10), pp.7322~7349 (2021), Springer Science and Business Media LLC[DOI]
- "Text classification based on multi-granularity attention hybrid neural network" (2020)[URL]
- Pu Huang, "Attention‐aware fully convolutional neural network with convolutional long short‐term memory network for ultrasound‐based motion tracking", Medical Physics, 46(5), pp.2275~2285 (2019), Wiley[DOI]
- N. Peinelt, "tBERT: topic models and BERT joining forces for semantic similarity detection", Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.7047~7055 (2020)
- R. Zhang, "A new measure of semantic similarity between unknown Chinese words based on HowNet", Journal of Chinese Information Processing, 26(1), pp.16~21 (2012)
- J. D. Becker, "The phrasal lexicon", Proceedings of the 1975 Workshop on Theoretical Issues in Natural Language Processing, pp.60~63 (1975)
- J. R. Nattinger, "Lexical Phrases and Language Teaching" (1992), Oxford University Press