Investigation on the Effect of Multi-Vector Document Embedding for Interdisciplinary Knowledge Representation
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Text is the most widely used means of exchanging or expressing knowledge and information in the real world. Recently, researches on structuring unstructured text data for text analysis have been actively performed. One of the most representative document embedding method (i.e. doc2Vec) generates a single vector for each document using the whole corpus included in the document. This causes a limitation that the document vector is affected by not only core words but also other miscellaneous words. Additionally, the traditional document embedding algorithms map each document into only one vector. Therefore, it is not easy to represent a complex document with interdisciplinary subjects into a single vector properly by the traditional approach. In this paper, we introduce a multivector document embedding method to overcome these limitations of the traditional document embedding methods. After introducing the previous study on multi-vector document embedding, we visually analyze the effects of the multi-vector document embedding method. Firstly, the new method vectorizes the document using only predefined keywords instead of the entire words. Secondly, the new method decomposes various subjects included in the document and generates multiple vectors for each document. The experiments for about three thousands of academic papers revealed that the single vector-based traditional approach cannot properly map complex documents because of interference among subjects in each vector. With the multi-vector based method, we ascertained that the information and knowledge in complex documents can be represented more accurately by eliminating the interference among subjects.
Abstract
Text is the most widely used means of exchanging or expressing knowledge and information in the real world. Recently, researches on structuring unstructured text data for text analysis have been actively performed. One of the most representative document embedding method (i.e. doc2Vec) generates a single vector for each document using the whole corpus included in the document. This causes a limitation that the document vector is affected by not only core words but also other miscellaneous words. Additionally, the traditional document embedding algorithms map each document into only one vector. Therefore, it is not easy to represent a complex document with interdisciplinary subjects into a single vector properly by the traditional approach. In this paper, we introduce a multivector document embedding method to overcome these limitations of the traditional document embedding methods. After introducing the previous study on multi-vector document embedding, we visually analyze the effects of the multi-vector document embedding method. Firstly, the new method vectorizes the document using only predefined keywords instead of the entire words. Secondly, the new method decomposes various subjects included in the document and generates multiple vectors for each document. The experiments for about three thousands of academic papers revealed that the single vector-based traditional approach cannot properly map complex documents because of interference among subjects in each vector. With the multi-vector based method, we ascertained that the information and knowledge in complex documents can be represented more accurately by eliminating the interference among subjects.
참고문헌 (22)
- 김도우, "Doc2Vec과 Word2Vec을 활용한 Convolutional Neural Network 기반 한국어 신문 기사 분류", 정보과학회논문지, 44(7), pp.742~747 (2017), 한국정보과학회
- 박종인, "복합 문서의 의미적 분해를 통한 다중 벡터 문서 임베딩 방법론", 지능정보연구, 25(3), pp.19~41 (2019), 한국지능정보시스템학회
- 송서하, "금융 데이터 및 텍스트 데이터를 활용한 금융 기업 조기 경보 모형 개발 : 부실은행 예측을 중심으로", 대한산업공학회지, 45(3), pp.248~259 (2019), 대한산업공학회[DOI]
- Bengio, Y, "Review: Knowledge Management and Knowledge Management Systems: Conceptual Foundations and Research Issues", MIS Quarterly, 25(1), pp.107~136 (2001)
- Aggarwal, C. C, "Mining Text Data" (2012), Springer
- Bengio, Y, "A Neural Probabilistic Language Model", The Journal of M achine Learning Research, (3), pp.1137~1155 (2003)
- Firth, J. R, "Studies in Linguistic Analysis", pp.1~32 (1957)
- Hinton, G. E, "Learning Distributed Representations of Concepts", Proceedings of the 8th Annual Conference of the Cognitive Science Society, (1), pp.1~12 (1986)
- Hotho, A, "A Brief Survey of Text Mining", LDV-Forum, 20(1), pp.19~62 (2005)
- Kenter, T, "Short Text Similarity with Word Embedding", Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp.1411~1420 (2015)
- Kim, Y, "Convolutional Neural Networks for Sentence Classification", Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP), pp.1746~1751 (2014)
- Kiros, R, "Skip-Thought Vectors", Proceedings of the 28th International Conference on Neural Information Processing Systems, (2), pp.3294~3302 (2015)
- Lai, S, "Recurrent Convolutional Neural Network for Text Classification", Proceedings of the 29th AAAI Conference on Artificial Intelligence, pp.2267~2273 (2015)
- Liu, J, "Deep Learning for Extreme Multi-label Text Classification", Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.115~124 (2017)
- Mikolov, T, "Strategies for Training Large Scale Neural Network Language Models", 2011 IEEE Workshop on Automatic Speech Recognition & Understanding, pp.196~201 (2011)
- Mikolov, T, "Distributed representations of words and phrases and their compositionality", Proceedings of the 26th International Conference on Neural Information Processing Systems, (2), pp.3111~3119 (2013)
- Quoc, L, "Distributed Representations of Sentences and Documents", Proceedings of t he 31st Inter nat iona l Conference on Machine Learning, (32), pp.1188~1196 (2014)
- Salton, G, "A Vector Space Model for Automatic Indexing", Communications of the ACM, 18(11), pp.613~620 (1975)
- Tan, A, "Text Mining: The State of the Art and the Challenges", Proceedings of the Pacific Asia Conference on Knowledge Discovery and Data Mining PAKDD’99 workshop on Knowledge Discovery from Advanced Databases, pp.65~70 (1999)
- Turian, J, "Word Representations: A Simple and General Method for Semi-Supervised Learning", Proceedings of the 48th Annual Meeting of t he A ssociat ion for Computat iona l Linguistics, pp.384~394 (2010)
- 유홍연, "실시간 뉴스 기반의 이슈 분석을 위한 점증적 군집화 및 다중 문서 요약", 정보과학회논문지, 46(4), pp.355~362 (2019), 한국정보과학회[DOI]
- Witten, I. H, "Text Mining, Practical Handbook of Internet Computing" (2004), CRC Press