본문 바로가기
KERD
논문KCI 등재

Combining Distributed Word Representation and Document Distance for Short Text Document Clustering

2

조회

0

다운로드

1

인용

0

참고문헌

키워드

초록

This paper presents a method for clustering short text documents, such as news headlines, social media statuses,or instant messages. Due to the characteristics of these documents, which are usually short and sparse, anappropriate technique is required to discover hidden knowledge. The objective of this paper is to identify thecombination of document representation, document distance, and document clustering that yields the bestclustering quality. Document representations are expanded by external knowledge sources represented by aDistributed Representation. To cluster documents, a Kmeans partitioningbased clustering technique is applied,where the similarities of documents are measured by word mover’s distance. To validate the effectiveness ofthe proposed method, experiments were conducted to compare the clustering quality against several leadingmethods. The proposed method produced clusters of documents that resulted in higher precision, recall, F1score, and adjusted Rand index for both realworld and standard data sets. Furthermore, manual inspection ofthe clustering results was conducted to observe the efficacy of the proposed method. The topics of eachdocument cluster are undoubtedly reflected by members in the cluster.

Abstract

This paper presents a method for clustering short text documents, such as news headlines, social media statuses,or instant messages. Due to the characteristics of these documents, which are usually short and sparse, anappropriate technique is required to discover hidden knowledge. The objective of this paper is to identify thecombination of document representation, document distance, and document clustering that yields the bestclustering quality. Document representations are expanded by external knowledge sources represented by aDistributed Representation. To cluster documents, a Kmeans partitioningbased clustering technique is applied,where the similarities of documents are measured by word mover’s distance. To validate the effectiveness ofthe proposed method, experiments were conducted to compare the clustering quality against several leadingmethods. The proposed method produced clusters of documents that resulted in higher precision, recall, F1score, and adjusted Rand index for both realworld and standard data sets. Furthermore, manual inspection ofthe clustering results was conducted to observe the efficacy of the proposed method. The topics of eachdocument cluster are undoubtedly reflected by members in the cluster.

참고문헌 (78)

  1. V. K. R. Sridhar, "Unsupervised topic modeling for short texts using distributed representations of words", Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pp.192~200 (2015)
  2. "A brief survey of text mining: classification, clustering and extraction techniques" (2017)[URL]
  3. V. Gupta, "A survey of text mining techniques and applications", Journal of Emerging Technologies in Web Intelligence, 1(1), pp.60~76 (2009)
  4. A. M. Jadhav, "A survey on text mining and its techniques", International Journal of Science and Research, 3(11), pp.2110~2113 (2014)
  5. C. C. Aggarwal, "Mining text and social streams : a review", ACM SIGKDD Explorations Newsletter, 15(2), pp.9~19 (2014)
  6. L. F. S. Coletta, "Combining classification and clustering for tweet sentiment analysis", Proceedings of 2014 Brazilian Conference on Intelligent Systems, pp.210~215 (2014)
  7. S. Sharma, "Recent developments in text clustering techniques", International Journal of Computer Applications, 37(6), pp.14~19 (2012)
  8. L. Rokach, "Clustering Methods" (2005), Springer
  9. C. C. Aggarwal, "Mining Text Data", pp.77~128 (2012), Springer
  10. K. S. Jones, "A statistical interpretation of term specificity and its application in retrieval", Journal of Documentation, 28(1), pp.11~21 (1972)
  11. G. Salton, "McGill, Introduction to Modern Information Retrieval" (1986), McGraw-Hill Inc
  12. J. Ramos, "Using tf-idf to determine word relevance in document queries", Proceedings of the 1st Instructional Conference on Machine Learning, pp.133~142 (2003)
  13. J. Xu, "Short text clustering via convolutional neural networks", Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing, pp.62~69 (2015)
  14. "A convolutional neural network for modelling sentences" (2014)[URL]
  15. J. Xu, "Self-taught convolutional neural networks for short text clustering", Neural Networks, 88, pp.22~31 (2017)
  16. C. Ma, "Short text classification based on distributional representations of words", IEICE Transactions on Information and Systems, 99(10), pp.2562~2565 (2016)
  17. Y. Yan, "Web and Big Data", pp.626~641 (2017), Springer
  18. L. Hong, "Empirical study of topic modeling in twitter", Proceedings of the 1st Workshop on Social Media Analytics, pp.80~88 (2010)
  19. J. Weng, "Twitterrank : finding topic-sensitive influential twitterers", Proceedings of the 3rd ACM International Conference on Web Search and Data Mining, pp.261~270 (2010)
  20. R. Mehrotra, "Improving LDA topic models for microblogs via tweet pooling and automatic labeling", Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.889~892 (2013)
  21. X. Quan, "Short and sparse text topic modeling via self-aggregation", Proceedings of the 24th International Conference on Artificial Intelligence, pp.2270~2276 (2015)
  22. T. Mikolov, "Distributed representations of words and phrases and their compositionality", Advances in Neural Information Processing Systems, 26, pp.3111~3119 (2013)
  23. M. Steinbach, "A comparison of document clustering techniques", Proceedings of the International KDD Workshop on Text Mining (2000)
  24. P. B. Nagpal, "Comparative study of density based clustering algorithms", International Journal of Computer Applications, 27(11), pp.421~435 (2011)
  25. G. Karypis, "Chameleon : hierarchical clustering using dynamic modeling", Computer, 32(8), pp.68~75 (1999)
  26. K. Mumtaz, "A novel density based improved k-means clustering algorithm-Dbkmeans", International Journal on Computer Science and Engineering, 2(2), pp.213~218 (2010)
  27. A. Karami, "Choosing DBSCAN parameters automatically using differential evolution", International Journal of Computer Applications, 91(7), pp.1~11 (2014)
  28. W. H. Gomaa, "A survey of text similarity approaches", International Journal of Computer Applications, 68(13), pp.13~18 (2013)
  29. M. Kusner, "From word embeddings to document distances", Proceedings of the 32nd International Conference on Machine Learning, pp.957~966 (2015)
  30. J. MacQueen, "Some methods for classification and analysis of multivariate observations", Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, pp.281~297 (1967)
  31. G. Salton, "Term-weighting approaches in automatic text retrieval", Information Processing and Management, 24(5), pp.513~523 (1988)
  32. "Efficient estimation of word representations in vector space" (2013)[URL]
  33. C. De Boom, "Learning semantic similarity for very short texts", Proceedings of 2015 IEEE International Conference on Data Mining Workshop(ICDMW), pp.1229~1234 (2015)
  34. S. Deerwester, "Indexing by latent semantic analysis", Journal of the American Society for Information Science, 41(6), pp.391~407 (1990)
  35. T. Hofmann, "Probabilistic latent semantic indexing", Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp.50~57 (1999)
  36. D. M. Blei, "Latent Dirichlet allocation", Journal of Machine Learning Research, 3, pp.993~1022 (2003)
  37. D. E. Rumelhart, "Parallel Distributed Processing: Explorations in the Microstructure of Cognition (Volume 1: Foundations)" (1986), MIT Press
  38. J. L. Elman, "Distributed representations, simple recurrent networks, and grammatical structure", Machine Learning, 7(2-3), pp.195~225 (1991)
  39. T. Kenter, "Short text similarity with word embeddings", Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp.1411~1420 (2015)
  40. J. Qiang, "Advances in Knowledge Discovery and Data Mining", pp.363~374 (2017), Springer International Publishing
  41. A. Karandikar, "Clustering short status messages: a topic model based approach", M. S., pp.2010
  42. A. Barron-Cedeno, "Plagiarism detection across distant language pairs", Proceedings of the 23rd International Conference on Computational Linguistics, pp.37~45 (2010)
  43. A. Huang, "Similarity measures for text document clustering", Proceedings of the Sixth New Zealand Computer Science Research Student Conference(NZCSRSC), pp.49~56 (2008)
  44. Y. Rubner, "A metric for distributions with applications to image databases", Proceedings of the 6th International Conference on Computer Vision (IEEE Cat. No. 98CH36271),, pp.59~66 (1998)
  45. Y. Rubner, "The earth mover’s distance as a metric for image retrieval", International Journal of Computer Vision, 40(2), pp.99~121 (2000)
  46. J. A. Hartigan, "Clustering Algorithms" (1975), John Wiley & Sons Inc
  47. P. N. Tan, "Kumar, Introduction to Data Mining" (2006), Pearson Education Inc
  48. J. Soler, "Data clustering and similarity", Proceedings of the 26th International Florida Artificial Intelligence Research Society Conference (FLAIRS’13),, pp.492~495 (2013)
  49. M. Steinbach, "A comparison of document clustering techniques", Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2000)
  50. D. A. Ingaramo, "Density-based clustering of short-text corpora", Procesamiento del Lenguaje Natural, 41, pp.81~87 (2008)
  51. A. Rangrej, "Comparative study of clustering techniques for short text documents", Proceedings of the 20th International Conference Companion on World Wide Web, pp.111~112 (2011)
  52. N. Singh, "A novel clustering technique for short texts", Proceedings of 2016 5th International Conference on Reliability, pp.228~232 (2016), Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO)
  53. "An overview on clustering methods" (2012)[URL]
  54. H. Singh, "Clustering of text documents by implementation of k-means algorithms", Streamed Info-Ocean, 1, pp.53~63 (2016)
  55. Y. Chen, "Density-based clustering for real-time stream data", Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.133~142 (2007)
  56. A. K. Jain, "Dubes, Algorithms for Clustering Data" (1988), Prentice-Hall Inc
  57. M. Ester, "A density-based algorithm for discovering clusters a densitybased algorithm for discovering clusters in large spatial databases with noise", Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, pp.226~231 (1996)
  58. Z. Miller, "Twitter spammer detection using data stream clustering", Information Sciences, 260, pp.64~73 (2014)
  59. E. K. Ikonomakisa, "Recent Progress in Computational Sciences and Engineering", pp.218~221 (2006), Taylor & Francis
  60. S. Yang, "Density-based clustering of massive short messages using domain ontology", Proceedings of 2009 Asia-Pacific Conference on Information Processing, pp.505~508 (2009)
  61. M. T. H. Elbatta, "A dynamic method for discovering density varied clusters", International Journal of Signal Processing, Image Processing and Pattern Recognition, 6(1), pp.123~134 (2013)
  62. M. Parimala, "A survey on density based clustering algorithms for mining large spatial databases", International Journal of Advanced Science and Technology, 31, pp.59~66 (2011)
  63. K. Sawant, "Adaptive methods for determining DBSCAN parameters", International Journal of Innovative Science, Engineering & Technology, 1(4), pp.329~334 (2014)
  64. A. K. Pujari, "Data Mining Techniques" (2001), Universities Press(India)Private Limited
  65. V. K. Singh, "Document clustering using k-means, heuristic k-means and fuzzy cmeans", Proceedings of the 2011 International Conference on Computational Intelligence and Communication Networks, pp.297~301 (2011)
  66. S. C. Punitha, "A comparative study to find a suitable method for text document clustering", International Journal of Computer Science & Information Technology(IJCSIT), 3(6), pp.49~59 (2011)
  67. S. T. Deokar, "Text documents clustering using k means algorithm", International Journal of Technology &Engineering Science(IJTES), 1(4), pp.282~286 (2013)
  68. R. Rehurek, "Software framework for topic modelling with large corpora", Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp.45~50 (2010)
  69. D. Sailaja, "An overview of pre-processing text clustering methods", International Journal of Computer Science & Information Technologies, 6(3), pp.3119~3124 (2015)
  70. A. I. Kadhim, "Text document preprocessing and dimension reduction techniques for text document clustering", Proceedings of the 2014 4th International Conference on Artificial Intelligence with Applications in Engineering and Technology, pp.69~73 (2014)
  71. Y. Song, "Unsupervised sparse vector densification for short text similarity", Proceedings of 2015 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies, pp.1275~1280 (2015)
  72. X. H. Phan, "Learning to classify short and sparse text & web with hidden topics from large-scale data collections", Proceedings of the 17th International Conference on World Wide Web, pp.91~100 (2008)
  73. P. Shrestha, "Corpus-based methods for short text similarity", Proceedings of the 17th Rencontre des Étudiants Chercheurs en Informatique pour le Traitement Automatique des Langues (2011)
  74. D. Greene, "Practical solutions to the problem of diagonal dominance in kernel document clustering", Proceedings of the 23rd International Conference on Machine Learning, pp.377~384 (2006)
  75. S. Vijayarani, "Preprocessing techniques for text mining-an overview", International Journal of Computer Science & Communication Networks, 5(1), pp.7~16 (2015)
  76. S. Seifzadeh, "Short-text clustering using statistical semantics", Proceedings of the 24th International Conference on World Wide Web, pp.805~810 (2015)
  77. M. Speriosu, "Twitter polarity classification with label propagation over lexical links and the follower graph", Proceedings of the 1st Workshop on Unsupervised Learning in NLP, pp.53~63 (2011)
  78. S. Baillargeon, "Stream clustering of tweets", Proceedings of 2016 IEEE, 2016, 1256~1261

관련 논문AI추천

이 간행물의 다른 논문 (18건)