단행본 서명의 단어 임베딩에 따른 자동분류의 성능 비교
Performance Comparison of Automatic Classification Using Word Embeddings of Book Titles
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
이 연구는 짧은 텍스트인 서명에 단어 임베딩이 미치는 영향을 분석하기 위해 Word2vec, GloVe, fastText 모형을 이용하여 단행본 서명을 임베딩 벡터로 생성하고, 이를 분류자질로 활용하여 자동분류에 적용하였다. 분류기는 k-최근접 이웃(kNN) 알고리즘을 사용하였고 자동분류의 범주는 도서관에서 도서에 부여한 DDC 300대 강목을 기준으로 하였다. 서명에 대한 단어 임베딩을 적용한 자동분류 실험 결과, Word2vec와 fastText의 Skip-gram 모형이 TF-IDF 자질보다 kNN 분류기의 자동분류 성능에서 더 우수한 결과를 보였다. 세 모형의 다양한 하이퍼파라미터 최적화 실험에서는 fastText의 Skip-gram 모형이 전반적으로 우수한 성능을 나타냈다. 특히, 이 모형의 하이퍼파라미터로는 계층적 소프트맥스와 더 큰 임베딩 차원을 사용할수록 성능이 향상되었다. 성능 측면에서 fastText는 n-gram 방식을 사용하여 하부문자열 또는 하위단어에 대한 임베딩을 생성할 수 있어 재현율을 높이는 것으로 나타났다. 반면에 Word2vec의 Skip-gram 모형은 주로 낮은 차원(크기 300)과 작은 네거티브 샘플링 크기(3이나 5)에서 우수한 성능을 보였다.
Abstract
To analyze the impact of word embedding on book titles, this study utilized word embedding models (Word2vec, GloVe, fastText) to generate embedding vectors from book titles. These vectors were then used as classification features for automatic classification. The classifier utilized the k-nearest neighbors (kNN) algorithm, with the categories for automatic classification based on the DDC (Dewey Decimal Classification) main class 300 assigned by libraries to books. In the automatic classification experiment applying word embeddings to book titles, the Skip-gram architectures of Word2vec and fastText showed better results in the automatic classification performance of the kNN classifier compared to the TF-IDF features. In the optimization of various hyperparameters across the three models, the Skip-gram architecture of the fastText model demonstrated overall good performance. Specifically, better performance was observed when using hierarchical softmax and larger embedding dimensions as hyperparameters in this model. From a performance perspective, fastText can generate embeddings for substrings or subwords using the n-gram method, which has been shown to increase recall. The Skip-gram architecture of the Word2vec model generally showed good performance at low dimensions(size 300) and with small sizes of negative sampling (3 or 5).
참고문헌 (21)
- 강형석, "Word2vec 및 fastText 임베딩 모델의 성능 비교", 디지털콘텐츠학회논문지, 21(7), pp.1335~1343 (2020), 한국디지털콘텐츠학회[DOI]
- 이다빈, "대용량 텍스트 자원을 활용한 한국어 형태소 임베딩의 모델별 성능 비교 분석", 정보과학회논문지, 46(5), pp.413~418 (2019), 한국정보과학회[DOI]
- 이용구, "목차 정보와 kNN 분류기를 이용한 사회과학 분야 도서 자동 분류에 관한 연구", 정보관리학회지, 37(1), pp.1~21 (2020), 한국정보관리학회[DOI]
- Bojanowski, P., "Enriching word vectors with subword information", Transactions of the Association for Computational Linguistics, 5, pp.135~146 (2017)[DOI]
- Chen, Q., "Specialists, scientists, and sentiments : Word2Vec and Doc2Vec in analysis of scientific and medical texts", SN Computer Science, 2(5), pp.414~ (2021)[DOI]
- Devlin, J., "BERT: pre-training of deep bidirectional transformers for language understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, 1, pp.4171~4186 (2019)
- Dharma, E. M., "The accuracy comparison among Word2vec, GloVe, and fastText towards convolution neural network(CNN)text classification", Journal of Theoretical and Applied Information Technology, 100(2), pp.349~359 (2022)[DOI]
- Goularas, D., "Evaluation of deep learning techniques in sentiment analysis from twitter data", 2019 International Conference on Deep Learning and Machine Learning in Emerging Applications(Deep-ML), pp.12~17 (2019)
- Harris, Z. S., "Distributional structure", Word, 10(2-3), pp.146~162 (1954)[DOI]
- Hinton, G. E., "Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume I", pp.77~109 (1986), Massachusetts Institute of Technology Press
- McCann, B., "Learned in translation: contextualized word vectors", Proceedings of the 31st International Conference on Neural Information Processing Systems, pp.6297~6308 (2017)
- "Efficient Estimation of Word Representations in Vector Space" (2013)[URL]
- Mikolov, T., "Distributed representations of words and phrases and their compositionality", Advances in Neural Information Processing Systems, 26 (2013)
- Park, S., "Subword-level word vector representations for Korean", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp.2429~2438 (2018)
- Pennington, J., "GloVe: global vectors for word representation", Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp.1532~1543 (2014)
- Peters, M. E., "Deep contextualized word representations", Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, 1, pp.2227~2237 (2018)
- Salton, G., "A vector space model for automatic indexing", Communications of the Association for Computing Machinery, 18(11), pp.613~620 (1975)[DOI]
- Sitender, Sangeeta, "Effect of GloVe, Word2Vec and fastText embedding on english and hindi neural machine translation systems", Proceedings of Data Analytics and Management 2022, pp.433~447 (2023)
- Wang, B., "Evaluating word embedding models: methods and experimental results", Asia Pacific Signal and Information Processing Association Transactions on Signal and Information Processing, 8, pp.e19~ (2019)[DOI]
- Wang, C., "A comparative study on word embeddings in deep learning for text classification", Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval, pp.37~46 (2020)
- Yang, X., "Using word embeddings in twitter election classification", Information Retrieval Journal, 21, pp.183~207 (2018)[DOI]