Refined Word Embeddings with Intensity Awareness for Fine-Level Sentiment Classification
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Sentiment analysis employs classification models to discern people's opinions automatically. Recent strides made with Large Language Models (LLMs) have significantly enhanced the accuracy of binary-level sentiment classification, particularly through zero-shot and few-shot learning approaches. However, when it comes to fine-level sentiment classification, LLMs face challenges because they are not specifically trained for this downstream task. In contrast, other classification models utilize word embedding, a vector representation of words, as input data. Contemporary word embedding algorithms create these embeddings by considering the surrounding context of each word. Nonetheless, these word embeddings often fail to capture the nuances of intensity differences between words. For example, words like 'more' and 'less' have embeddings closely positioned in the semantic space, despite representing distinct intensity levels. These intensity words such as 'much', 'more', and 'less' are frequently used to convey the strength of opinions. Their intensity distinctions are crucial in fine-level sentiment classification. This paper introduces an innovative intensity-aware feed-forward neural network, equipped with a novel referential loss function designed to capture these intensity differences between words. The proposed model effectively separates words of varying intensities while bringing together words sharing the same intensity in the semantic space. To assess the effectiveness of this refined word embedding in sentiment analysis tasks, diverse fine-level sentiment datasets are employed. The results demonstrate that the refined word embedding surpasses original embeddings and popular LLMs for fine-level sentiment analysis.
Abstract
Sentiment analysis employs classification models to discern people's opinions automatically. Recent strides made with Large Language Models (LLMs) have significantly enhanced the accuracy of binary-level sentiment classification, particularly through zero-shot and few-shot learning approaches. However, when it comes to fine-level sentiment classification, LLMs face challenges because they are not specifically trained for this downstream task. In contrast, other classification models utilize word embedding, a vector representation of words, as input data. Contemporary word embedding algorithms create these embeddings by considering the surrounding context of each word. Nonetheless, these word embeddings often fail to capture the nuances of intensity differences between words. For example, words like 'more' and 'less' have embeddings closely positioned in the semantic space, despite representing distinct intensity levels. These intensity words such as 'much', 'more', and 'less' are frequently used to convey the strength of opinions. Their intensity distinctions are crucial in fine-level sentiment classification. This paper introduces an innovative intensity-aware feed-forward neural network, equipped with a novel referential loss function designed to capture these intensity differences between words. The proposed model effectively separates words of varying intensities while bringing together words sharing the same intensity in the semantic space. To assess the effectiveness of this refined word embedding in sentiment analysis tasks, diverse fine-level sentiment datasets are employed. The results demonstrate that the refined word embedding surpasses original embeddings and popular LLMs for fine-level sentiment analysis.
참고문헌 (48)
- "LM-CPPF : Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning" (2023)[URL]
- "Improving Language Understanding by Generative Pre-Training Alec" (2018)[URL]
- Baccianella, S., "SENTIWORDNET 3.0: An enhanced lexical resource for sentiment analysis and opinion mining", Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010, pp.2200~2204 (2010)
- Biber D, "Grammar of Spoken and Written English" (2019)
- Blitzer, J., "ACL07 Biographies, Bollywood, Boom-boxes and Blenders", Association for Computational Linguistics - ACL 2007, pp.440~447 (2007)
- Bollegala, D., "Unsupervised cross-domainword representation learning", ACL-IJCNLP 2015 - 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Proceedings of the Conference, 1, pp.730~740 (2015)
- Borg, A., "Using VADER sentiment and SVM for predicting customer response sentiment", Expert Systems with Applications, 162, pp.113746~ (2020)[DOI]
- Brown, T. B., "Language models are few-shot learners -- special version", Advances in Neural Information Processing Systems, 2020-Decem(NeurIPS) (2020)
- Chikersal, P., "SeNTU : Sentiment Analysis of Tweets by Combining a Rule-based Classifier with Supervised Learning", SemEval 2015 - 9th International Workshop on Semantic Evaluation, Co-Located with the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2015 - Proceedings, pp.647~651 (2015)
- "Scaling Instruction-Finetuned Language Models" (2022)[URL]
- Conneau, A., "Very deep convolutional networks for text classification", 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017 - Proceedings of Conference, 2(2001), pp.1107~1116 (2017)
- Cui, L., "Same: Sentiment-aware multi-modal embedding for detecting fake news", Proceedings of the 2019 IEEE, 2019, pp.2019, 41~48
- Devlin, J., "BERT: Pre-training of deep bidirectional transformers for language understanding", NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, 1, pp.4171~4186 (2019)
- Gavali, P. M., "A Supervised Framework for Sentiment Analysis : A Two-Stage Approach", Journal of Cognitive Science, 24(3), pp.283~312 (2023)
- Gregory Koch, "Siamese Neural Networks for One-shot Image Recognition Gregory", Proceedings of the 32nd International Conference on Machine Learning, Lille, 37 (2015)
- Hinton, G., "Stochastic neighbor embedding", Advances in Neural Information Processing Systems (2003)
- Large-scale evaluation of dependency-based DSMs : Are they worth the effort ?, "Lapesa, G.", 394~400 (2017)
- Lei, X., "Rating Prediction Based on Social Sentiment from Textual Reviews", IEEE Transactions on Multimedia, 18(9), pp.1910~1921 (2016)[DOI]
- Li, Y., "Learning Word Representations for Sentiment Analysis", Cognitive Computation, 9(6), pp.843~851 (2017)[DOI]
- Liang, B., "Context-aware embedding for targeted aspect-based sentiment analysis", ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, pp.4678~4683 (2020)
- Liang, B., "Embedding Refinement Framework for Targeted Aspect-Based Sentiment Analysis", IEEE Transactions on Affective Computing, 14(1), pp.279~293 (2023)[DOI]
- Liu, J., "Cross-domain sentiment aware word embeddings for review sentiment analysis", International Journal of Machine Learning and Cybernetics, 12(2), pp.343~354 (2021)[DOI]
- Liu, P., "Recurrent neural network for text classification with multi-task learning", IJCAI International Joint Conference on Artificial Intelligence, pp.2873~2879 (2016)
- "RoBERTa: A Robustly Optimized BERT Pretraining Approach" (2019)[URL]
- Maas, A. L., "Learning word vectors for sentiment analysis", ACL-HLT 2011-Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics : Human Language Technologies, 1, pp.142~150 (2011)
- Mao, X., "Sentiment-aware word embedding for emotion classification", Applied Sciences (Switzerland), 9(7) (2019)[DOI]
- McAuley, J. J., "From amateurs to connoisseurs", pp.897~908 (2013)
- Mikolov, T., "Efficient estimation of word representations in vector space", 1st International Conference on Learning Representations, ICLR 2013 - Workshop Track Proceedings, pp.1~12 (2013)
- Naderalvojoud, B., "Sentiment aware word embeddings using refinement and senti-contextualized learning approach", Neurocomputing, 405, pp.149~160 (2020)[DOI]
- Pang, B., "Thumbs up? Sentiment Classification using Machine Learning Techniques", Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, EMNLP, pp.79~86 (2002)
- Rezaeinia, S. M., "Sentiment analysis based on improved pre-trained word embeddings", Expert Systems with Applications, 117, pp.139~147 (2019)[DOI]
- "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter" (2019)[URL]
- Santur, Y., "Sentiment analysis based on gated recurrent unit", 2019 International Conference on Artificial Intelligence and Data Processing Symposium, IDAP 2019, 2019, pp.1~5 (2019)
- "The truth of the F-measure" (2007)[URL]
- Schroff, F., "FaceNet : A unified embedding for face recognition and clustering", Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.815~823 (2015)
- Shi, B., "Learning Domain-Sensitive and Sentiment-Aware Word Embeddings *", pp.2494~2504 (2018)
- Socher, R., "Recursive deep models for semantic compositionality over a sentiment treebank", EMNLP 2013 - 2013 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp.1631~1642 (2013)
- Tang, D., "Document modeling with gated recurrent neural network for sentiment classification", Conference Proceedings - EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, pp.1422~1432 (2015)
- Tang, D., "Sentiment Embeddings with Applications to Sentiment Analysis", IEEE Transactions on Knowledge and Data Engineering, 28(2), pp.496~509 (2016)[DOI]
- Learning Sentiment-Specific Word Embedding, "Tang, D.", Acl, 1555~1565 (2014)
- Tian, Y., "Cross-Domain End-To-End Aspect-Based Sentiment Analysis with Domain-Dependent Embeddings", Complexity (2021)[DOI]
- Vaswani, A., "Attention is all you need", Advances in Neural Information Processing Systems, pp.5999~6009 (2017)
- Wang, Y., "Refined Global Word Embeddings Based on Sentiment Concept for Sentiment Analysis", IEEE Access, 9(1), pp.37075~37085 (2021)[DOI]
- Yang, Z., "XLNet: Generalized autoregressive pretraining for language understanding", Advances in Neural Information Processing Systems, 32, pp.1~18 (2019)
- Yu, L. C., "Refining Word Embeddings Using Intensity Scores for Sentiment Analysis", IEEE, 2018, pp.26(3), 671~681
- Yue, C., "Collaborative attention neural network for multi-domain sentiment classification", Applied Intelligence, 51(6), pp.3174~3188 (2021)[DOI]
- Zainuddin, N., "Hybrid sentiment classification on twitter aspect-based sentiment analysis", Applied Intelligence, 48(5), pp.1218~1232 (2018)[DOI]
- "Sentiment Analysis in the Era of Large Language Models: A Reality Check" (2023)[URL]