A Supervised Framework for Sentiment Analysis: A Two-Stage Approach
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Sentiment analysis (SA) is a vital part of natural language processing (NLP). It involves analyzing textual data to determine expressed sentiment, be it positive or negative. Transformer-based models have gained popularity for sentiment prediction from the text. However, achieving promising results with these models requires extensive training data and processing power. Additionally, when using pre-trained transformer models for sentiment analysis, they do not generate sentiment-specific text embeddings as they are trained on large general-purpose corpora. To address these challenges, this paper presents a two-step framework. Firstly, the framework aims to learn high-level sentiment-oriented embeddings of text. It generates embeddings with Siamese Network and compares them based on their sentiment class using the triplet loss function. This approach enables the framework to capture distinctions between positive and negative sentences, ensuring the generation of sentiment-specific embeddings. Secondly, it incorporates a classification layer on top of the embedding layer to enhance sentiment classification. Experimental results showcase the effectiveness of our proposed framework, surpassing baseline sentiment analysis results on various benchmark datasets.
Abstract
Sentiment analysis (SA) is a vital part of natural language processing (NLP). It involves analyzing textual data to determine expressed sentiment, be it positive or negative. Transformer-based models have gained popularity for sentiment prediction from the text. However, achieving promising results with these models requires extensive training data and processing power. Additionally, when using pre-trained transformer models for sentiment analysis, they do not generate sentiment-specific text embeddings as they are trained on large general-purpose corpora. To address these challenges, this paper presents a two-step framework. Firstly, the framework aims to learn high-level sentiment-oriented embeddings of text. It generates embeddings with Siamese Network and compares them based on their sentiment class using the triplet loss function. This approach enables the framework to capture distinctions between positive and negative sentences, ensuring the generation of sentiment-specific embeddings. Secondly, it incorporates a classification layer on top of the embedding layer to enhance sentiment classification. Experimental results showcase the effectiveness of our proposed framework, surpassing baseline sentiment analysis results on various benchmark datasets.
참고문헌 (55)
- Abaskohi, A., "LM-CPPF: ParaphrasingGuided Data Augmentation for Contrastive Prompt-Based Few-Shot FineTuning", arXiv:2211.11754v3, ~ (2023)
- Angelovska, M., "Siamese Neural Networks for Detecting Complementary Products", Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop, pp.65~70 (2021), Association for Computational Linguistics
- Arbane, M., "Social media-based Covid-19 sentiment classification model using Bi-LSTM", Expert Systems with Applications, 212 (2023)[DOI]
- Baccianella, S., "SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining", Proceedings of the International Conference on Language Resources and Evaluation (2010)
- Anton Borg, "Using VADER sentiment and SVM for predicting customer response sentiment", Expert Systems with Applications, 162, pp.113746~ (2020), Elsevier BV[DOI]
- Susann Boy, "Emoji-Based Transfer Learning for Sentiment Tasks", Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (2021), Association for Computational Linguistics[DOI]
- Chikersal, P., "SeNTU: Sentiment Analysis of Tweets by Combining a Rule-based Classifier with Supervised Learning", Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp.647~651 (2015), Association for Computational Linguistics
- Chen, H., "Neural sentiment classification with user and product attention", Proceedings of the 2016conference on Empirical Methods in Natural Language processing, pp.1650~1659 (2016)
- Text Sentiment Orientation Analysis Based on Multi-Channel CNN and Bidirectional GRU with Attention Mechanism, "Cheng, Y.", IEEE Access, ~ (2020)
- Choudhary, N., "Emotions are Universal: Learning Sentiment Based Representations of Resource-Poor Languages using Siamese Networks", arxiv. arXiv:1804.00805, ~ (2018)
- Conneau, A., "Very deep convolutional networks for text classification", Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 1, pp.1107~1116 (2017)
- Anan Dai, "Learning from word semantics to sentence syntax by graph convolutional networks for aspect-based sentiment analysis", International Journal of Data Science and Analytics, 14(1), pp.17~26 (2022), Springer Science and Business Media LLC[DOI]
- Devlin, J., "BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.4171~4186 (2019)
- Mauro Dragoni, "A Neural Word Embeddings Approach for Multi-Domain Sentiment Analysis", IEEE Transactions on Affective Computing, 8(4), pp.457~470 (2017), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
- "From Group to Individual Labels using Deep Features"[URL]
- Duncan, B., "Neural networks for sentiment analysis on Twitter", 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC) (2015)
- Gleize, M., "Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network", Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.967~976 (2019), Association for Computational Linguistics
- Hinton, G., "Stochastic Neighbor Embeddings" (2013), Natural Information Processing Systems
- Huang, M., "Siamese Network-Based Supervised Topic Modeling", Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp.4652~4662 (2018), Association for Computational Linguistics
- Joulin, A., "Bag of Tricks for Efficient Text Classification", Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2, pp.427~431 (2017)
- Abdullah Talha Kabakus, "A novel <scp>COVID<", scp>‐19 sentiment analysis in Turkish based on the combination of convolutional neural network and bidirectional long‐short term memory on Twitter, 2022, pp.34(22), ~[DOI]
- Ke, P., "SentiLARE: SentimentAware Language Representation Learning with Linguistic Knowledge", arXiv:1911.02493, ~ (2020)
- Kim, Y., "Convolutional neural networks for sentence classification", Proceedings of the (2014) Conference on Empirical Methods in Natural Language Processing, pp.1746~1751 (2014)
- Koch, G., "Siamese Neural Networks for One-shot Image Recognition", Proceedings of the 32nd International Conference on Machine Learning. Lille, France (2015)
- Lapesa, G., "Large Scale Evaluation of Dependency-based DSMs: Are They worth of Efforts", 15th Conference of the European Chapter of the Association for Computational Linguistics, 2, pp.394~400 (2017)
- Maas, A. L, "Learning word vectors for sentiment analysis", Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 1, pp.142~150 (2011)
- Mikolov, T., "Efficient estimation of word representations in vector space", Proc. Int. Conf. Learning Representations (2013)
- Müller, T., "Few-Shot Learning with Siamese Networks and Label Tuning", Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.8532~8545 (2022), Association for Computational Linguistics
- Oniani, D., "Few Shot Learning for Clinical Natural Language Processing using Siamese Neural Network", arXiv:2208.14923v2, ~ (2022)
- Pang, B., "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales", Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pp.115~124 (2005)
- Pang, B., "Thumbs up? Sentiment classification using machine learning techniques", Proc. Assoc. Comput. Linguistics Conf. Empirical Methods Natural Lang. Process, 10, pp.79~86 (2002)
- Pang, L., "Text Matching as Image Recognition" (2016)[DOI]
- Pennington, J., "GloVe: Global Vectors for Word Representation", Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.1532~1543 (2014)
- M POONGOTHAI, "Chronological-brain storm optimization based support vector neural network for sentiment classification using map reduce framework", Sādhanā, 45(1) (2020), Springer Science and Business Media LLC[DOI]
- Radford, A., "Improving language understanding by generative pre-training", ~ (2018)
- Sasaki Y., "The Truth of the F-measure", ~ (2007)
- Nicolás Serrano, "Siamese neural networks in recommendation", Neural Computing and Applications, 35(19), pp.13941~13953 (2023), Springer Science and Business Media LLC[DOI]
- Yong Shi, "Stock movement prediction with sentiment analysis based on deep learning networks", Concurrency and Computation: Practice and Experience, 33(6) (2020), Wiley[DOI]
- Shonibare, O., ". ASBERT: Siamese and Triplet network embedding for Open Question Answering", arXiv:2104.08558, ~ (2021)
- M. Sivakumar, "Aspect-based sentiment analysis of mobile phone reviews using LSTM and fuzzy logic", International Journal of Data Science and Analytics, 12(4), pp.355~367 (2021), Springer Science and Business Media LLC[DOI]
- Socher, R., "Recursive deep models for semantic compositionality over a sentiment treebank", Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp.1631~1642 (2013)
- Tan, L. I., "Rule-Based Sentiment Analysis for Financial News", 2015 IEEE International Conference on Systems, Man, and Cybernetics, pp.1601~1606 (2015)
- Tang, D., "Sentiment Embeddings with Applications to Sentiment Analysis", IEEE Transactions on Knowledge and Data Engineering, 28(2) (2016)
- Tang, D., "Document modeling with gated recurrent neural network for sentiment classification", Proceedings of the 2015Conference on Empirical Methods in Natural Language Processing, pp.1422~1432 (2015)
- Rashmi K Thakur, "Kernel Optimized-Support Vector Machine and Mapreduce framework for sentiment classification of train reviews", Sādhanā, 44(1) (2018), Springer Science and Business Media LLC[DOI]
- Toledo, G., "Transfer Learning with Joint Fine-Tuning for Multimodal Sentiment Analysis", arxiv-preprint. arXiv:2210.05790, ~ (2022)
- Vaswani, A., "Attention is all you need", arXiv:1706.03762, ~ (2017)
- Refined Global Word Embeddings Based on Sentiment Concept for Sentiment Analysis, "Wang, Y.", IEEE, ~ (2021)
- Wiggers, K. L., "Image Retrieval and Pattern Spotting using Siamese Neural Network", 2019International Joint Conference on Neural Networks (IJCNN), 2019, pp.1~8 (2019)
- Yu, L., "Refining Word Embeddings Using Intensity Scores for Sentiment Analysis", IEEE Transactions on Audio, Speech, and Language Processing, 26(3) (2018)
- Zhang, S., "Irony detection via sentiment-based transfer learning", Information Processing & Management, 56(5), pp.1633~1644 (2019)
- Zhang, J., "KASN: Knowledge-Aware Siamese Network for sentiment analysis. AIIPCC 2022", The Third International Conference on Artificial Intelligence. Information Processing and Cloud Computing, pp.1~8 (2022)
- Zhao, C., "Deep transfer learning for social media cross-domain sentiment classification", Chinese National Conference on Social Media Processing, pp.232~243 (2017)
- Sentiment analysis of steam review datasets using naive bayes and decision tree classifier, "Zuo, Z.", ~ (2018)
- Zhou, J., "SentiX: A Sentiment-Aware Pre-Trained Model for Cross-Domain Sentiment Analysis", Proceedings of the 28th International Conference on Computational Linguistics, pp.568~579 (2020)
