본문 바로가기
KERD
논문KCI 등재

대규모 언어 모델을 사용한 제로샷 한국어 감성 분석: 사전 학습된 언어 모델과의 비교

Zero-shot Korean Sentiment Analysis with Large Language Models: Comparison with Pre-trained Language Models

이동희|장백철|권순찬
한국컴퓨터정보학회 (The Korean Society Of Computer And Information)|2024.02|한국컴퓨터정보학회논문지 (Journal of The Korea Society of Computer and Information)|제29권 제2호|pp. 43-50|DOI https://doi.org/10.9708/jksci.2024.29.02.043

2

조회

0

다운로드

1

인용

0

참고문헌

키워드

자연어처리AI인공 지능Natural Language Processing딥러닝Deep Learning감성 분석Language ModelSentiment analysis언어 모델
주제분류컴퓨터학

초록

본 논문은 GPT-3.5 및 GPT-4와 같은 대규모 언어 모델의 한국어 감성 분석 성능을 ChatGPT API 를 활용한 zero-shot 방법으로 평가하고, 이를 KoBERT와 같은 사전 학습된 한국어 모델들과 비교한다. 실험을 통해 영화, 게임, 쇼핑 등 다양한 분야의 한국어 감성 분석 데이터셋을 사용하여 모델들의 효율성을 검증한다. 실험 결과, LMKor-ELECTRA 모델이 F1-score 기준으로 가장 높은 성능을보여주었으며, GPT-4는 특히 영화 및 쇼핑 데이터셋에서 높은 정확도와 F1-score를 기록하였다. 이는 zero-shot 학습 방식의 대규모 언어 모델이 특정 데이터셋에 대한 사전 학습 없이도 한국어 감성분석에서 높은 성능을 발휘할 수 있음을 시사한다. 그러나 일부 데이터셋에서의 상대적으로 낮은성능은 zero-shot 기반 방법론의 한계점으로 지적될 수 있다. 본 연구는 대규모 언어 모델의 한국어감성 분석 활용 가능성을 탐구하며, 이 분야의 향후 연구 방향에 중요한 시사점을 제공한다.

Abstract

This paper evaluates the Korean sentiment analysis performance of large language models like GPT-3.5 and GPT-4 using a zero-shot approach facilitated by the ChatGPT API, comparing them to pre-trained Korean models such as KoBERT. Through experiments utilizing various Korean sentiment analysis datasets in fields like movies, gaming, and shopping, the efficiency of these models is validated. The results reveal that the LMKor-ELECTRA model displayed the highest performance based on F1-score, while GPT-4 particularly achieved high accuracy and F1-scores in movie and shopping datasets. This indicates that large language models can perform effectively in Korean sentiment analysis without prior training on specific datasets, suggesting their potential in zero-shot learning. However, relatively lower performance in some datasets highlights the limitations of the zero-shot based methodology. This study explores the feasibility of using large language models for Korean sentiment analysis, providing significant implications for future research in this area.

참고문헌 (38)

  1. Vaswani, Ashish, "Attention is all you need", Advances in neural information processing systems, 30 (2017)
  3. Min, Bonan, "Recent advances in natural language processing via large pre-trained language models: A survey", ACM Computing Surveys, 56(2), pp.1~40 (2023)[DOI]
  4. "ChatGPT"[URL]
  5. "BARD"[URL]
  8. Medhat, Walaa, "Sentiment analysis algorithms and applications: A survey", Ain Shams engineering journal, 5(4), pp.1093~1113 (2014)[DOI]
  9. Al-Thubaity, Abdulmohsen, "Evaluating ChatGPT and Bard AI on Arabic Sentiment Analysis", Proceedings of ArabicNLP 2023 (2023)
  11. Radford, Alec, "Improving language understanding by generative pre-training", ~ (2018)
  12. Radford, Alec, "Better language models and their implications", OpenAI blog, 1(2) (2019)
  13. Raffel, Colin, "Exploring the limits of transfer learning with a unified text-to-text transformer", The Journal of Machine Learning Research, 21(1), pp.5485~5551 (2020)[DOI]
  17. Luo, Renqian, "BioGPT: generative pre-trained transformer for biomedical text generation and mining", Briefings in Bioinformatics, 23(6), pp.bbac409~ (2022)[DOI]
  18. Floridi, Luciano, "GPT-3: Its nature, scope, limits, and consequences", Minds and Machines, 30, pp.681~694 (2020)[DOI]
  19. Lund, Brady D., "Chatting about ChatGPT: how may AI and GPT impact academia and libraries?", Library Hi Tech News, 40(3), pp.26~29 (2023)[DOI]
  21. Ouyang, Xi, "Sentiment analysis using convolutional neural network", 2015 IEEE international conference on computer and information technology; ubiquitous computing and communications; dependable, autonomic and secure computing;pervasive intelligence and computing (2015), IEEE
  22. Yoosin Kim, "Stock-Index Invest Model Using News Big Data Opinion Mining", Journal of Intelligence and Information Systems, 18(2), pp.143~156 (2012)[DOI]
  23. Sang Hoon Lee, "Sentiment analysis on movie review through building modified sentiment dictionary by movie genre", Journal of Intelligence and Information Systems, 22(2), pp.97~113 (2016)[DOI]
  24. Sang-Min Park, "KNU Korean Sentiment Lexicon-Bi-LSTM-based Method for Building a Korean Sentiment Lexicon-", Journal of Intelligence and Information Systems, 24(4), pp.219~240 (2018)[DOI]
  25. Hyun-jung Park, "Sentiment Analysis of Korean Reviews Using CNN-Focusing on Morpheme Embedding-", Journal of Intelligence and Information Systems, 24(2), pp.59~83 (2018)[DOI]
  26. Jinju Hong, "A Malicious Comments Detection Technique on the Internet using Sentiment Analysis and SVM", Journal of the Korea Institute of Information and Communication Engineering, 20(2), pp.260~267 (2016)[DOI]
  27. Jiyeon Hyun, "How to improve the accuracy of recommendation systems : Combining ratings and review texts sentiment scores", Journal of Intelligence and Information Systems, 25(1), pp.219~239 (2019)[DOI]
  28. "KoBERT"[URL]
  29. "LMkor"[URL]
  30. Jae-hong Lee, "Comparison of Sentiment Classification Performance of for RNN and Transformer-Based Models on Korean Reviews", Journal of The Korea Institute of Electronic Communication Sciences, 18(04), pp.693~700 (2023)[DOI]
  32. Kocoń, Jan, "ChatGPT: Jack of all trades, master of none", Information Fusion, pp.101861~ (2023)[DOI]
  36. Al-Thubaity, Abdulmohsen, "Evaluating ChatGPT and Bard AI on Arabic Sentiment Analysis", Proceedings of ArabicNLP 2023 (2023)
  37. HyoungDo Kim, "Zero-shot Sentiment Analysis of Movie Reviews Based on GPT-4", JOURNAL OF THE KOREA CONTENTS ASSOCIATION, 23(11), pp.185~196 (2023)[DOI]
  38. Li, Xiang, "A shallow BERT-CNN model for sentiment analysis on MOOCs comments", 2019 IEEE International Conference on Engineering, Technology and Education (TALE) (2019), IEEE

이 간행물의 다른 논문 (19건)

앙상블 학습 접근법을 활용한 경로 손실 예측노언수, 권범pp. 1-12YOLOv8과 ORB 알고리즘 이용한 원통형 약통의 식별김대현, 최효현pp. 13-20딥러닝 기반 웹페이지 자동생성시스템 설계 연구고영범, 이한진, 최지훈, 김정환pp. 21-30엣지 기반 적대적 예제 탐지 모델의 효과에 대한 연구: 다양한 적대적 알고리즘을 대상으로심재성, 조겨리pp. 31-41XGBoost, LightGBM을 이용한 종목추천 뉴스기사 기반 주식 수익률 방향성 예측이준기, 황유진, 손승연pp. 51-59
원문 보기