Out-of-Vocabulary 단어에 강건한 병렬 Tri-LSTM 문장 임베딩을 이용한 감정분석
Sentiment Analysis using Robust Parallel Tri-LSTM Sentence Embedding in Out-of-Vocabulary Word
0
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
word2vec 등 기존의 단어 임베딩 기법은 원시 말뭉치에 출현한 단어들만을 대상으로 각 단어를 다차원 실수 벡터 공간에 고정된 길이의 벡터로 표현하기 때문에 형태론적으로 풍부한 표현체계를 가진 언어에 대한 단어 임베딩 기법에서는 말뭉치에 출현하지 않은 단어들에 대한 단어 벡터를 표현할 때 OOV(out-of-vocabulary) 문제가 빈번하게 발생한다. 문장을 구성하는 단어 벡터들로부터 문장 벡터를 구성하는 문장 임베딩의 경우에도 OOV 단어가 포함되었을 때 문장 벡터를 정교하게 구성하지 못하는 문제점이 있다. 특히, 교착어인 한국어는 어휘형태소와 문법형태소가 결합되는 형태론적 특성 때문에 미등록어의 임베딩 기법은 성능 향상의 중요한 요인이다. 본 연구에서는 단어의 형태학적인 정보를 이용하는 방식을 문장 수준으로 확장하고 OOV 단어 문제에 강건한 병렬 Tri-LSTM 문장 임베딩을 제안한다. 한국어 감정 분석 말뭉치에 대해 성능 평가를 수행한 결과 한국어 문장 임베딩을 위한 임베딩 단위는 형태소 단위보다 문자 단위가 우수한 성능을 보였으며, 병렬 양방향 Tri-LSTM 문장 인코더는 86.17%의 감정 분석 정확도를 달성하였다.
Abstract
The exiting word embedding methodology such as word2vec represents words, which only occur in the raw training corpus, as a fixed-length vector into a continuous vector space, so when mapping the words incorporated in the raw training corpus into a fixed-length vector in morphologically rich language, out-of-vocabulary (OOV) problem often happens. Even for sentence embedding, when representing the meaning of a sentence as a fixed-length vector by synthesizing word vectors constituting a sentence, OOV words make it challenging to meaningfully represent a sentence into a fixed-length vector. In particular, since the agglutinative language, the Korean has a morphological characteristic to integrate lexical morpheme and grammatical morpheme, handling OOV words is an important factor in improving performance. In this paper, we propose parallel Tri-LSTM sentence embedding that is robust to the OOV problem by extending utilizing the morphological information of words into sentence-level. As a result of the sentiment analysis task with corpus in Korean, we empirically found that the character unit is better than the morpheme unit as an embedding unit for Korean sentence embedding. We achieved 86.17% accuracy on the sentiment analysis task with the parallel bidirectional Tri-LSTM sentence encoder.
참고문헌 (24)
- 이태석, "LSTM 기반의 sequence-to-sequence 모델을 이용한 한글자동 띄어쓰기", 스마트미디어저널, 7(4), pp.17~23 (2018), (사)한국스마트미디어학회[DOI]
- 이태석, "미등록 어휘에 대한 선택적 복사를 적용한 문서 자동요약", 스마트미디어저널, 8(2), pp.58~65 (2019), (사)한국스마트미디어학회[DOI]
- 이명호, "단어와 문장의 의미를 고려한 비속어 판별 방법", 스마트미디어저널, 9(3), pp.98~106 (2020), (사)한국스마트미디어학회[DOI]
- 이현영, "종단 간 심층 신경망을 이용한 한국어 문장 자동 띄어쓰기", 정보처리학회논문지. 소프트웨어 및 데이터 공학, 8(11), pp.441~448 (2019), 한국정보처리학회[DOI]
- Marco Baroni, "Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors", Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp.238~247 (2014)
- Tomas Mikolov, "Distributed representations of words and phrases and their compositionality", Proceedings of Advances in neural information processing systems, pp.3111~3119 (2013)
- Jeffrey Pennington, "GloVe: Global vectors for word representation", Proceedings of Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.1532~1543 (2014)
- Dzmitry Bahdanau, "Neural machine translation by jointly learning to align and translate", arXiv preprint arXiv : 1409. 0473, ~ (2014)
- Piotr Bojanowski, "Enriching word vectors with subword information", Transactions of the Association for Computational Linguistics, 5, pp.135~146 (2017)
- Sungjoon Park, "Subword-level word vector representations for Korean", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp.2429~2438 (2018)
- Rico Sennrich, "Neural machine translation of rare words with subword units", Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp.1715~1725 (2016)
- Nicolas Garneau, "Predicting and interpreting embeddings for out of vocabulary words in downstream tasks", Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp.331~333 (2018)
- Sébastien Jean, "On using very large target vocabulary for neural machine translation", Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp.1~10 (2015)
- Thang Luong, "Addressing the rare word problem in neural machine translation", Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp.11~19 (2015)
- 조단비, "형태소 임베딩과 SVM을 이용한 뉴스 기사 정치적 편향성의 자동 분류", 한국정보처리학회 2020년 춘계학술발표대회, 27(01), pp.451~454 (2020)
- Thang Luong, "Better word representations with recursive neural networks for morphology", Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pp.104~113 (2013)
- Wang Ling, "Character-based neural machine translation", arXiv preprint arXiv:1511.04586, ~ (2015)
- Jianpeng Cheng, "Neural summarization by extracting sentences and words", Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp.484~494 (2016)
- Andrew M. Dai, "Document embedding with paragraph vectors", arXiv preprint arXiv:1507.07998, ~ (2015)
- Ranjan Kumar Behera, "Co-LSTM : Convolutional LSTM model for sentiment analysis in social big data", Information Processing & Management, 58(1) (2021)
- Dan-Bi Cho, "Sentiment analysis for informal text by using SentencePiece tokenizer and subword embedding", Proceedings of Korea Computer Congress 2020 (online), 47(1), pp.395~397 (2020)
- Wang Ling, "Finding Function in Form: Compositional character models for open vocabulary word representation", Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp.1520~1530 (2015)
- Sepp Hochreiter, "Long short-term memory", Neural Computation, 9(8), pp.1735~1780 (1997)
- 이현영, "문맥 의존 병렬 Trigram 문장 임베딩", 한국정보과학회 2020 한국소프트웨어종합학술대회 (online), pp.305~306 (2020)