한국어에 대한 양방향 순환 신경망을 이용한 동형이의어 중의성 해소 연구
A Study on Word Sense Disambiguation Using Bidirectional Recurrent Neural Network for Korean Language
3
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Word sense disambiguation(WSD) that determines the exact meaning of homonym which can be used in different meanings even in one form is very important to understand the semantical meaning of text document. Many recent researches on WSD have widely used NNLM(Neural Network Language Model) in which neural network is used to represent a document into vectors and to analyze its semantics. Among the previous WSD researches using NNLM, RNN(Recurrent Neural Network) model has better performance than other models because RNN model can reflect the occurrence order of words in addition to the word appearance information in a document. However, since RNN model uses only the forward order of word occurrences in a document, it is not able to reflect natural language’s characteristics that later words can affect the meanings of the preceding words. In this paper, we propose a WSD scheme using Bidirectional RNN that can reflect not only the forward order but also the backward order of word occurrences in a document. From the experiments, the accuracy of the proposed model is higher than that of previous method using RNN. Hence, it is confirmed that bidirectional order information of word occurrences is useful for WSD in Korean language.
Abstract
Word sense disambiguation(WSD) that determines the exact meaning of homonym which can be used in different meanings even in one form is very important to understand the semantical meaning of text document. Many recent researches on WSD have widely used NNLM(Neural Network Language Model) in which neural network is used to represent a document into vectors and to analyze its semantics. Among the previous WSD researches using NNLM, RNN(Recurrent Neural Network) model has better performance than other models because RNN model can reflect the occurrence order of words in addition to the word appearance information in a document. However, since RNN model uses only the forward order of word occurrences in a document, it is not able to reflect natural language’s characteristics that later words can affect the meanings of the preceding words. In this paper, we propose a WSD scheme using Bidirectional RNN that can reflect not only the forward order but also the backward order of word occurrences in a document. From the experiments, the accuracy of the proposed model is higher than that of previous method using RNN. Hence, it is confirmed that bidirectional order information of word occurrences is useful for WSD in Korean language.
참고문헌 (24)
- M. Sundermeyer, "LSTM Neural Networks for Language Modeling", Interspeech, pp.194~197 (2012)
- mykang, bgkim, "Word Sense Disambiguation using Word2Vec", Proceeding of 27th Conference on Human & Cognitive Language Technology, pp.81~84 (2015)
- jcshin, "Homograph Word Sense Disambiguation using Korean Lexical Semantic Map(UWordMap) and Word-Embedding", Proceeding of Korean Computer Congress 2016, pp.702~704 (2016)
- D. Yuan, "Word Sense Disambiguation with Neural Language Models", arXivpreprint arXiv:1603.07012, ~ (2016)
- jhmin, "Study on Word Sense Disambiguation Using Recurrent Neural Network for Korean", Proceeding of Winter Conference on Korean Association of Computer Education 2017, pp.93~96 (2017)
- jsbae, "End-to-end Learning of Korean Semantic Role Labeling Using Bidirectional LSTM CRF", Proceeding of 42th Winter Conference on Korean Institute of Information Scientists and Engineers, pp.566~568 (2015)
- C. Irsoy, "Opinion Mining with Deep Recurrent Neural Networks", Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing, pp.720~728 (2014)
- H. Sak, "Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition", arXiv preprint arXiv:1402.1128, ~ (2014)
- Junyoung Chung, "Empirical evaluation of gated recurrent neural networks on sequence modeling", arXiv preprint arXiv:1412.3555, ~ (2014)
- 정순호, "이진 삼차 재귀 신경망과 유전자 알고리즘을 이용한 문맥-자유 문법의 추론", 한국컴퓨터정보학회논문지, 17(3), pp.11~25 (2012), 한국컴퓨터정보학회
- hmkim, "Syllable-based Korean POS Tagging using POS Distribution and Bidirectional LSTM CRFs", Proceeding of 28th Conference on Human & Cognitive Language Technology, pp.3~8 (2016)
- 안성만, "딥러닝의 모형과 응용사례", 지능정보연구, 22(2), pp.127~142 (2016), 한국지능정보시스템학회
- Y. Bengio, "Learning Long-Term Dependencies with Gradient Descent is Difficult", IEEE Transactions on Neural Networks, 5(2) (1994)
- 강범모, "텍스트 맥락과 단어 의미: 잠재의미분석", 언어학, (68), pp.3~34 (2014), 사단법인 한국언어학회
- "Gensim"[URL]
- T. Mikolov, "Efficient estimation of word representations in vector space", arXivpreprint arXiv: 1301.3781, ~ (2013)
- 정명범, "색맹인 사람들을 도울 수 있는 스마트 폰 기반 색상 매칭 애플리케이션", 한국컴퓨터정보학회논문지, 20(5), pp.65~72 (2015), 한국컴퓨터정보학회
- "National Institute of Korean Language"[URL]
- hgkim, "21st Century Sejong Modern Korean Corpora: Results and Expectations", Proceeding of 19th Conference on Human & Cognitive Language Technology, pp.311~316 (2007)
- shchoi, "On Word Embedding Models and Parameters Optimized for Korean", Proceeding of 28th Conference on Human & Cognitive Language Technology, pp.252~256 (2016)
- "Korean Wikipedia"[URL]
- "Namuwiki"[URL]
- "SENSEAVAL-2"[URL]
- "Keras"[URL]