다중작업학습 기법을 적용한 Bi-LSTM 개체명 인식 시스템 성능 비교 분석
Performance Comparison Analysis on Named Entity Recognition system with Bi-LSTM based Multi-task Learning
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
다중작업학습(Multi-Task Learning, MTL) 기법은 하나의 신경망을 통해 다양한 작업을 동시에 수행하고 각 작업 간에 상호적으로 영향을 미치면서 학습하는 방식을 말한다. 본 연구에서는 전통문화 말뭉치를 직접 구축 및 학습데이터로 활용하여 다중작업학습 기법을 적용한 개체명 인식 모델에 대해 성능 비교 분석을 진행한다. 학습 과정에서 각각의 품사 태깅(Part-of-Speech tagging, POS-tagging) 과 개체명 인식(Named Entity Recognition, NER) 학습 파라미터에 대해 Bi-LSTM 계층을 통과시킨 후 각각의 Bi-LSTM을 계층을 통해 최종적으로 두 loss의 joint loss를 구한다. 결과적으로, Bi-LSTM 모델을 활용하여 단일 Bi-LSTM 모델보다 MTL 기법을 적용한 모델에서 1.1%~4.6%의 성능 향상이 있음을 보인다.
Abstract
Multi-Task Learning(MTL) is a training method that trains a single neural network with multiple tasks influences each other. In this paper, we compare performance of MTL Named entity recognition(NER) model trained with Korean traditional culture corpus and other NER model. In training process, each Bi-LSTM layer of Part of speech tagging(POS-tagging) and NER are propagated from a Bi-LSTM layer to obtain the joint loss. As a result, the MTL based Bi-LSTM model shows 1.1%~4.6% performance improvement compared to single Bi-LSTM models.
참고문헌 (17)
- S. Ruder, "An overview of multi-task learning in deep neural networks", arXiv preprint arXiv:1706.05098, ~ (2017)
- R. Caruana, "Multitask learning", Machine learning, 28(1), pp.41~75 (1997)
- M. Long, "Learning multiple tasks with deep relationship networks", arXiv preprint arXiv:1506.02117, 2~ (2015)
- Y. Zhang, "Learning to multitask", In Advances in Neural Information Processing Systems, pp.5771~5782 (2018)
- T. Mikolov, "Distributed representations of words and phrases and their compositionality", In Advances in neural information processing systems, pp.3111~3119 (2013)
- J. Pennington, "Glove: Global vectors for word representation", Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp.1532~1543 (2014)
- P. Bojanowski, "Enriching word vectors with subword information", Transactions of the Association for Computational Linguistics, 5, pp.135~146 (2017)
- M. E. Peters, "Deep contextualized word representations", arXiv preprint arXiv:1802.05365, ~ (2018)
- J. Devlin, "Bert: Pre-training of deep bidirectional transformers for language understanding", arXiv preprint arXiv:1810.04805, ~ (2018)
- Z. Yang, "XLNet : Generalized Autoregressive Pretraining for Language Understanding. arXiv preprint", arXiv preprint arXiv:1906.08237, ~ (2019)
- Y. Liu, "Roberta: A robustly optimized bert pretraining approach", arXiv preprint arXiv:1907.11692, ~ (2019)
- A. Lamurias, "BO-LSTM : classifying relations via long short-term memory networks along biomedical ontologies", BMC bioinformatics, 20(1), pp.10~ (2019)
- C. Lyu, "Long short-term memory RNN for biomedical named entity recognition", BMC bioinformatics, 18(1), pp.462~ (2017)
- A. R. Tuor, "Recurrent neural network language models for open vocabulary event-level cyber anomaly detection", In Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence (2018)
- S. KP, "RNNSecureNet: Recurrent neural networks for Cyber security use-cases", arXiv preprint arXiv:1901.04281, ~ (2019)
- 김경민, "한국 전통문화 말뭉치구축 및 Bi-LSTM-CNN-CRF를 활용한 전통문화 개체명 인식 모델 개발", 한국융합학회논문지, 9(12), pp.47~52 (2018), 한국융합학회[DOI]
- 이동엽, "자질 보강과 양방향 LSTM-CNN-CRF 기반의 한국어 개체명 인식 모델", 한국융합학회논문지, 8(12), pp.55~62 (2017), 한국융합학회[DOI]