본문 바로가기
KERD
논문KCI 등재

Encoding Dictionary Feature for Deep Learning-based Named Entity Recognition

Chirawan Ronran|Sayan Unankard|Seungwoo Lee
한국콘텐츠학회 (The Korea Contents Association)|2021.12|International Journal of Contents|제17권 제4호|pp. 1-15|DOI https://doi.org/10.5392/IJoC.2021.17.4.001

1

조회

0

다운로드

0

인용

0

참고문헌

키워드

BiLSTM DictionaryCNN DictionarySelf-attention DictionaryGFID DatasetBiomedical NER.
주제분류컴퓨터학

초록

Named entity recognition (NER) is a crucial task for NLP, which aims to extract information from texts. To build NER systems, deep learning (DL) models are learned with dictionary features by mapping each word in the dataset to dictionary features and generating a unique index. However, this technique might generate noisy labels, which pose significant challenges for the NER task. In this paper, we proposed DLdictionary features, and evaluated them on two datasets, including the OntoNotes 5.0 dataset and our new infectious disease outbreak dataset named GFID. We used (1) a Bidirectional Long Short-Term Memory (BiLSTM) character and (2) pre-trained embedding to concatenate with (3) our proposed features, named the Convolutional Neural Network (CNN), BiLSTM, and self-attention dictionaries, respectively. The combined features (1-3) were fed through BiLSTM - Conditional Random Field (CRF) to predict named entity classes as outputs. We compared these outputs with other predictions of the BiLSTM character, pre-trained embedding, and dictionary features from previous research, which used the exact matching and partial matching dictionary technique. The findings showed that the model employing our dictionary features outperformed other models that used existing dictionary features. We also computed

Abstract

Named entity recognition (NER) is a crucial task for NLP, which aims to extract information from texts. To build NER systems, deep learning (DL) models are learned with dictionary features by mapping each word in the dataset to dictionary features and generating a unique index. However, this technique might generate noisy labels, which pose significant challenges for the NER task. In this paper, we proposed DLdictionary features, and evaluated them on two datasets, including the OntoNotes 5.0 dataset and our new infectious disease outbreak dataset named GFID. We used (1) a Bidirectional Long Short-Term Memory (BiLSTM) character and (2) pre-trained embedding to concatenate with (3) our proposed features, named the Convolutional Neural Network (CNN), BiLSTM, and self-attention dictionaries, respectively. The combined features (1-3) were fed through BiLSTM - Conditional Random Field (CRF) to predict named entity classes as outputs. We compared these outputs with other predictions of the BiLSTM character, pre-trained embedding, and dictionary features from previous research, which used the exact matching and partial matching dictionary technique. The findings showed that the model employing our dictionary features outperformed other models that used existing dictionary features. We also computed

참고문헌 (39)

  1. Jing Li, "A Survey on Deep Learning for Named Entity Recognition", IEEE Transactions on Knowledge and Data Engineering, 34(1), pp.50~70 (2022), Institute of Electrical and Electronics Engineers (IEEE)[DOI]
  2. Chirawan Ronran, "Delayed Combination of Feature Embedding in Bidirectional LSTM CRF for NER", Applied Sciences, 10(21), pp.7557~ (2020), MDPI AG[DOI]
  3. J. Shang, "Learning Named Entity Tagger using Domain-Specific Dictionary", Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (2018)
  4. W. W. Cohen, "Exploiting dictionaries in named entity extraction", Proceedings of the 2004 ACM SIGKDD international conference on Knowledge discovery and data mining - KDD (2004)
  5. L. Ratinov, "Design challenges and misconceptions in named entity recognition", Proceedings of the Thirteenth Conference on Computational Natural Language Learning CoNLL 2009 (2009)
  6. S. Sarawagi, "Semi-Markov conditional random fields for information extraction", Proc. Neural Inform. Process. Syst., pp.1185~1192 (2004)
  7. "Named entity recognition with bidirectional LSTM-CNNs, 2015, arXiv:1511.08308" (2015)[URL]
  8. T. Liu, "Towards Improving Neural Named Entity Recognition with Gazetteers", Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (2019)
  9. Qi Wang, "Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition", Journal of Biomedical Informatics, 92, pp.103133~ (2019), Elsevier BV[DOI]
  10. "kermitt2/delft: a Deep Learning Framework for Text - GitHub"[URL]
  11. P. Bojanowski, "Enriching word vectors with subword information", Trans. Assoc. Comput. Linguistics, 5, pp.135~146 (2017)[DOI]
  12. Devlin, "BERT : Pre-training of deep bidirectional transformers for language understanding", arXiv : 1810. 04805, ~ (2018)
  13. M. Peters, "Deep contextualized word representations", Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics: Human Lang. Technol, 1, pp.2227~2237 (2018)[DOI]
  14. F. Dugas, "Applying BLSTM-CNNs and extended lexicons to named entity recognition in tweets", Proc. W-NUT 2016, pp.178~187 (2016)
  15. A. Neelakantan, "Learning dictionaries for named entity recognition using minimal supervision", Computer Science (2015)
  16. "spaCy ꞏ Industrial-strength Natural Language Processing in Python"[URL]
  17. C. Ronran, "Lexical Feature Representation for Named Entity Recognition", ICCC 2020, pp.375~376 (2020)
  18. Z. Jie, "Dependency-guided LSTM-CRF for named entity recognition", Proc. Conf. Empirical Methods Natural Lang. Process., 9th Int. Joint Conf. Natural Lang. Process. (EMNLP-IJCNLP),, pp.1~3 (2019)
  19. T. Mikolov, "Strategies for training large scale neural network language models", Proc. 2011 IEEE Workshop Automatic Speech Recognition and Understanding, pp.196~201 (2011)
  20. R. Collobert, "Natural language processing (almost)from scratch", J. Mach. Learn. Res.,, 12, pp.2493~2537 (2011)
  21. X. Ma, "End-to-end sequence labeling via bi-directional LSTM_CNNs-CRF", Proc. 54th Annu. Meet. Assoc. Comput. Linguistics (Volume 1: Long Papers)., pp.1064~1074 (2016)
  22. G. Lample, "Neural architectures for named entity recognition", Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics, Human Lang. Technol, pp.260~270 (2016)
  23. L. Liu, "Empower sequence labeling with task-aware neural language model", Proc. Assoc. Adv. Artif. Intell, pp.5253~5260 (2018)
  24. M. Peters, "Semi-supervised sequence tagging with bidirectional language models", Proc. 55th Annu. Meeting Assoc. Comput. Linguistics, 1, pp.1756~1765 (2017)
  25. H. A. Nayel, "Integrating Dictionary Feature into A Deep Learning Model for Disease Named Entity Recognition", CoRR (2019)
  26. H. Wu, "An Effective Approach of Named Entity Recognition for Cyber Threat Intelligence", 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC),, pp.1370~1374 (2020)
  27. X. Cui., "BiLSTM-Attention-CRF model for entity extraction in internet recruitment data", Procedia Computer Science, 183, pp.706~712 (2021)[DOI]
  28. G. Wu, "An Attention-Based BiLSTM-CRF Model for Chinese Clinic Named Entity Recognition", in IEEE Access, 7, pp.113942~113949 (2019)[DOI]
  29. "Softmax Function Definition | DeepAI"[URL]
  30. F. Wu, "Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation", Proc. World Wide Web Conf. (WWW),, pp.3342~ (2019)
  31. P. H. Li, "Why Attention? Analyze BiLSTM Deficiency and Its Remedies in the Case of Ner", Proceedings of the AAAI Conference on Artificial Intelligence (2020)
  32. A. Vaswani, "Attention is all you need", arXiv Preprint, arXiv : 1706. 03762, ~ (2017)
  33. Y. Zhu, "CAN-NER: Convolutional attention network for Chinese named entity recognition", Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics Hum. Lang. Technol, 1, pp.3384~3393 (2019)
  34. "keras-self-attention: Attention mechanism for processing sequential data that considers the context for each timestamp"[URL]
  35. H. Chen, "GRN: Gated relation network to enhance convolutional neural network for named entity recognition", Proc. AAAI Conf. Artif. Intell., (AAAI),, pp.6236~6243 (2019)
  36. 김재훈, "기계학습 기반 개체명 인식을 위한 사전 자질 생성", Journal of Information Science Theory and Practice, 41(2), pp.31~46 (2010), 한국과학기술정보연구원
  37. J. Xie, "Self-attention-based BiLSTM model for short text fine-grained sentiment classification", IEEE Access, 7, pp.180558~180570 (2019)
  38. T. Liu, "Towards improving neural named entity recognition with gazetteers", ACL, pp.5301~5307 (2019)
  39. Z. -X. Ye, "Hybrid semi-markov crf for neural sequence labeling", ACL, pp.235~240 (2018)

관련 논문AI추천

Syllable-based Korean named entity recognition using convolutional neural network박혁로, 유연수한국마린엔지니어링학회 · 한국마린엔지니어링학회지2020 · 제44권 제1호Evaluating and applying deep learning-based multilingual named entity recognition김재훈, 김창현, 박호민, 천민아한국마린엔지니어링학회 · 한국마린엔지니어링학회지2018 · 제42권 제2호Named Entity Recognition in Electronic Health Records: A Methodological ReviewAndrés Orozco-Duque, Ever A. Torres-Silva, María C. Durango대한의료정보학회 · Healthcare Informatics Research2023 · 제29권 제4호딥러닝 기반 한국어 개체명 인식의 평가와 오류 분석 연구김민수, 송영숙, 유현조, 윤기현, 정유남한국언어학회 · 언어2021 · 제46권 제3호Improved Character-Based Neural Network for POS Tagging on Morphologically Rich LanguagesAlim Murat, Samat Ali한국정보처리학회 · JIPS(Journal of Information Processing Systems)2023 · 제19권 제3호An Active Co-Training Algorithm for Biomedical Named-Entity RecognitionKeun Ho Ryu, Meijing Li, Oyun-Erdene Namsrai, Tsendsuren Munkhdalai, Unil Yun한국정보처리학회 · JIPS(Journal of Information Processing Systems)2012 · 제8권 제4호

이 간행물의 다른 논문 (8건)

An Exploratory Study on the Usage Patterns of Softwarebased Design Tools in Designers' Ideation and Collaboration Activities김성범, Dongwook Kimpp. 16-34An Analysis of Effective Factors in Public Awareness Campaigns through Facebook: Focus on Fine Dust Issues조삼섭, 응웬 탄 마이pp. 35-45Citizen Science approach and Datification: Pilot Study on Factors Influencing the Dementia among Older AdultsNa-Li Park, 임홍탁, Jeong-won Han, 성덕현, Kyung Won Park, Woo-Kyong Kimpp. 46-51Rethinking the US Presidential Election: Feminism and Big Data정세원, 박한우pp. 52-61Individual Networks of Practice of EFL Learners at a Chinese University: Their Impact on English Language SocializationQi Lixia, 김정인pp. 62-78
원문 보기