심층 신경망을 활용한 진료 기록 문헌에서의 종단형 개체명 및 관계 추출 비교 연구 - 파이프라인 모델과 결합 모델을 중심으로 -
A Comparative Research on End-to-End Clinical Entity and Relation Extraction using Deep Neural Networks: Pipeline vs. Joint Models
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
정보추출은 문헌 내에 존재하는 개체명을 인식함과 동시에 이들 간의 의미적 관계까지도 식별하여 최종적으로 문헌 내에 포함된 의미적 트리플을 자동으로 추출하여 활용할 수 있으므로 문헌에 대한 심층적인 분석과 이해에 많은 도움을 줄 수 있다. 그러나 지금까지 대부분의 정보추출에 대한 연구는 개체명 인식과 관계추출이 개별 연구로 각각 분리되어 진행되었으며, 그 결과 입력 문헌에 대한 정보추출의 최종 출력인 의미적 트리플 추출 성능에 대한 객관적이고 정확한 평가가 제대로 이루어지지 않았다. 이에 본 논문에서는 진료 기록 문헌에 나타나는 개체명과 그들 간의 관계를 트리플 형태로 직접 추출할 수 있는 종단형 정보추출의 2가지 모델인 파이프라인 및 결합형 모델을 구축하는 구체적인 방법론을 제시하고 성능 비교 실험을 진행하였다. 우선 파이프라인 모델은 양방향 GRU-CRFs를 활용한 개체명 인식 모듈과 다중 인코딩 기반 관계추출 모듈로 구현되었고, 결합형 모델을 위해서는 다중 헤드 레이블링 기반의 양방향 GRU-CRFs이 적용되었다. 두 가지 시스템을 바탕으로 진료기록 문헌 내의 개체명과 관계를 모두 태깅하여 구축된 i2b2/VA 2010 데이터셋을 활용한 비교 실험에서 파이프라인 모델의 성능이 5.5%(F-measure) 더 높게 나타났다. 추가적으로, 대규모 신경망 언어모델과 수작업으로 구축된 자질 정보를 활용한 최고 수준의 기존 시스템과의 비교 실험을 통해, 본 논문에서 구현한 종단형 모델의 객관적인 성능 수준을 파악할 수 있었다.
Abstract
Information extraction can facilitate the intensive analysis of documents by providing semantic triples which consist of named entities and their relations recognized in the texts. However, most of the research so far has been carried out separately for named entity recognition and relation extraction as individual studies, and as a result, the effective performance evaluation of the entire information extraction systems was not performed properly. This paper introduces two models of end-to-end information extraction that can extract various entity names in clinical records and their relationships in the form of semantic triples, namely pipeline and joint models and compares their performances in depth. The pipeline model consists of an entity recognition sub-system based on bidirectional GRU-CRFs and a relation extraction module using multiple encoding scheme, whereas the joint model was implemented with a single bidirectional GRU-CRFs equipped with multi-head labeling method. In the experiments using i2b2/VA 2010, the performance of the pipeline model was 5.5% (F-measure) higher. In addition, through a comparative experiment with existing state-of-the-art systems using large-scale neural language models and manually constructed features, the objective performance level of the end-to-end models implemented in this paper could be identified properly.
참고문헌 (38)
- Bekoulis, G., "Adversarial training for multi-context joint entity and relation extraction", ArXiv:1808.06876, ~ (2018)
- Bekoulis, G., "Joint entity recognition and relation extraction as a multi-head selection problem", Expert Systems with Applications, 114, pp.34~45 (2018)
- End-to-end Relation Extraction using Neural Networks and Markov Logic Networks, "Bhattacharyya, P.", EACL, ~ (2017)
- Chiu, J. P., "Named entity recognition with bidirectional LSTM-CNNs", Transactions of the Association for Computational Linguistics, 4, pp.357~370 (2016)
- "2014", ~
- Choi, S. -P., "An intensive case study on kernel-based relation extraction", Multimedia Tools and Applications, 71(2), pp.741~767 (2014)
- "2014", ~
- Collobert, R., "Natural language processing(almost)from scratch", Journal of Machine Learning Research, 12, pp.2493~2537 (2011)
- Dang, T. H., "D3NER: Biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information", Bioinformatics (Oxford, England), 34(20), pp.3539~3546 (2018)
- "2016", ~
- Du, C., "Text classification research with attention-based recurrent neural networks", International Journal of Computers Communications & Control, 13(1), pp.50~61 (2018)
- Fang, X., "Joint extraction of clinical entities and relations using multi-head selection method", 2021 International Conference on Asian Language Processing (IALP), pp.99~104 (2021)
- "2019", ~
- Gupta, P, "Table filling multi-task recurrent neural network for joint entity and relation extraction", Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp.2537~2547 (2016)
- "2015", ~
- Ji, G., "Distant supervision for relation extraction with sentence-level attention and entity descriptions", Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp.3060~3066 (2017)
- Ji, Z., "A discrete joint model for entity and relation extraction from clinical notes", AMIA Summits on Translational Science Proceedings, pp.315~324 (2021)
- Kate, R. J., "Joint entity and relation extraction using card-pyramid parsing", Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pp.203~212 (2010)
- 김선우, "생의학 분야 학술 문헌에서의 이벤트 추출을 위한 심층 학습 모델 구조 비교 분석 연구", 한국문헌정보학회지, 51(4), pp.77~97 (2017), 한국문헌정보학회[DOI]
- "2016", ~
- Li, F., "A neural joint model for entity and relation extraction from biomedical text", BMC Bioinformatics, 18(1), pp.198~ (2017)
- Li, Q., "Incremental joint extraction of entity mentions and relations", Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp.402~412 (2014)
- Li, Z., "Integrating shortest dependency path and sentence sequence into a deep learning framework for relation extraction in clinical text", BMC Medical Informatics and Decision Making, 19(1), pp.22~ (2019)
- he Stanford CoreNLP natural language processing toolkit, "Manning, C. D.", Association for Computational Linguistics (ACL) System Demonstrations, 55~60 (2014)
- Misawa, S., "Character-based Bidirectional LSTM-CRF with words and characters for Japanese Named Entity Recognition", Proceedings of the First Workshop on Subword and Character Level Models in NLP, pp.97~102 (2017)
- Miwa, M., "End-to-end relation extraction using LSTMs on sequences and tree structures", ArXiv Preprint ArXiv:1601.00770, ~ (2016)
- Miwa, M., "Modeling joint entity and relation extraction with table representation", Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.1858~1869 (2014)
- Mollá, D., "Named entity recognition for question answering", Proceedings of the 2006 Australasian Language Technology Workshop, pp.51~58 (2006)
- "2019", 729~738
- Nguyen, N. T., "Wide-coverage relation extraction from MEDLINE using deep syntax", BMC Bioinformatics, 16(1), pp.107~ (2015)
- Nguyen, T. H., "Relation Extraction: Perspective from Convolutional Neural Networks", Proceedings of NAACL Workshop on Vector Space Modeling for NLP (2015)
- Unanue, I. J., "Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition", Journal of Biomedical Informatics, 76, pp.102~109 (2017)
- "2016", ~
- Yang, Z., "Hierarchical attention networks for document classification", Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.1480~1489 (2016)
- Zhang, J., "ME-MD: An effective framework for neural machine translation with multiple encoders and decoders", Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp.3392~3398 (2017)
- Meishan Zhang, "End-to-End Neural Relation Extraction with Global Optimization", Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (2017), Association for Computational Linguistics[DOI]
- Zheng, S., "Joint entity and relation extraction based on a hybrid neural network", Neurocomputing, 257, pp.59~66 (2017)
- "2017", ~