토픽 모델을 이용한 비지도 요약 기법의 연구
A Study of Unsupervised Summarization Using Topic Model
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
자동 문서 요약(Automatic Document Summarization)은 문서의 중요한 내용은 유지하면서 길이가 짧은 요약문을 만들어 내는 것을 목표로 하는 연구 분야이다. 그동안 자동 문서 요약은 대용량의 데이터셋을 이용한지도 학습 기반의 심층 신경망 모델을 사용해왔다. 하지만 늘어나는 산업의 수요와는 달리 자동 요약을 위한요약 데이터셋이 여전히 부족한 실정이다. 이런 데이터 부족 문제는 요약 분야뿐만 아니라 자연어 처리 전반에 걸쳐 새로운 문제점으로 대두되고 있다. 이를 해결하기 위해 Zero-Shot Learning이나 자가 지도 학습 등의 기법이 등장했다. 이들의 공통점은 기존의 데이터에 대한 좋은 표현을 만들어 모델이 기존에 보지 못한 데이터에 대해서도 잘 다룰 수 있도록 하는것을 목표로 한다. 이에 본 논문에서는 토픽 모델을 이용한 비지도 추출 요약 모델 TES(Topic model based Extractive Summarization)에 대해 제안하고, 이에 대한 실험을 통해 기존의 비지도 추출 요약 모델들과 비슷한 성능을 기록하는 것을 확인하고, TES가 기존의 모델 대비 가지는 장점을 제시했다.
Abstract
Automatic Document Summarization is a research field that aims to create short summaries of documents while maintaining their important content. So far, document summarization has relied on supervised deep neural network models trained on large datasets. However, despite increasing demand from industry, there is still a shortage of summarization datasets. Data scarcity problem is not only a challenge for summarization but also for natural language processing in general. To address this, techniques such as Zero-Shot Learning and Self-Supervised Learning have emerged, which aim to create good representations so that models can handle unseen data well. In this paper, we propose a topic model-based unsupervised extractive summarization model called TES(Topic model based Extractive Summarization). Through experiments, we confirm that it performs similarly to existing models while suggesting its advantages over existing models.
참고문헌 (27)
- Linguistic Data Consortium, "The New York Times Annotated Corpus", Linguistic Data Consortium, ~ (2008)
- Hermann, Karl Moritz, "Teaching machines to read and comprehend", Advances in neural information processing systems, 28 (2005)
- Yangqiu Song, "Toward any-language zero-shot topic classification of textual documents", Artificial Intelligence, 274, pp.133~150 (2019), Elsevier BV[DOI]
- Ben Zhou, "Zero-Shot Open Entity Typing as Type-Compatible Grounding", Proceedings of the 2018Conference on Empirical Methods in Natural Language Processing, pp.2065~2076 (2018)
- Omer Levy, "Zero-Shot Relation Extraction via Reading Comprehension", Proceedings of the 21st Conference on Computational Natural Language Learning, pp.333~342 (2017)
- Devlin, Jacob, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, 1, pp.4171~4186 (2019)
- Mike Lewis, "BART:Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension", Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.7871~7880 (2020)
- Genta Indra Winata, "Language Models are Few-shot Multilingual Learners", Proceedings of the 1st Workshop on Multilingual Representation Learning, pp.1~15 (2021)
- David M. Blei, "Probabilistic topic models", Communications of the ACM, 55(4), pp.77~84 (2012), Association for Computing Machinery (ACM)[DOI]
- Rada Mihalcea, "TextRank: Bringing Order into Text", Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp.404~411 (2004)
- David Blei, "Latent Dirichlet Allocation", Journal of Machine Learning Research, 3, pp.993~1022 (2003)
- Akash Srivastava, "Autoencoding Variational Inference For Topic Models", International Conference on Learning Representations (2017)
- Hao Zheng, "Sentence Centrality Revisited for Unsupervised Summarization", Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.6236~6247 (2019)
- Ramesh Nallapati, "SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents", Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp.3075~3081 (2017)
- Kyunghyun Cho, "On the Properties of Neural Machine Translation: Encoder–Decoder Approaches", Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp.103~111 (2014)
- Yang Liu, "Text Summarization with Pretrained Encoders", Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp.3730~3740 (2019)
- Shusheng Xu, "Unsupervised Extractive Summarization by Pre-training Hierarchical Transformers", Findings of the Association for Computational Linguistics: EMNLP 2020, pp.1784~1795 (2020)
- Vaswani, Ashish, "Attention is All you Need", Advances in Neural Information Processing Systems, 30, pp.03762~ (2017)[DOI]
- Karl Moritz Hermann, "Teaching machines to read and comprehend", Proceedings of the 28th International Conference on Neural Information Processing Systems, pp.1693~1701 (2015)
- Christopher Manning, "The Stanford CoreNLP Natural Language Processing Toolkit", Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations (2014)
- See, Abigail, "Get To The Point:Summarization with Pointer-Generator Networks", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 1, pp.2073~1083 (2017)
- "ROUGE: A Package for Automatic Evaluation of Summaries. Text Summarization Branches Out" (2004)[URL]
- Guillaume Klein, "OpenNMT: Open-Source Toolkit for Neural Machine Translation", Proceedings of ACL 2017, System Demonstrations, pp.67~72 (2017)
- Romain Paulus, "A Deep Reinforced Model for Abstractive Summarization", 6th International Conference on Learning Representations, ICLR 2018 (2018)
- Kingma, Diederik P, "Adam: A method for stochastic optimization", arXiv preprint arXiv:1412.6980, ~ (2014)
- Thomas Wolf, "Transformers : State-of-the-Art Natural Language Processing", Proceedings of the 2020Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.38~45 (2020)
- Shashi Narayan, "Ranking Sentences for Extractive Summarization with Reinforcement Learning", Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.1747~1759 (2018)