Document Summarization Model Based on General Context in RNN
0
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
In recent years, automatic document summarization has been widely studied in the field of natural languageprocessing thanks to the remarkable developments made using deep learning models. To decode a word,existing models for abstractive summarization usually represent the context of a document using the weightedhidden states of each input word when they decode it. Because the weights change at each decoding step, theseweights reflect only the local context of a document. Therefore, it is difficult to generate a summary that reflectsthe overall context of a document. To solve this problem, we introduce the notion of a general context andpropose a model for summarization based on it. The general context reflects overall context of the documentthat is independent of each decoding step. Experimental results using the CNN/Daily Mail dataset show thatthe proposed model outperforms existing models.
Abstract
In recent years, automatic document summarization has been widely studied in the field of natural languageprocessing thanks to the remarkable developments made using deep learning models. To decode a word,existing models for abstractive summarization usually represent the context of a document using the weightedhidden states of each input word when they decode it. Because the weights change at each decoding step, theseweights reflect only the local context of a document. Therefore, it is difficult to generate a summary that reflectsthe overall context of a document. To solve this problem, we introduce the notion of a general context andpropose a model for summarization based on it. The general context reflects overall context of the documentthat is independent of each decoding step. Experimental results using the CNN/Daily Mail dataset show thatthe proposed model outperforms existing models.
참고문헌 (20)
- D. R. Radev, "Introduction to the special issue on summarization", Computational linguistics, 28(4), pp.399~408 (2002)
- K. Hong, "A repository of state of the art and competitive baseline summaries for generic news summarization", Proceedings of the 9th International Conference on Language Resources and Evaluation, pp.1608~1616 (2014)
- G. Erkan, "Lexrank: graph-based lexical centrality as salience in text summarization", Journal of Artificial Intelligence Research, 22(1), pp.457~479 (2004)
- I. Sutskever, "Sequence to sequence learning with neural networks", Proceedings of the 27th International Conference on Neural Information Processing Systems, pp.3104~3112 (2014)
- S. Chopra, "Abstractive sentence summarization with attentive recurrent neural networks", Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.93~98 (2016)
- R. Nallapati, "Abstractive text summarization using sequence-tosequence RNNs and beyond", Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pp.280~290 (2016)
- R. Nallapati, "SummaRuNNer: a recurrent neural network based sequence model for extractive summarization of documents", Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp.3075~3081 (2017)
- D. Bahdanau, "Neural machine translation by jointly learning to align and translate", Proceedings of the 3rd International Conference on Learning Representations (2015)
- M. T. Luong, "Effective approaches to attention-based neural machine translation", Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp.1412~1421 (2015)
- O. Vinyals, "Pointer networks", Proceedings of the 28th International Conference on Neural Information Processing Systems, pp.2692~2700 (2015)
- C. Gulcehre, "Pointing the unknown words", Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp.140~149 (2016)
- A. See, "Get to the point: summarization with pointer-generator networks", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp.1073~1083 (2017)
- K. M. Hermann, "Teaching machines to read and comprehend", Proceedings of the 28th International Conference on Neural Information Processing Systems, pp.1693~1701 (2015)
- S. Hochreiter, "Long short-term memory", Neural Computation, 9(8), pp.1735~1780 (1997)
- K. Cho, "Learning phrase representations using RNN encoder-decoder for statistical machine translation", Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp.1724~1734 (2014)
- Z. Tu, "Modeling coverage for neural machine translation", Proceedings of the 54th Annual meeting of the Association for Computational Linguistics, pp.76~85 (2016)
- R. Paulus, "A deep reinforced model for abstractive summarization", Proceedings of the 6th International Conference on Learning Representations (2018)
- V. Anderson, "Teaching students to summarize", Educational Leadership, 46(4), pp.26~28 (1988)
- J. Duchi, "Adaptive subgradient methods for online learning and stochastic optimization", Journal of Machine Learning Research, 12, pp.2121~2159 (2011)
- C. Y. Lin, "Rouge: a package for automatic evaluation of summaries", Proceedings of the Workshop on Text Summarization Branches Out, pp.74~81 (2004)