언어 정보가 반영된 문장 점수를 활용하는삭제 기반 문장 압축

Abstract

Sentence compression is a natural language processing task that generates concise sentences that preserves the important meaning ofthe original sentence. For grammatically appropriate sentence compression, early studies utilized human-defined linguistic rules. Furthermore, while the sequence-to-sequence models perform well on various natural language processing tasks, such as machine translation,there have been studies that utilize it for sentence compression. However, for the linguistic rule-based studies, all rules have to be definedby human, and for the sequence-to-sequence model based studies require a large amount of parallel data for model training. In orderto address these challenges, Deleter, a sentence compression model that leverages a pre-trained language model BERT, is proposed. Becausethe Deleter utilizes perplexity based score computed over BERT to compress sentences, any linguistic rules and parallel dataset is not requiredfor sentence compression. However, because Deleter compresses sentences only considering perplexity, it does not compress sentencesby reflecting the linguistic information of the words in the sentences. Furthermore, since the dataset used for pre-learning BERT are farfrom compressed sentences, there is a problem that this can lad to incorrect sentence compression. In order to address these problems,this paper proposes a method to quantify the importance of linguistic information and reflect it in perplexity-based sentence scoring. Furthermore, by fine-tuning BERT with a corpus of news articles that often contain proper nouns and often omit the unnecessary modifiers,we allow BERT to measure the perplexity appropriate for sentence compression. The evaluations on the English and Korean dataset confirmthat the sentence compression performance of sentence-scoring based models can be improved by utilizing the proposed method.