병렬 말뭉치 필터링을 적용한 Filter-mBART기반 기계번역 연구
Filter-mBART Based Neural Machine Translation Using Parallel Corpus Filtering
1
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
최신 기계번역 연구 동향을 살펴보면 대용량의 단일말뭉치를 통해 모델의 사전학습을 거친 후 병렬 말뭉치로 미세조정을 진행한다. 많은 연구에서 사전학습 단계에 이용되는 데이터의 양을 늘리는 추세이나, 기계번역 성능 향상을 위해 반드시 데이터의 양을 늘려야 한다고는 보기 어렵다. 본 연구에서는 병렬 말뭉치 필터링을 활용한 mBART 모델 기반의 실험을 통해, 더 적은 양의 데이터라도 고품질의 데이터라면 더 좋은 기계번역 성능을 낼 수 있음을 보인다. 실험결과 병렬 말뭉치 필터링을 거친 사전학습모델이 그렇지 않은 모델보다 더 좋은 성능을 보였다. 본 실험결과를 통해 데이터의 양보다 데이터의 질을 고려하는 것이 중요함을 보이고, 해당 프로세스를 통해 추후 말뭉치 구축에 있어 하나의 가이드라인으로 활용될 수 있음을 보였다.
Abstract
In the latest trend of machine translation research, the model is pretrained through a large mono lingual corpus and then finetuned with a parallel corpus. Although many studies tend to increase the amount of data used in the pretraining stage, it is hard to say that the amount of data must be increased to improve machine translation performance. In this study, through an experiment based on the mBART model using parallel corpus filtering, we propose that high quality data can yield better machine translation performance, even utilizing smaller amount of data. We propose that it is important to consider the quality of data rather than the amount of data, and it can be used as a guideline for building a training corpus.
참고문헌 (17)
- A. Vaswani, "Attention is all you need", arXiv preprint arXiv:1706.03762., ~ (2017)
- J. Devlin, "Bert : Pre-training of deep bidirectional transformers for language understanding", arXiv preprint arXiv:1810.04805., ~ (2018)
- G. Lample, "Cross-lingual language model pretraining", arXiv preprint arXiv:1901.07291., ~ (2019)
- K. Song, "Mass : Masked sequence to sequence pre-training for language generation", arXiv preprint arXiv:1905.02450., ~ (2019)
- M. Lewis, "Bart : Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension", arXiv preprint arXiv:1910.13461., ~ (2019)
- 박찬준, "공공 한영 병렬 말뭉치를 이용한 기계번역 성능 향상 연구", 디지털융복합연구, 18(6), pp.271~277 (2020), 한국디지털정책학회[DOI]
- H. Khayrallah, "On the impact of various types of noise on neural machine translation", arXiv preprint arXiv:1805.12282, ~ (2018)
- P. Koehn, "Findings of the WMT 2020 shared task on parallel corpus filtering and alignment", Proceedings of the Fifth Conference on Machine Translation, pp.726~742 (2020)
- Yinhan Liu, "Multilingual Denoising Pre-training for Neural Machine Translation", Transactions of the Association for Computational Linguistics, 8, pp.726~742 (2020), MIT Press - Journals[DOI]
- Mandar Joshi, "SpanBERT: Improving Pre-training by Representing and Predicting Spans", Transactions of the Association for Computational Linguistics, 8, pp.64~77 (2020), MIT Press - Journals[DOI]
- C. Park, "Quality, not Quantity? : Effect of parallel corpus quantity and quality on Neural Machine Translation", The 32st Annual Conference on Human Cog-nitive Language Technology (2020)
- W. A. Gale, "A program for aligning sentences in bilingual corpora", Computational linguistics, 19(1), pp.75~102 (1993)
- M. Cettolo, "Overview of the iwslt 2017evaluation campaign", International Workshop on Spoken Language Translation, pp.2~14 (2017)
- M. Ott, "fairseq : A fast, extensible toolkit for sequence modeling", arXiv preprint arXiv:1904.01038, ~ (2019)
- T. Kudo, "Sentencepiece : A simple and language independent subword tokenizer and detokenizer for neural text processing", arXiv preprint arXiv:1808.06226, ~ (2018)
- K. Papineni, "Bleu: a method for automatic evaluation of machine translation", Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp.311~318 (2002)
- C. Park, "Decoding strategies for improving low-resource machine translation", Electronics, 9(10), pp.1562~ (2020)