본문 바로가기
KERD
논문KCI 등재

인공신경망 기계번역에서 디코딩 전략에 대한 연구

Study on Decoding Strategies in Neural Machine Translation

임희석|문현석|어수경|박찬준|서재형
한국융합학회 (Korea Convergence Society)|2021.11|한국융합학회논문지 (Journal of the Korea Convergence Society)|제12권 제11호|pp. 69-80|DOI https://doi.org/10.15207/JKCS.2021.12.11.069

1

조회

0

다운로드

1

인용

0

참고문헌

키워드

딥러닝Deep learningDecodingNatural language generation디코딩Neural machine translation자연어 생성Language convergence언어융합인공신경망 기계번역빔 서치Beam search
주제분류학제간연구

초록

딥러닝 모델을 활용한 인공신경망 기계번역 (Neural machine translation)이 주류 분야로 떠오르면서 최고의 성능을 위해 모델과 데이터 언어 쌍에 대한 많은 투자와 연구가 활발하게 진행되고 있다. 그러나, 최근 대부분의 인공신경망 기계번역 연구들은 번역 문장의 품질을 극대화하는 자연어 생성을 위한 디코딩 전략 (Decoding strategy)에 대해서는 미래 연구 과제로 남겨둔 채 다양한 실험과 구체적인 분석이 부족한 상황이다. 기계번역에서 디코딩 전략은 번역 문장을 생성하는 과정에서 탐색 경로를 최적화 하고, 모델 변경 및 데이터 확장 없이도 성능 개선이 가능하다. 본 논문은 시퀀스 투 시퀀스 (Sequence to Sequence) 모델을 활용한 신경망 기반의 기계번역에서 고전적인 그리디 디코딩 (Greedy decoding)부터 최신의 방법론인 Dynamic Beam Allocation (DBA)까지 비교 분석하여 디코딩 전략의 효과와 그 의의를 밝힌다.

Abstract

Neural machine translation using deep neural network has emerged as a mainstream research, and an abundance of investment and studies on model structure and parallel language pair have been actively undertaken for the best performance. However, most recent neural machine translation studies pass along decoding strategy to future work, and have insufficient a variety of experiments and specific analysis on it for generating language to maximize quality in the decoding process. In machine translation, decoding strategies optimize navigation paths in the process of generating translation sentences and performance improvement is possible without model modifications or data expansion. This paper compares and analyzes the significant effects of the decoding strategy from classical greedy decoding to the latest Dynamic Beam Allocation (DBA) in neural machine translation using a sequence to sequence model.

참고문헌 (38)

  1. P. Koehn, "Statistical phrase-based translation", Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pp.127~133 (2003)
  2. Y. Wu, "Google’s neural machine translation system : Bridging the gap betweenhuman and machine translation", arXiv preprint arXiv:1609.08144., ~ (2016)
  3. D. Bahdanau, "Neural machine translation by jointly learning to align and translate", International Conference on Learning Representations, pp.1~15 (2015)
  4. I. Sutskever, "Sequence to sequence learning with neural networks", Advances in neural information processing systems, 27, pp.3104~3112 (2014)[DOI]
  5. Park, C., "Should we find another model?: Improving Neural Machine Translation Performance with ONE-Piece Tokenization Method without Model Modification", Proceedings of the 2021Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers, pp.97~104 (2021)
  6. 문현석, "병렬 말뭉치 필터링을 적용한 Filter-mBART기반 기계번역 연구", 한국융합학회논문지, 12(5), pp.1~7 (2021), 한국융합학회[DOI]
  7. C. Park, "Quality, not quantity?: Effect of parallel corpus quantity and quality on neural machine translation", The 32st Annual Conference on Human Cognitive Language Technology, pp.363~368 (2020)
  8. A. Vaswani, "Attention is all you need", Advances in neural information processing systems, 2, pp.5998~6008 (2017)
  9. A., Conneau, "Cross-lingual language model pretraining", Advances in Neural Information Processing Systems, 32, pp.7059~7069 (2019)
  10. Yinhan Liu, "Multilingual Denoising Pre-training for Neural Machine Translation", Transactions of the Association for Computational Linguistics, 8, pp.726~742 (2020), MIT Press - Journals[DOI]
  11. B. Lowerre, "The harpy speech recognition system", pp.1976
  12. S. Wiseman, "Sequence-to-Sequence Learning as Beam-Search Optimization", Proceedings of the 2016Conference on Empirical Methods in Natural Language Processing, pp.1296~1306 (2016)
  13. M. Freitag, "Beam Search Strategies for Neural Machine Translation", Proceedings of the First Workshop on Neural Machine Translation, pp.56~60 (2017)
  14. X. Hu, "Improved beam search with constrained softmax for nmt", Proceedings of MT Summit XV, pp.297~ (2015)
  15. J. Li, "Mutual information and diverse decoding improve neural machine translation", arXiv preprint arXiv:1601.00372, ~ (2016)
  16. J. Li, "Learning to decode for future success", arXiv preprint arXiv:1701.06549., ~ (2017)
  17. R. Paulus, "A deep reinforced model for abstractive summarization", arXiv preprint arXiv:1705.04304., ~ (2017)
  18. S. Jean, "Montreal neural machine translation systems for WMT’15", Proceedings of the Tenth Workshop on Statistical Machine Translation, pp.134~140 (2015)
  19. P. Koehn, "Six Challenges for Neural Machine Translation", Proceedings of the First Workshop on Neural Machine Translation, pp.28~39 (2017)
  20. K. Murray, "Correcting Length Bias in Neural Machine Translation", Proceedings of the Third Conference on Machine Translation: Research Papers, pp.212~223 (2018)
  21. W. He, "Improved neural machine translation with SMT features", Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp.151~157 (2016)
  22. Y. Yang, "Breaking the Beam Search Curse: A Study of (Re-) Scoring Methods and Stopping Criteria for Neural Machine Translation", Proceedings of the 2018Conference on Empirical Methods in Natural Language Processing, pp.3054~3059 (2018)
  23. N. S. Keskar, "Ctrl : A conditional transformer language model for controllable generation", arXiv preprint arXiv:1909.05858., ~ (2019)
  24. A. Holtzman, "Learning to Write with Cooperative Discriminators", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 1, pp.1638~1649 (2018)
  25. Q. Huang, "Hierarchically structured reinforcement learning for topically coherent visual story generation", Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), pp.8465~8472 (2019)
  26. A. Holtzman, "The Curious Case of Neural Text Degeneration", International Conference on Learning Representations (2019)
  27. M. Post, "Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation", Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies, 1, pp.1314~1324 (2018)
  28. J. E. Hu, "Improved lexically constrained decoding for translation and monolingual rewriting", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language Technologies, 1, pp.839~850 (2019)
  29. L. Huang, "When to Finish? Optimal Beam Search for Neural Text Generation (modulo beam size)", Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp.2134~2139 (2017)
  30. R. Shu, "Improving beam search by removing monotonic constraint for neural machine translation", Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2, pp.339~344 (2018)
  31. Byte Pair encoding: A text compression scheme that accelerates pattern matching, "Y. Shibata", ~ (1999)
  32. T. Kudo, "SentencePiece:A simple and language independent subword tokenizer and detokenizer for Neural Text Processing", Proceedings of the 2018Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.66~71 (2018)
  33. I. Provilkov, "BPE-Dropout: Simple and Effective Subword Regularization", Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.1882~1892 (2020)
  34. C. Hokamp, "Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search", Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 1, pp.1535~1546 (2017)
  35. 박찬준, "공공 한영 병렬 말뭉치를 이용한 기계번역 성능 향상 연구", 디지털융복합연구, 18(6), pp.271~277 (2020), 한국디지털정책학회[DOI]
  36. K. Papineni, "Bleu: a method for automatic evaluation of machine translation", Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp.311~318 (2002)
  37. Chanjun Park, "Decoding Strategies for Improving Low-Resource Machine Translation", Electronics, 9(10), pp.1562~ (2020), MDPI AG[DOI]
  38. C. Park, "BTS : Back TranScription for Speech-to-Text Post-Processor using Text-to-Speech-to-Text", Proceedings of the 8th Workshop on Asian Translation, pp.106~116 (2021)

관련 논문AI추천

신경망기계번역 기술 진화와 번역품질 분석김희동, 지인영통번역연구소 · 통번역학연구2020 · 제24권 제3호인공신경망 기계번역에서 말뭉치 간의 균형성을 고려한 성능 향상 연구문현석, 박기남, 박찬준, 어수경, 임희석한국융합학회 · 한국융합학회논문지2021 · 제12권 제5호심층학습을 이용한 기계번역 기술과 정확도 연구김희동, 지인영국제언어인문학회 · 인문언어2017 · 제19권 제2호영어자원문법을 활용한 신경망 기계번역의 데이터 증강과 성능 평가송상헌, 왕규현언어정보연구소 · 언어와 정보 사회2021 · 제42권 공공 한영 병렬 말뭉치를 이용한 기계번역 성능 향상 연구박찬준, 임희석한국디지털정책학회 · 디지털융복합연구2020 · 제18권 제6호신경망기계번역의 객관적 평가를 위한 예비연구: 자동평가와 수동평가의 균형점이준호통번역연구소 · 통번역학연구2019 · 제23권 제3호

이 간행물의 다른 논문 (52건)

심리학적 언어분석 프로그램 개발을 위한 융합연구: 기존 프로그램의 비교와 관련 문헌의 동향 분석김영준, 최원일, 김태훈pp. 1-18AMI 사업에 민간투자를 유인하기 위한 우크라이나 배전서비스 요금정책 연구김철년pp. 19-26자기조직화지도를 통한 아파트 가격의 패턴 분석유재필, 이지영pp. 27-33Zero-Shot 기반 기계번역 품질 예측 연구문현석, 임희석, 서재형, 박찬준, 어수경pp. 35-43딥러닝 기반의 Multi Scale Attention을 적용한 개선된 Pyramid Scene Parsing Network김준혁, 한현호, 이상훈pp. 45-51
원문 보기