인공신경망 특허 기계번역 성능에 관한 연구 - Patent Translate와 WIPO Translate 한영 번역 결과물의 누락과 통사 오류 분석을 중심으로 -
A study on the quality of patent neural machine translation: A comparison of omission and syntactic errors in the Korean-English translations by patent-specialized Patent Translate and WIPO Translate.
0
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
Jieun Lee and Hyoeun Choi (2022). A study on the quality of patent neural machine translation: A comparison of omission and syntactic errors in the Korean-English translations by patent-specialized Patent Translate and WIPO Translate. This paper aims to evaluate the quality of patent translations produced by the two patent-specialized machine translation engines, EPO’s Patent Translate and WIPO’s WIPO Translate. For manual evaluation, four experienced patent translators or patent translation service managers evaluated the quality of 106 English sentences from the translations of 30 Korean patent abstracts by the two MT engines. In the automatic evaluation, Patent Translate slightly outperformed WIPO Translate, whereas in the manual evaluation WIPO Translate outperformed Patent Translate. According to the error annotations provided by the evaluators, WIPO Translate produced more omission errors than Patent Translate but handled the complex syntax of the source text better, while Patent Translate produced more syntactic errors than WIPO Translate. The results indicate that in automatic evaluation, MT outputs with fewer omissions were rated higher, while in manual evaluation, comprehensible and accurate syntactic structures appeared to determine the overall quality evaluation. (Ewha Womans University, Korea)
Abstract
Jieun Lee and Hyoeun Choi (2022). A study on the quality of patent neural machine translation: A comparison of omission and syntactic errors in the Korean-English translations by patent-specialized Patent Translate and WIPO Translate. This paper aims to evaluate the quality of patent translations produced by the two patent-specialized machine translation engines, EPO’s Patent Translate and WIPO’s WIPO Translate. For manual evaluation, four experienced patent translators or patent translation service managers evaluated the quality of 106 English sentences from the translations of 30 Korean patent abstracts by the two MT engines. In the automatic evaluation, Patent Translate slightly outperformed WIPO Translate, whereas in the manual evaluation WIPO Translate outperformed Patent Translate. According to the error annotations provided by the evaluators, WIPO Translate produced more omission errors than Patent Translate but handled the complex syntax of the source text better, while Patent Translate produced more syntactic errors than WIPO Translate. The results indicate that in automatic evaluation, MT outputs with fewer omissions were rated higher, while in manual evaluation, comprehensible and accurate syntactic structures appeared to determine the overall quality evaluation. (Ewha Womans University, Korea)
참고문헌 (44)
- 강병규, "신경망 기계번역의 작동 원리와 번역의 정확률 - 중한 번역을 실례로", 중어중문학, (73), pp.253~295 (2018), 한국중어중문학회[DOI]
- 곽중철, "포스트에디팅 측정지표를 통한 기계번역 오류 유형화 연구", 통번역학연구, 22(1), pp.1~25 (2018), 통번역연구소[DOI]
- 김동미, "의약품 설명서의 인공지능 번역에 대한 전략연구", 인문사회 21, 9(6), pp.533~544 (2018), 인문사회 21[DOI]
- 정혜연, "번역자동평가에서 풀리지 않은 과제", 번역학연구, 21(1), pp.9~29 (2020), 한국번역학회[DOI]
- 박건영, "정보성 텍스트의 한영 기계번역 포스트에디팅 가이드라인 제시—신경망 기계번역(NMT)을 사용한 뉴스 기사문 번역의 사례", 번역학연구, 22(1), pp.109~137 (2021), 한국번역학회[DOI]
- 전종섭, "웹기반 영한 번역을 위한 중간 언어의 효율성 연구", 통번역학연구, 17(3), pp.201~229 (2013), 통번역연구소[DOI]
- 서보현, "기계번역 결과물의 오류유형 고찰", 번역학연구, 19(1), pp.99~117 (2018), 한국번역학회[DOI]
- 이상빈, "기계번역에 관한 KCI 연구논문 리뷰: 인문학 저널 논문(2011∼2020년 초)의 논의내용과 연구방법을 중심으로", 통역과 번역, 22(2), pp.75~104 (2020), 한국통역번역학회[DOI]
- 이준호, "신경망기계번역의 객관적 평가를 위한 예비연구: 자동평가와 수동평가의 균형점", 통번역학연구, 23(3), pp.171~202 (2019), 통번역연구소[DOI]
- 정혜연, "번역의 자동평가: 기계번역 평가를 인간번역 평가에 적용해보기", 통번역학연구, 22(4), pp.265~287 (2018), 통번역연구소[DOI]
- 최효은, "한영 특허 번역 품질 평가 연구", 박사, 통역번역학과, pp.2016
- 최효은, "특허 기계번역 결과물의 평가 - KIPRIS의 무료 한영 기계번역을 중심으로", 통역과 번역, 19(1), pp.139~178 (2017), 한국통역번역학회[DOI]
- 한승희, "기계번역 품질에 대한 요인분석", 통번역학연구, 25(2), pp.147~170 (2021), 통번역연구소[DOI]
- 한현희, "한-노 기계 번역, 어디까지 왔나?:Google과 Papago 번역 성능 비교를 기반으로", 노어노문학, 32(3), pp.63~94 (2020), 한국노어노문학회
- Bazrafshan, M., "Semantic Features for Statistical Machine Translation", Doctor, pp.2014
- Brkić, M., "Automatic and human evaluation on English-Croatian legislative test set", Computer Science, 7816, pp.311~378 (2013)
- Castilho, S., "Is neural machine translation the new state of the art?", The Prague Bulletin of Mathematical Linguistics, 108, pp.109~120 (2017)
- Castilho, S., "Translation Quality Assessment: From Principles to Practice", pp.9~38 (2018), Springer
- Chatzikoumi, E., "How to evaluate machine translation : A review of automated and human metrics", Natural Language Engineering, 26, pp.137~161 (2020)
- Costa A., "A linguistically motivated taxonomy for Machine Translation error analysis", Machine Translation, 29, pp.127~161 (2015)
- Coughlin, D, "Correlating automated and human assessments of machine translation quality", Paper presented at MT Summit Ⅸ (2003)
- Doherty, S., "Human Issues in Translation Technology", pp.131~148 (2017), Routledge
- Doherty, S., "The Routledge Handbook of translation and technology", pp.339~353 (2019), Routledge
- Dorr, B., "Handbook of Natural Language Processing and Machine Translation", pp.745~843 (2011), Springer
- Gonzàlez, M., "An open toolkit for automatic machine translation (meta-)evaluation. Technical Manual, version 3.0", TALP Research Center, LSI Department, Universitat Politècnica de Catalunya, ~ (2014)
- Giménez, J, "Linguistic measures for automatic machine translation evaluation", Machine Translation, 24, pp.209~240 (2010)
- Klubička, F., "Quantitative fine-grained human evaluation of machine translation systems : A case study on English to Croatian", Machine Translation, 32, pp.195~215 (2018)
- "A new look for Google Translate on the web" (2018)[URL]
- Lavie, A., "Automated metrics for mt evaluation", Machine Translation, 11, pp.731~ (2013)
- Lavie, A., "Meteor, m-bleu and m-ter: Evaluation metrics for high-correlation with human rankings of machine translation output", Paper presented at the Third Workshop on Statistical Machine Translation (2007)
- Läubli, S., "Has machine translation achieved human parity? A case for document-level evaluation", Paper presented at the 2018 Conference on Empirical Methods in Natural Language Processing (2018)
- Lommel, A., "Translation Quality Assessment: From Principles to Practice", pp.109~128 (2018), Springer
- Nurminen, M., "Raw machine translation use by patent professionals : A case of distributed cognition", Translation, Cognition & Behavior, 3(1), pp.100~121 (2020)
- Olohan, M., "Scientific and Technical Translation" (2015), Routledge
- Papineni, K., "BLEU: A method for automatic evaluation of machine translation", Paper presented at the 40th Annual Meeting of the Association for Computational Linguistics (2002)
- Poliquen, "Full-text patent translation at WIPO: Scalability, quality and usability", Paper presented at MT Summit XV: The Sixth Workshop on Patent and Scientific Literature Translation (PSLT6) (2015)
- Poliquen, B, "WIPO Translate: Patent neural machine translation publicly available in 10languages", Paper presented at the Seventh Workshop on Patent and Scientific Literature Translation (2017)
- Popovic, M., "Translation Quality Assessment:From Principles to Practice", pp.129~158 (2018), Springer
- Premoli, V., "MTPE in patents: A successful business story", Paper presented at the MT Summit XVII (2019)
- Rossi, L., "Applicability and application of machine translation quality metrics in the patent field", World Patent Information, 35, pp.115~125 (2013)
- Tsai, Y., "Linguistic evaluation of translation errors in Chinese–English machine translations of patent titles", Forum, 15(1), pp.142~156 (2017)
- "Manual of patent examining procedure. USPTO United States Patent and Trademark" (2018)[URL]
- Wu, Y., "Google's neural machine translation system: Bridging the gap between human and machine translation" (2016)
- Ying, C., "Errors of machine translation of terminology in the patent text from English into Chinese", ASP Transactions on Comuputers, 1(1), pp.12~17 (2021)
