자동화된 기계학습(AutoML)을 활용한 특허 특화 번역엔진의 영한번역 성능 평가
Evaluation of Patent English-Korean Machine Translations by a Patent-Specific NMT Engine Using AutoML
0
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
This paper compares the quality of English-Korean patent translations by a patent-specific NMT engine trained using AutoML with the general Google Translate. The evaluation was based on both automatic and human evaluations of the Korean translations of 200 English patent sentences excerpted from a number of semiconductor patent gazettes. In automatic evaluation, BLEU scores showed that the patent-specific NMT engine significantly outperformed Google Translate. Human evaluation, carried out by sampling as well as error detection and correction analysis, confirmed the results of automatic evaluation, revealing that patent-specific NMT results were better than Google Translate results. In the error detection and correction analysis, Google Translate had more major errors than patent-specific NMT. Moreover, most errors in Google Translate were addressed in the patent-specific NMT, while errors in the patent-specific NMT still remained in Google Translate. In the sampling analysis, shorter sentences and longer sentences were sampled and analyzed. According to the results, both patent-specific NMT and Google Translate showed better performance in translating shorter sentences. In translating longer sentences, both translation engines exhibited accuracy-related errors and syntactic errors, though patent-specific NMT slightly outperformed Google Translate. Overall, translation results by patent-specific NMT showed better quality than those by Google Translate.
Abstract
This paper compares the quality of English-Korean patent translations by a patent-specific NMT engine trained using AutoML with the general Google Translate. The evaluation was based on both automatic and human evaluations of the Korean translations of 200 English patent sentences excerpted from a number of semiconductor patent gazettes. In automatic evaluation, BLEU scores showed that the patent-specific NMT engine significantly outperformed Google Translate. Human evaluation, carried out by sampling as well as error detection and correction analysis, confirmed the results of automatic evaluation, revealing that patent-specific NMT results were better than Google Translate results. In the error detection and correction analysis, Google Translate had more major errors than patent-specific NMT. Moreover, most errors in Google Translate were addressed in the patent-specific NMT, while errors in the patent-specific NMT still remained in Google Translate. In the sampling analysis, shorter sentences and longer sentences were sampled and analyzed. According to the results, both patent-specific NMT and Google Translate showed better performance in translating shorter sentences. In translating longer sentences, both translation engines exhibited accuracy-related errors and syntactic errors, though patent-specific NMT slightly outperformed Google Translate. Overall, translation results by patent-specific NMT showed better quality than those by Google Translate.
참고문헌 (19)
- 김세린, "도메인 지식에 특화된 신경망 기계번역", 한국정보과학회 학술발표논문집, pp.678~680 (2022)
- 김정희, "한국어 자모 단위 구성과 높임말을 반영한 한영 신경 기계 번역", 정보과학회논문지, 49(11), pp.1017~1025 (2022), 한국정보과학회[DOI]
- 박찬준, "Coronavirus Disease-19(COVID-19)에 특화된 인공신경망 기계번역기", 한국융합학회논문지, 11(9), pp.7~13 (2020), 한국융합학회[DOI]
- 박찬준, "공공 한영 병렬 말뭉치를 이용한 기계번역 성능 향상 연구", 디지털융복합연구, 18(6), pp.271~277 (2020), 한국디지털정책학회[DOI]
- 이준호, "법률 특화 번역엔진 성능 평가 - 한영 계약서 번역을 중심으로 -", T&I REVIEW, 12(1), pp.169~192 (2022), 통역번역연구소[DOI]
- 이지은, "인공신경망 기반 맞춤형 기계번역엔진의 성능 평가: 법률 및 특허 한영번역 결과물 평가 사례를 중심으로", 번역학연구, 24(1), pp.9~37 (2023), 한국번역학회[DOI]
- 정영준, "MASS와 상대 위치 표현을 이용한 영어-한국어 신경망 기계 번역", 정보과학회논문지, 47(11), pp.1038~1043 (2020), 한국정보과학회[DOI]
- 최승권, "영어 특허문서 자동번역을 위한 특허번역패턴 연구", 번역학연구, 8(1), pp.301~322 (2007), 한국번역학회[DOI]
- 최효은, "특허 기계번역 결과물의 평가 - KIPRIS의 무료 한영 기계번역을 중심으로", 통역과 번역, 19(1), pp.139~178 (2017), 한국통역번역학회[DOI]
- Castilho, Sheila, "Is Neural Machine Translation the New State of the Art?", The Prague Bulletin of Mathematical Linguistics, 108, pp.109~120 (2017)
- Etchegoyhen, Thierry, "Evaluating Domain Adaptation in Machine Translation Across Scenarios", Proceedings of the Eleventh International Conference on Language Resources Evaluation (LREC 2018), pp.6~15 (2018)
- Guo, Xinze, "Kingsoft s Neural Machine Translation System for WMT19", Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), pp.196~202 (2019)
- Kinoshita, Satoshi, "Comparison of SMT and NMT Trained with Large Patent Corpora: Japio at WAT2017", Proceedings of the 4th Workshop on Asian Translation, pp.140~145 (2017)
- Rossi, Laura, "Applicability and Application of Machine Translation Quality Metrics in the Patent Field", World Patent Information, 35, pp.115~125 (2013)
- Tsai, Yvonne, "Linguistic Evaluation of Translation Errors in Chinese–English Machine Translations of Patent Titles", Forum, 15(1), pp.142~156 (2017)
- Wan, Yu, "Challenges of Neural Machine Translation for Short Texts", Computational Linguistics, 48(2), pp.321~342 (2022)
- Wang, Xu, "Domain-specific Machine Translation with Recurrent Neural Network for Software Localization", Empirical Software Engineering, 24, pp.3514~3545 (2019)
- "Executive Summary: PCT Yearly Review 2022" (2022)[URL]
- "글로벌 IP번역 품질 경쟁 속, 최고의 IP번역 마스터 가려졌다" (2022), 특허뉴스[URL]