The quality and productivity of post-edited machine translation in comparison with human translation: A case of Korean-English legal translation
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This paper addresses the quality and productivity issues in post-edited machine translation, analyzing Korean-English legal translation outputs produced in two ways: human translation and post-edited machine translation. For this study, two sets of translated texts were produced by professional legal translators and student translators. The quality of the raw machine translation by DeepL was assessed through automatic and manual evaluations, and its scores were compared with those of human and post-edited machine translation outputs. To investigate the productivity of post-editing, we analyzed the temporal and technical efforts involved in human translation as well as post-editing. The analysis indicated the superiority of post-edited machine translation both in terms of quality and productivity. Compared with human translation, post-editing improved the average number of processed words by about 20%, while significantly reducing technical efforts. Correction rates demonstrated that the major errors, which are related to accuracy, fluency, and terminology, in the machine translation output were effectively addressed through post-editing. Despite the limitations of this small-scale study, the findings suggest that post-editing has the potential to efficiently elevate the machine translation output quality to a level that surpasses that of human translation, and thus facilitates translation of legal texts of medium difficulty.
Abstract
This paper addresses the quality and productivity issues in post-edited machine translation, analyzing Korean-English legal translation outputs produced in two ways: human translation and post-edited machine translation. For this study, two sets of translated texts were produced by professional legal translators and student translators. The quality of the raw machine translation by DeepL was assessed through automatic and manual evaluations, and its scores were compared with those of human and post-edited machine translation outputs. To investigate the productivity of post-editing, we analyzed the temporal and technical efforts involved in human translation as well as post-editing. The analysis indicated the superiority of post-edited machine translation both in terms of quality and productivity. Compared with human translation, post-editing improved the average number of processed words by about 20%, while significantly reducing technical efforts. Correction rates demonstrated that the major errors, which are related to accuracy, fluency, and terminology, in the machine translation output were effectively addressed through post-editing. Despite the limitations of this small-scale study, the findings suggest that post-editing has the potential to efficiently elevate the machine translation output quality to a level that surpasses that of human translation, and thus facilitates translation of legal texts of medium difficulty.
참고문헌 (36)
- Aranberri, N., "Comparison of post-editing productivity between professional translators and lay users", Proceedings of the 11th Conference of the Association for Machine Translation in the Americas, pp.20~33 (2014), Association for Machine Translation in the Americas
- Carl, M., "The process of post-editing : A pilot study", Copenhagen Studies in Language, 41(1), pp.131~142 (2011)
- Carl, M., "Translog-II: A program for recording user activity data for empirical reading and writing research", Paper presented at the eighth international conference on language resources and evaluation (LREC12) (2012)
- Daems, J., "The impact of machine translation error types on post-editing effort indicators", Proceedings of the 4th Workshop on Post-Editing Technology and Practice, pp.31~45 (2015), Association for Machine Translation in the Americas
- Dames, J., "Translationsese and post-editese : How comparable is comparable quality", Linguistica Antverpiensia, 16, pp.89~103 (2017)
- Depraetere, I., "Post-editing of Machine Translation: Processes and Applications", pp.78~108 (2014), Cambridge Scholars Publishing
- Fiedere, R\., "Quality and machine translation : A realistic objective", The Journal of Specialised Translation, 11, pp.52~74 (2009)
- García, I., "Is machine translation ready yet", Target, 22(1), pp.7~21 (2010)
- García, I., "Translating by post-editing : Is it the way forward", Machine Translation, 25, pp.217~237 (2011)
- Green, S., "The efficacy of human post-editing for language translation", Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp.439~448 (2013), Association for Computing Machinery
- Guerberof Arenas, A., "Productivity and quality in MT post-editing", MT Summit XII-Workshop: Beyond Translation Memories: New Tools for Translators Workshop, pp.26~30 (2009), Association for Computational Linguistics
- Guerberof Arenas, A, "The Bloomsbury Companion to Language Industry Studies", pp.333~360 (2020), Bloomsbury Academic
- Jia, Y., "How does the post-editing of neural machine translation compare with from-scratch translation? A product and process study", The Journal of Specialised Translation, 31, pp.60~86 (2019)
- Jung, J., "Accuracy errors remaining uncorrected in post-edited output", Interpreting and Translation Studies, 27(3), pp.177~201 (2023)
- Killman, J., "PE vs Translating in the legal context : Quality and time effects from English to Spanish", Revista de Llengua I Diret, 2022, pp.78, 56~72
- Kim, J., "An investigation into temporal, technical, and web-searching efforts during Korean-to-English post-editing process compared with from-scratch translation", Interpreting and Translation Studies, 26(2), pp.1~24 (2022)
- Kim, J., "An investigation into correction of accuracy errors in post-editing", The Journal of Translation Studies, 23(5), pp.91~116 (2022)
- Kim, S. M., "A university-industry joint study on machine translation post-editing productivity and MT engine error rate", The Journal of Translation Studies, 20(1), pp.41~76 (2019)
- Koponen, M., "Post-editing quality : Analysing the correctness and necessity of post-editor corrections", Linguistica Antverpiensia, 16, pp.137~148 (2017)
- Krings, H, "Repairing Texts : Empirical Investigations of Machine Translation Post-editing Processes" (2001), Kent State University Press
- Lee, J. H., "A pilot investigation into Korean to English post-editing efforts", The Journal of Translation Studies, 22(5), pp.271~298 (2021)
- Lee, J. H., "Can anybody perform Korean to English post-editing tasks? A pilot study for mtpe module design", The Journal of Translation Studies, 22(1), pp.171~198 (2021)
- Lee, J. H., "A study on performance evaluation of a specialized machine translation engine in the legal domain : Focusing on the Korean-to-English translation of legal contracts", T&I Review, 12(1), pp.162~192 (2022)
- Lee, J. H., "An investigation into English to Korean full post-editing : Factors affecting productivity of full post-editing", The Journal of Translation Studies, 23(5), pp.119~146 (2022)
- Lee, J., "A case study on the evaluation of Korean-English legal translations by generic and custom neural machine translation engines", Interpretation and Translation, 25(1), pp.75~98 (2023)
- Lee, J., "An analysis of errors in human translation, machine translation and post-edited machine translation outputs : A case of Korean-English legal translation", The Journal of Translation Studies, 25(1), pp.11~39 (2024)
- Lee, S. W., "Comparing human translation and post-edited machine translation : A case study of Korean undergraduate students", T&I Review, 11(2), pp.101~123 (2021)
- O’Brien, S., "The Handbook of Translation and Cognition", pp.311~331 (2017), John Wiley & Sons
- Popović, M., "Relations between different types of post-editing operations, cognitive effort and temporal effort", Proceedings of the Seventeenth Annual Conference of the European Association for Machine Translation, pp.191~198 (2014), European Association for Machine Translation
- "Machine vs. Human translation: When to use which for legal translation" (2022)[URL]
- Şahin, M., "Translation testing and evaluation : A study on methods and needs", Translation & Interpreting, 6(2), pp.67~90 (2014)
- Screen, B., "Machine translation and Welsh : Analysing free statistical machine translation for the professional translation of an under-researched language pair", The Journal of Specialised Translation, 28, pp.317~344 (2017)
- Snover, M., "A study of translation edit rate with targeted human annotation", Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pp.223~231 (2006), Association for Machine Translation in the Americas
- Temnikova, I., "Cognitive evaluation approach for a controlled language post-editing experiment", Proceedings of the 7th International Conference on Language Resources and Evaluation, pp.3485~3490 (2010), European Language Resources Association
- Terribile, S., "Is post-editing really faster than human translation?", Translation Spaces (2023)[DOI]
- Vieira, N. L., "Translation in Transition: Between cognition, computing and technology", pp.161~187 (2017), European Language Resources Association