Digital enhancement of pronunciation assessment: Automated speech recognition and human raters
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This study explores the potential of automated speech recognition (ASR) in assessing English learners’ pronunciation. We employed ASR technology, acknowledged for its impartiality and consistent results, to analyze speech audio files, including synthesized speech, both native-like English and Korean-accented English, and speech recordings from a native English speaker. Through this analysis, we establish baseline values for the word error rate (WER). These were then compared with those obtained for human raters in perception experiments that assessed the speech productions of 30 first-year college students before and after taking a pronunciation course. Our sub-group analyses revealed positive training effects for Whisper, an ASR tool, and human raters, and identified distinct human rater strategies in different assessment aspects, such as proficiency, intelligibility, accuracy, and comprehensibility, that were not observed in ASR. Despite such challenges as recognizing accented speech traits, our findings suggest that digital tools such as ASR can streamline the pronunciation assessment process. With ongoing advancements in ASR technology, its potential as not only an assessment aid but also a self-directed learning tool for pronunciation feedback merits further exploration.
Abstract
This study explores the potential of automated speech recognition (ASR) in assessing English learners’ pronunciation. We employed ASR technology, acknowledged for its impartiality and consistent results, to analyze speech audio files, including synthesized speech, both native-like English and Korean-accented English, and speech recordings from a native English speaker. Through this analysis, we establish baseline values for the word error rate (WER). These were then compared with those obtained for human raters in perception experiments that assessed the speech productions of 30 first-year college students before and after taking a pronunciation course. Our sub-group analyses revealed positive training effects for Whisper, an ASR tool, and human raters, and identified distinct human rater strategies in different assessment aspects, such as proficiency, intelligibility, accuracy, and comprehensibility, that were not observed in ASR. Despite such challenges as recognizing accented speech traits, our findings suggest that digital tools such as ASR can streamline the pronunciation assessment process. With ongoing advancements in ASR technology, its potential as not only an assessment aid but also a self-directed learning tool for pronunciation feedback merits further exploration.
참고문헌 (30)
- Abercrombie, D., "Teaching pronunciation", English Language Teaching Journal, 3(5), pp.113~122 (1949)
- Archibald, J., "Second language phonology" (1998), John Benjamins
- Baralt, M., "Using wordles to teach foreign language writing", Language Learning & Technology, 15(2), pp.12~22 (2011)
- (2023)[URL]
- Brown, A., "Some thoughts on intelligibility", The English Teacher, 18, pp.1~16 (1989)
- Carey, M. D., "Does a rater’s familiarity with a candidate’s pronunciation affect the rating in oral proficiency interviews", Language Testing, 28(2), pp.201~219 (2011)
- Dale, P., "English pronunciation made simple" (2005), Pearson Education
- Derwing, T. M., "Pronunciation fundamentals: Evidence-based perspectives for L2 teaching and research. Amsterdam" (2015), John Benjamins
- Derwing, T. M., "Does popular speech recognition software work with ESL speech", TESOL Quarterly, 34(3), pp.592~603 (2000)
- Derwing, T. M., "Teaching native speakers to listen to foreign-accented speech", Journal of Multilingual and Multicultural Development, 23(4), pp.245~259 (2002)
- Fairbanks, G., "Voice and articulation drillbook" (1960), Harper & Row
- Flege, J. E., "Effects of age of second-language learning on the production of English consonants", Speech Communication, 16(1), pp.1~26 (1995)
- Hincks, R., "Speech technologies for pronunciation feedback and evaluation", ReCALL, 15(1), pp.3~20 (2003)
- 홍연정, "Evaluating Score Reliability of Automatic English Pronunciation Assessment System for Education", 외국어교육연구, 35(1), pp.91~104 (2021), 외국어교육연구소[DOI]
- Jenkins, J., "The phonology of English as an international language" (2000), Oxford University Press
- Kissling, E. M., "Teaching pronunciation : Is explicit phonetics instruction beneficial for FL learners", The Modern Language Journal, 97(3), pp.720~744 (2013)
- Lado, R., "Linguistics across cultures: Applied Linguistics for Language Teachers" (1957), University of Michigan Press
- Lado, R., "Language testing: The construction and use of foreign language tests" (1961), Longman
- Levenshtein, V. I., "Binary codes capable of correcting deletions, insertions, and reversals", Soviet Physics-Doklady, 10(8), pp.707~710 (1966)
- Levis, J., "Assessing speech intelligibility: Experts listen to two students", Proceedings of the 2nd Pronunciation in Second Language Learning and Teaching Conference, pp.56~69 (2010), Iowa State University
- Munro, M. J., "Intelligibility: Buzzword or buzzworthy?", Proceedings of the 2nd Pronunciation in Second Language Learning and Teaching Conference, pp.7~16 (2010), Iowa State University
- (2023)[URL]
- 박아영, "The Study on Automatic Speech Recognizer Utilizing Mobile Platform on Korean EFL Learners’ Pronunciation Development", 디지털콘텐츠학회논문지, 18(6), pp.1101~1107 (2017), 한국디지털콘텐츠학회[DOI]
- 박혜빈, "비원어민 교수자 음성모델을 이용한 자동발음평가 시스템", 한국인터넷방송통신학회 논문지, 16(2), pp.131~136 (2016), 한국인터넷방송통신학회[DOI]
- Purcell, E. T., "Predictors of pronunciation accuracy : A reexamination", Language Learning, 30(2), pp.271~287 (1980)
- Schulz, K. U., "Fast string correction with Levenshtein automata", International Journal of Document Analysis and Recognition, 5(1), pp.67~85 (2002)
- Spring, R., "The role of ASR training in EFL pronunciation improvement : An in-depth look at the impact of treatment length and guided practice on specific pronunciation points", Computer Assisted Language Learning Electronic Journal, 23(3), pp.163~185 (2022)
- Van Rossum, G., "Python 3 reference manual" (2009), CreateSpace
- Winke, P., "Raters’ L2 background as a potential source of bias in rating oral performance", Language Testing, 30(2), pp.231~252 (2013)
- 양병곤, "An evaluation of Korean students' pronunciation of an English passage by a speech recognition application and two human raters", 말소리와 음성과학, 12(4), pp.19~25 (2020), 한국음성학회[DOI]