Comparing ChatGPT 4o and Gemini 2.5 Pro in ESP Speaking Assessment
4
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
As English continues to function as a global lingua franca, achieving a single native-like pronunciation standard has become increasingly unrealistic. Since most English interactions now occur between non-native speakers, communicative effectiveness and mutual intelligibility are more essential than phonetic accuracy. This research investigates how large language models (LLMs) can support meaning-centered speaking assessment in global English and ESP contexts. Three hypotheses were tested: (1) ChatGPT 4o delivers significantly lower Word Error Rate (WER) than Gemini 2.5 Pro in transcribing non-native speech; (2) learners perceive a meaning-centered evaluation form as more valuable than a pronunciation-centered one; and (3) learners rate ChatGPT 4o’s feedback as clearer and more helpful. Twenty adult L2 learners, working professionals from diverse industries in Korea, participated to improve workplace communication and real-life fluency. Results revealed a substantial difference in transcription accuracy, with ChatGPT 4o achieving a 4.19% WER compared to Gemini 2.5 Pro’s 34.49%. Even though the study found no significant difference between meaning-centered and pronunciation-centered evaluation forms, the results demonstrate the potential of AI-driven speaking assessment to enhance clarity, accessibility, and communicative effectiveness for adult learners in both general and ESP contexts.
Abstract
As English continues to function as a global lingua franca, achieving a single native-like pronunciation standard has become increasingly unrealistic. Since most English interactions now occur between non-native speakers, communicative effectiveness and mutual intelligibility are more essential than phonetic accuracy. This research investigates how large language models (LLMs) can support meaning-centered speaking assessment in global English and ESP contexts. Three hypotheses were tested: (1) ChatGPT 4o delivers significantly lower Word Error Rate (WER) than Gemini 2.5 Pro in transcribing non-native speech; (2) learners perceive a meaning-centered evaluation form as more valuable than a pronunciation-centered one; and (3) learners rate ChatGPT 4o’s feedback as clearer and more helpful. Twenty adult L2 learners, working professionals from diverse industries in Korea, participated to improve workplace communication and real-life fluency. Results revealed a substantial difference in transcription accuracy, with ChatGPT 4o achieving a 4.19% WER compared to Gemini 2.5 Pro’s 34.49%. Even though the study found no significant difference between meaning-centered and pronunciation-centered evaluation forms, the results demonstrate the potential of AI-driven speaking assessment to enhance clarity, accessibility, and communicative effectiveness for adult learners in both general and ESP contexts.