본문 바로가기
KERD
논문KCI 등재

The Effect of Voice-enabled ChatGPT on Speaking Proficiency, Affective and Metacognitive Domains of Korean EFL Learners

2

조회

0

다운로드

1

인용

0

참고문헌

키워드

초록

AI chatbots have opened new possibilities for interactive and meaningful communication in foreign language learning. However, the effectiveness of AI chatbots in enhancing L2 speaking abilities remains unexplored. Furthermore, unlike existing chatbots, ChatGPT offers personalized instruction, feedback, and interactive conversations, holding high potential for L2 learning. Therefore, this study compared voice-enabled ChatGPT to peer practice for 33 Korean Grade 1 middle school students concerning overall speaking proficiency, sub-skills (vocabulary, grammar, fluency, pronunciation, and task completion), and affective/metacognitive aspects. Participants were randomly assigned to either the experimental group utilizing ChatGPT or the control group with peer interactions. Results revealed that the ChatGPT group significantly outperformed the group that interacted with their peers in L2 speaking proficiency. When it came to sub-skills, the gains were significantly higher in the ChatGPT group’s vocabulary. Moreover, the ChatGPT group showed significantly more improvement in all affective/metacognitive domains, while the control group only showed changes in confidence and motivation. The ChatGPT group showed positive changes in reduced anxiety and willingness to communicate in the classroom, significantly more than those in the peer interaction group. These findings highlight the efficacy of incorporating ChatGPT into L2 speaking classrooms and bring forward pedagogical implications for L2 acquisition

Abstract

AI chatbots have opened new possibilities for interactive and meaningful communication in foreign language learning. However, the effectiveness of AI chatbots in enhancing L2 speaking abilities remains unexplored. Furthermore, unlike existing chatbots, ChatGPT offers personalized instruction, feedback, and interactive conversations, holding high potential for L2 learning. Therefore, this study compared voice-enabled ChatGPT to peer practice for 33 Korean Grade 1 middle school students concerning overall speaking proficiency, sub-skills (vocabulary, grammar, fluency, pronunciation, and task completion), and affective/metacognitive aspects. Participants were randomly assigned to either the experimental group utilizing ChatGPT or the control group with peer interactions. Results revealed that the ChatGPT group significantly outperformed the group that interacted with their peers in L2 speaking proficiency. When it came to sub-skills, the gains were significantly higher in the ChatGPT group’s vocabulary. Moreover, the ChatGPT group showed significantly more improvement in all affective/metacognitive domains, while the control group only showed changes in confidence and motivation. The ChatGPT group showed positive changes in reduced anxiety and willingness to communicate in the classroom, significantly more than those in the peer interaction group. These findings highlight the efficacy of incorporating ChatGPT into L2 speaking classrooms and bring forward pedagogical implications for L2 acquisition

참고문헌 (67)

  1. Abdelmageed, M., "Digital storytelling enhances students' speaking skills at Zewail university of science and technology in Egypt", Paper presented at the Society for Information Technology &Teacher Education International Conference (2018)
  2. Abdullah, M., "ChatGPT: Fundamentals, applications and social impacts", Paper Presented at the 9th International Conference on Social Networks Analysis (2022)
  3. Adiguzel, T., "Revolutionizing education with AI : Exploring the transformative potential of ChatGPT", Contemporary Educational Technology, 15(3), pp.1~13 (2023)[DOI]
  4. Al-Abdullatif, A. M., "Implementing the Bashayer chatbot in Saudi higher education : Measuring the influence on students'motivation and learning strategies", Frontiers in Psychology, 14, pp.1~16 (2023)[DOI]
  5. Alrabai, F., "A model of foreign language anxiety in the Saudi EFL context", English Language Teaching, 7(7), pp.82~101 (2014)[DOI]
  6. Ayedoun, E., "A conversational agent to encourage willingness to communicate in the context of English as a foreign language", Procedia Computer Science, 60, pp.1433~1442 (2015)[DOI]
  7. Ayedoun, E., "Adding communicative and affective strategies to an embodied conversational agent to enhance second language learners’ willingness to communicate", International Journal of Artificial Intelligence in Education, 29(1), pp.29~57 (2019)[DOI]
  8. Ayedoun, E., "Toward personalized scaffolding and fading of motivational support in L2 learner–dialogue agent interactions : An exploratory study", IEEE Transactions on Learning Technologies, 13(3), pp.604~616 (2020)[DOI]
  9. Bao, M., "Can home use of speech-enabled artificial intelligence mitigate foreign language anxiety – Investigation of a concept", Arab World English Journal, 5, pp.28~40 (2019)[DOI]
  10. Belda-Medina, J., "Using chatbots as AI conversational partners in language learning", Applied Sciences, 12(17), pp.1~16 (2022)[DOI]
  11. Çakmak, F., "Chatbot-human interaction and its effects on EFL students' L2 speaking performance and anxiety", Novitas-ROYAL(Research on Youth and Language), 16(2), pp.113~131 (2022)
  12. Chen, H. H., "Investigating college EFL learners’ perceptions toward the use of Google assistant for foreign language learning", Interactive Learning Environments, 31(3), pp.1335~1350 (2023)[DOI]
  13. Chen, J. C., "The interplay of tasks, strategies and negotiations in second life", Computer Assisted Language Learning, 31(8), pp.960~986 (2018)[DOI]
  14. Chen, M. A., "AI chatbot-supported grammar tense learning on EFL students learning effectiveness", Paper presented at the 17th International Technology, Education and Development Conference (2023)
  15. Chen, X., "Artificial intelligence-assisted personalized language learning:Systematic review and co-citation analysis", Paper presented at 2021 International Conference on Advanced Learning Technologies (2021)
  16. Dizon, G., "Using intelligent personal assistants for second language learning : A case study of Alexa", TESOL Journal, 8(4), pp.811~830 (2017)[DOI]
  17. Dizon, G., "Evaluating intelligent personal assistants for L2 listening and speaking development", Language Learning & Technology, 24(1), pp.16~26 (2020)
  18. Dizon, G., "Intelligent personal assistants for autonomous second language learning : An investigation of Alexa", The JALT CALL Journal, 16(2), pp.107~120 (2020)[DOI]
  19. Duffy, M. C., "Motivation matters : Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system", Computers in Human Behavior, 52, pp.338~348 (2015)[DOI]
  20. El Shazly, R., "Effects of artificial intelligence on English speaking anxiety and speaking performance : A case study", Expert Systems, 38(3), pp.1~15 (2021)[DOI]
  21. Fryer, L., "Bots as language learning tools", Language Learning & Technology, 10(3), pp.8~14 (2006)
  22. Goda, Y., "Conversation with a chatbot before an online EFL group discussion and the effects on critical thinking", The Journal of Information and Systems in Education, 13(1), pp.1~7 (2014)[DOI]
  23. Gonulal, T., "Investigating EFL learners’ humorous interactions with an intelligent personal assistant", Interactive Learning Environments, pp.1~14 (2021)[DOI]
  24. 한다은, "The Effects of Voice-based AI Chatbots on Korean EFL Middle School Students’ Speaking Competence and Affective Domains", 아시아태평양융합연구교류논문지, 6(7), pp.71~80 (2020), 사단법인 한국융합기술연구학회[DOI]
  25. Hapsari, I. P., "AI Chatbots learning model in english speaking skill: Alleviating speaking anxiety, boosting enjoyment, and fostering critical thinking", Innovative technologies and learning. ICITL 2022, pp.444~453 (2022), Springer
  26. Hew, K. F., "Using chatbots to support student goal setting and social presence in fully online activities : Learner engagement and perceptions", Journal of Computing in Higher Education, 35(1), pp.40~68 (2022)[DOI]
  27. Hinton, P. R., "SPSS Explained" (2004), Routledge
  28. Hsu, H., "Investigating the impact of the Amazon Alexa on the development of L2listening and speaking skills", Interactive Learning Environments, pp.1~14 (2021)[DOI]
  29. Hsu, M. H., "Proposing a taskoriented chatbot system for EFL learners speaking practice", Interactive Learning Environments, pp.1~12 (2021)[DOI]
  30. Hwang, W., "Facilitating authentic contextual EFL speaking and conversation with smart mechanisms and investigating its influence on learning achievements", Computer Assisted Language Learning, pp.1~27 (2022)[DOI]
  31. Jeon, J., "Exploring a self-directed interactive app for informal EFL learning : A self-determination theory perspective", Education and Information Technologies, 27(4), pp.5767~5787 (2022)[DOI]
  32. 정남숙, "The Effects of Task Type and Group Structure on Meaning Negotiation in Synchronous CMC", 멀티미디어 언어교육, 11(3), pp.9~28 (2008), 한국멀티미디어언어교육학회[DOI]
  33. Junaidi, J., "Artificial intelligence in EFL Context : Rising students' speaking performance with lyra virtual assistance", International Journal of Advanced Science and Technology, 29(5), pp.6735~6741 (2020)
  34. Hea-Suk Kim, "Is It Beneficial to Use AI Chatbots to Improve Learners’ Speaking Performance?", Journal of Asia TEFL, 18(1), pp.161~178 (2021), 아시아테플
  35. Hea-Suk Kim, "Effects of AI Chatbots on EFL Students’ Communication Skills", 영어학, 21, pp.712~734 (2021), 한국영어학회[DOI]
  36. 김정옥, "Ongoing Speaking Anxiety of Korean EFL Learners: Case Study of a TOEIC Intensive Program", Journal of Asia TEFL, 15(1), pp.17~31 (2018), 아시아테플
  37. 김나영, "Effects of Different Types of Chatbots on EFL Learners' Speaking Competence and Learner Perception", 비교문화연구, 48, pp.223~252 (2017), 비교문화연구소[DOI]
  38. Koraishi, O., "Teaching English in the age of AI : Embracing ChatGPT to optimize EFL materials and assessment", Language Education and Technology, 3(1), pp.55~72 (2023)
  39. Landis, J. R., "The measurement of observer agreement for categorical data", Biometrics, 33(1), pp.159~174 (1977)[DOI]
  40. 임경진, "Effects of culturally responsive teaching with technology on Korean EFL learners' cultural awareness and speaking performance", 박사, 영어교육학과, pp.2013
  41. Yac Hao Liu, "A Study of Korean EFL Learners` WTC and Motivation", Journal of Pan-Pacific Association of Applied Linguistics, 16(2), pp.35~58 (2012), 범태평양 응용언어학회
  42. Mahmoud, R. H., "Implementing AI-based conversational chatbots in EFL speaking classes: An evolutionary perspective", Research Square, 1~21 (2022)
  43. Moussalli, S., "Are commercial ‘personal robots’ ready for language learning? Focus on second language speech", CALL communities and culture – short papers from EUROCALL 2016, pp.325~329 (2016), Research-publishing.net
  44. Oliver, R., "Multiple perspectiveson interaction: Second language research in honor of Susan M. Gass", pp.135~156 (2009), Routledge
  45. Oscoz, A., "Jigsaw and free discussion in synchronous computer-mediated communication (SCMC)", doctoral, pp.2003
  46. Panda, S., "Exploring the viability of ChatGPT as an alternative to traditional chatbot systems in library and information centers", Library Hi Tech News, 40(3), pp.22~25 (2023)[DOI]
  47. Park, G. P., "Factor analysis of the foreign language classroom anxiety scale in Korean learners of English as a foreign language", Psychological Reports, 115(1), pp.261~275 (2014)[DOI]
  48. Pica, T., "Contemporary approaches to second language acquisition", pp.49~70 (2013), John Benjamins
  49. Pica, T., "Tasks and second language learning", pp.9~34 (1993), Multilingual Matters
  50. Pyun, D. O., "Impact of affective variables on Korean as a foreign language learners' oral achievement", System, 47, pp.53~63 (2014)[DOI]
  51. Roever, C., "What scores from monologic speaking tests can(not)tell us about interactional competence", Language Testing, 39(1), pp.7~29 (2022)[DOI]
  52. Ruan, S. S., "Smart tutoring through conversational interfaces", doctoral, pp.2021
  53. "Native, non-native, or bilingual? A concise assessment of ChatGPT’s suitability for second-language instruction as a native or non-native pedagogue" (2023)[URL]
  54. Mapping the redesigned TOEIC Bridge®test scores to proficiency levels of the common european framework of reference for languages, "Schmidgall, J.", ETS, ~, pp.research memorandum No. RM-21-01 (2021)
  55. Shanti Manipuspika, Y., "Correlation between anxiety and willingness to communicate in the Indonesian EFL context", Arab World English Journal, 9(2), pp.200~217 (2018)[DOI]
  56. Sulistyowati, T., "Developing and validating the metacognitive awareness speaking questionnaire", Advances in Social Sciences, 624, pp.210~217 (2022)
  57. Tai, T. Y., "Effects of intelligent personal assistants on EFL learners’ oral proficiency outside the classroom", Computer Assisted Language Learning, 35, pp.1~30 (2022)[DOI]
  58. Tai, T. Y., "The impact of Google assistant on adolescent EFL learners’ willingness to communicate", Interactive Learning Environments, 31(3), pp.1485~1502 (2020)[DOI]
  59. Tai, T. Y., "The impact of intelligent personal assistants on adolescent EFL learners’ speaking proficiency", Computer Assisted Language Learning, pp.1~28 (2022)[DOI]
  60. Underwood, J., "Exploring AI language assistants with primary EFL students", CALL in a climate of change: Adapting to turbulent global conditions - short papers from EUROCALL 2017, pp.317~321 (2017), Research-publishing.net
  61. Wang, Y., "Using learning analytics to understand the design of an intelligent language tutor –chatbot lucy", Journal of Advanced Computer Science and Applications, 4(11), pp.124~131 (2013)[DOI]
  62. Winkler, R., "Unleashing the potential of chatbots in education: A state-of-the-art analysis", Paper presented at Academy of Management Annual Meeting (2018)
  63. Yang, H., "Implementation of an AI chatbot as an English conversation partner in EFL speaking classes", ReCALL, 34(3), pp.327~343 (2022)[DOI]
  64. Yeh, H., "Speaking progress and meaning negotiation processes in synchronous online tutoring", System, 81, pp.179~191 (2019)[DOI]
  65. Zhai, C., "A systematic review on cross-culture, humor and empathy dimensions in conversational chatbots : The case of second language acquisition", Heliyon, 8(12), pp.1~31 (2022)[DOI]
  66. Zhai, X., "ChatGPT user experience : Implications for education", SSRN Electronic Journal, pp.1~10 (2022)[DOI]
  67. Zhang, W., "Understanding individual differences in metacognitive strategy use, task demand, and performance in integrated L2 speaking assessment tasks", Frontiers in Psychology, 13, pp.1~13 (2022)[DOI]

관련 논문AI추천

이 간행물의 다른 논문 (5건)