본문 바로가기
KERD
논문KCI 등재

A Meta-analysis of the Effects of Using AI Chatbot in Korean EFL Education

0

조회

0

다운로드

13

인용

0

참고문헌

키워드

주제분류영어와문학

초록

The main purpose of this study is to perform a meta-analysis from a macro perspective, synthesizing the results of experimental research for the use of AI chatbots on educational purposes and practices in EFL environments. From the analysis of the results of 58 individual research cases in 16 experimental trials, EFL classes employing chatbot technology showed a mean effect size of .689, which is near to a large effect size. The following are the key findings in terms of moderators (e.g., school level, publication kind, treatment period, chatbot type, interaction method, and device type). The results show that 1) the lower the school level, the larger the effect size was calculated, 2) treatment that lasted fewer than 8 weeks proved to be more beneficial, and 3) the effect of using a purpose-built chatbot was greater than that of a general-purpose chatbot. On the other hand, the effect sizes calculated for dependent variables regarding linguistic competence and affective domains showed values greater than the medium effect size. The overall outcomes of this study shed light on why AI chatbots have become popular in an EFL setting as a learning and teaching tool to support a set of English capabilities and communication skills. This research has confirmed how AI chatbots may improve the quality, equity, and efficiency of education, as well as where they can optimize their potential and achieve their goals. Based on the findings, this study further suggests valuable insights into future AI chatbot-based research and education.

Abstract

The main purpose of this study is to perform a meta-analysis from a macro perspective, synthesizing the results of experimental research for the use of AI chatbots on educational purposes and practices in EFL environments. From the analysis of the results of 58 individual research cases in 16 experimental trials, EFL classes employing chatbot technology showed a mean effect size of .689, which is near to a large effect size. The following are the key findings in terms of moderators (e.g., school level, publication kind, treatment period, chatbot type, interaction method, and device type). The results show that 1) the lower the school level, the larger the effect size was calculated, 2) treatment that lasted fewer than 8 weeks proved to be more beneficial, and 3) the effect of using a purpose-built chatbot was greater than that of a general-purpose chatbot. On the other hand, the effect sizes calculated for dependent variables regarding linguistic competence and affective domains showed values greater than the medium effect size. The overall outcomes of this study shed light on why AI chatbots have become popular in an EFL setting as a learning and teaching tool to support a set of English capabilities and communication skills. This research has confirmed how AI chatbots may improve the quality, equity, and efficiency of education, as well as where they can optimize their potential and achieve their goals. Based on the findings, this study further suggests valuable insights into future AI chatbot-based research and education.

참고문헌 (63)

  1. Adamopoulou, E., "Chatbots: History, technology, and applications", Machine Learning with Applications, 2(1), pp.1~18 (2020)
  2. 배경진, "챗봇을 이용한 토론 수업 보조 도구 개발", 현대영미어문학, 38(1), pp.169~190 (2020), 현대영미어문학회[DOI]
  3. Bansal, H., "A review paper on human computer interaction", International Journal of Advanced Research in Computer Science and Software Engineering, 8(4), pp.53~56 (2018)
  4. Bao, M, "Can home use of speech-enabled artificial intelligence mitigate foreign language anxiety: Investigation of a concept", Arab World English Journal, 5, pp.28~40 (2019)
  5. "IBM Watson: The inside story of how the Jeopardy-winning supercomputer was born, and what it wants to do next" (2013)[URL]
  6. Borenstein, M., "Introduction to meta-analysis" (2009), John Wiley & Sons
  7. 최원경, "AI 챗봇을 활용한 초등영어 과정중심 말하기 평가: 가능성과 한계", 초등영어교육, 26(1), pp.131~152 (2020), 한국초등영어교육학회[DOI]
  8. 추성엽, "영어 상호작용 촉진을 위한 과업 기반 AI 챗봇 활용 및 학생 발화 분석", 초등영어교육, 25(2), pp.27~52 (2019), 한국초등영어교육학회[DOI]
  9. 추성엽, "자연어 툴킷 및 AI 챗봇을 활용한초등영어 어휘평가 자동화 알고리즘 개발 연구", 초등영어교육, 26(2), pp.55~80 (2020), 한국초등영어교육학회[DOI]
  10. "Putting the human back into the equation: Personalization and AI-driven digital banking experiences" (2021)[URL]
  11. Fryer, L. K., "Bots as language learning tools", Language Learning &Technology, 10, pp.8~14 (2006)
  12. Fryer, K. L., "Chatbot learning partners: Connecting learning experiences, interest and competence", Computers in Human Behavior, 93, pp.279~289 (2019)
  13. Gabrielli, S., "A chatbot-based coaching intervention for adolescents to promote life skills: Pilot study", JMIR Human Factors, 7(1), pp.1~7 (2020)
  14. Goda, Y., "Conversation with a Chatbot before an Online EFL Group Discussion and the Effects on Critical Thinking", Journal of Information and Systems in Education, 13(1), pp.1~7 (2014)
  15. Han, D. G., "AI Speaking with Google" (2020), Crown
  16. 한다은, "The Effects of Voice-based AI Chatbots on Korean EFL Middle School Students’ Speaking Competence and Affective Domains", 아시아태평양융합연구교류논문지, 6(7), pp.71~80 (2020), 사단법인 한국융합기술연구학회[DOI]
  17. Han, D. E., "Effects of AI chatbot on Korean EFL learners’ speaking ability and affective factors", doctor, pp.2020
  18. Harmer, J., "The practice of English language teaching" (2015), Pearson Education
  19. Hobert, S., "Say hello to your new automated tutor: A structured literature review on pedagogical conversational agents", Proceedings of the 14th International Conference, pp.301~314, Wirtschaftsinformatik
  20. 황요한, "AI 기술을 활용한 영어교육의 가능성: 영어 예비교사들의 인공지능 챗봇 사용과 개발을 중심으로", 멀티미디어 언어교육, 24(1), pp.104~133 (2021), 한국멀티미디어언어교육학회[DOI]
  21. 현주은, "영어학습도구로서 인공지능 스피커의 분석 및 시사점", 영어영문학, 24(1), pp.201~219 (2019), 미래영어영문학회[DOI]
  22. Kerly, A., "CALMsystem: A Conversational Agent for Learner Modelling", Knowledge-Based System, 21(3), pp.238~246 (2008)
  23. Khan, R., "Standardized architecture for conversational agents a.k.a. chatbots", International Journal of Computer Trends and Technology, 50, pp.114~121 (2017)
  24. Kim, H. S., "Application and creation of an AI chatbot for English learning" (2019), Kyoyookbook
  25. 김혜숙, "Impact of Mobile Interactions With AI on Writing Performance", 현대영어교육, 21(2), pp.1~13 (2020), 현대영어교육학회[DOI]
  26. Hea-Suk Kim, "Is It Beneficial to Use AI Chatbots to Improve Learners’ Speaking Performance?", Journal of Asia TEFL, 18(1), pp.161~178 (2021), 아시아테플
  27. 김인석, "인공지능형 대화형 챗봇 현황과 영어교육용 챗봇 개발 방안", 영어영문학21, 33(2), pp.73~92 (2020), 21세기영어영문학회[DOI]
  28. Kim, J. S., "The effects of human-AI assistant interactions on children’s collaborative language acquisition", MA, pp.2017
  29. 김나영, "Effects of Voice Chat on EFL Learners’ Speaking Ability according to Proficiency Levels", 멀티미디어 언어교육, 19(4), pp.63~88 (2016), 한국멀티미디어언어교육학회[DOI]
  30. 김나영, "Effects of Different Types of Chatbots on EFL Learners' Speaking Competence and Learner Perception", 비교문화연구, 48, pp.223~252 (2017), 비교문화연구소[DOI]
  31. 김나영, "A Study on Chatbots for Developing Korean College Students' English Listening and Reading Skills", 디지털융복합연구, 16(8), pp.19~26 (2018), 한국디지털정책학회[DOI]
  32. 김나영, "Chatbots and Korean EFL Students' English Vocabulary Learning", 디지털융복합연구, 16(2), pp.1~7 (2018), 한국디지털정책학회[DOI]
  33. 김나영, "Effect of Text Chat on EFL Writing Fluency, Accuracy, and Complexity by Interlocutors", Foreign Languages Education, 25(1), pp.27~54 (2018), 한국외국어교육학회[DOI]
  34. 김나영, "Different Chat Modes of a Chatbot and EFL Students’ Writing Skills Development", 외국어교육연구, 32(1), pp.263~290 (2018), 외국어교육연구소[DOI]
  35. 김나영, "A Study on the Use of Artificial Intelligence Chatbots for Improving English Grammar Skills", 디지털융복합연구, 17(8), pp.37~46 (2019), 한국디지털정책학회[DOI]
  36. 김성우, "인공지능기술 활용 대화형 영어 학습 앱을 적용한 초등영어 말하기 수업 분석: 학습자 몰입 및 수업에 대한 학생과 교사의 인식을 중심으로", 초등영어교육, 26(3), pp.177~202 (2020), 한국초등영어교육학회[DOI]
  37. 김소연, "AI 챗봇을 도입한 초등학교 영어 디지털 교과서 활용 수업의 정의적 효과", 학습자중심교과교육연구, 21(10), pp.37~49 (2021), 학습자중심교과교육학회
  38. 김태웅, "인공지능 채팅로봇인 채터봇을 활용한 실시간 온라인 채팅수업방법과 컴퓨터 흥미도의 교수-학습적 영향 분석", 공학교육연구, 11(3), pp.19~33 (2008), 한국공학교육학회[DOI]
  39. 이동한, "인공지능 기반 음성로봇을 활용한 영어 말하기 학습 시스템 개발 방안", 영상영어교육, 20(1), pp.189~211 (2019), 영상영어교육학회[DOI]
  40. 이장호, "외국어학습을 위한 대화형 챗봇의 담화 분석을 통한 개선 방안 연구", 멀티미디어 언어교육, 22(1), pp.132~153 (2019), 한국멀티미디어언어교육학회[DOI]
  41. 이제영, "코퍼스 기반 국내 CALL 연구의 동향: 메타분석을 통한 효과 분석", 멀티미디어 언어교육, 15(3), pp.83~111 (2012), 한국멀티미디어언어교육학회[DOI]
  42. 이제영, "Effects of Using Subtitles of Video Contents on L2 Learners’ Listening and Vocabulary Development: A Meta-Analysis", 응용언어학, 33(2), pp.137~158 (2017), 한국응용언어학회
  43. 이제영, "스토리텔링의 활용이 영어 능력 향상에 미치는 영향: 메타분석", 언어과학, 28(1), pp.137~161 (2021), 한국언어과학회[DOI]
  44. 이제영, "중등학교 영어 학습부진학생 지도의 효과: 메타분석", Studies in English Education, 22(4), pp.251~278 (2017), 글로벌영어교육학회[DOI]
  45. 이삭, "게이미피케이션 기반 AI 챗봇 활용 수업이 초등학생의 영어 말하기 수행 및 정의적 영역에 미치는 영향", 초등영어교육, 25(3), pp.75~98 (2019), 한국초등영어교육학회[DOI]
  46. 민덕기, "초등영어 예비교사들의 AI 챗봇 개발 활동 연구", 초등영어교육, 25(4), pp.169~190 (2019), 한국초등영어교육학회[DOI]
  47. Park, G., "55 tips for English classes by artificial intelligence" (2020), Chunjae Education
  48. Park, G., "Okay Google, Please Take Care of My English" (2020), Saramin
  49. 박명호, "AI 챗봇을 활용한 고등학생들의 영어 글쓰기 상호작용 분석", 학습자중심교과교육연구, 21(9), pp.1~13 (2021), 학습자중심교과교육학회
  50. Pérez, J. Q., "Rediscovering the use of chatbots in education:A Systematic Literature Review", Computer Applications in Engineering Education, 28(1), pp.1549~1565 (2020)
  51. Perez-Marin, D., "Conversational agents and natural language interaction:Techniques and effective practices" (2011), Information Science Reference
  52. Seo, J. I., "The effects of AI chatbot on the acquisition of English vocabulary and affective areas of elementary school students", MA, pp.2021
  53. 서숙영, "CMC를 활용한 영어학습의 효과성: 메타분석", 교원교육, 36(4), pp.373~400 (2020), 교육연구원[DOI]
  54. Seo, S. J., "Learning English vocabulary with a smart speaker: Opportunities and challenges for the elderly", MA, pp.2020
  55. 신동광, "영어 쓰기 능력 향상을 위한 AI 챗봇 활용 방안 탐색", 교원교육, 35(1), pp.41~55 (2019), 교육연구원[DOI]
  56. 성민창, "초등영어 예비교사들이 개발한 인공지능 챗봇", 어학연구, 56(1), pp.97~115 (2020), 언어교육원[DOI]
  57. "National Strategy for Artificial Intelligence" (2019)[URL]
  58. 양혜진, "Development and Application of AI Chatbot for Cabin Crews", 영어학, 21, pp.1085~1104 (2021), 한국영어학회[DOI]
  59. 윤숙경, "채터봇과 메신저를 이용한 채팅이 한국 대학생 영어학습자의 표현 능력에 미치는 영향에 관한 사례 연구", 멀티미디어 언어교육, 7(2), pp.269~292 (2004), 한국멀티미디어언어교육학회
  60. 윤여범, "인공지능과 초등영어교육: 챗봇의 현황과 발전 방향을 중심으로", 한국초등교육, 31, pp.77~90 (2020), 초등교육연구원[DOI]
  61. 윤정민, "플립러닝이 학습자의 영어 능력과 정의적 영역에 미치는 효과에 대한 메타분석", 영어교육연구, 32(2), pp.127~148 (2020), 팬코리아영어교육학회
  62. Wollny, S., "Are we there yet? A systematic literature review on chatbots in Education", Frontiers in Artificial Intelligence, 4, pp.654924~ (2020)[DOI]
  63. Xiao, Z., "Who should be my teammates: Using a conversational agent to understand individuals and help teaming", Paper presented at the 24th International Conference on Intelligent User Interfaces (2019)

관련 논문AI추천

이 간행물의 다른 논문 (10건)