Utilizing ChatGPT for Analysis and Evaluation of Korean English Learners’ Presentations
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study explores the utilization of ChatGPT for evaluating English oral presentations by Korean university students. By comparing assessments made by an instructor and ChatGPT, this study aims to determine the efficacy and reliability of AI-assisted evaluation in language education. The findings indicate that ChatGPT can effectively learn and apply the evaluation criteria provided by instructors, particularly in aspects such as delivery and time management. However, limitations exist, including the AI’s tendency to overlook critical areas emphasized by human evaluators, such as content depth and the logical cohesion of ideas. Additionally, non-linguistic elements like body posture and eye contact were less accurately assessed by the AI. These discrepancies highlight the need for continuous refinement of AI algorithms to better mimic human judgment and incorporate detailed evaluation factors. The study also underscores the potential of AI-generated feedback to support a learner-centric approach, enabling students to access tailored feedback and improve their language skills independently. Future research should focus on refining AI evaluation algorithms and exploring the long-term impact of AI-assisted feedback on student performance. The integration of AI tools like ChatGPT into educational practices holds promise, but ongoing evaluation and refinement are crucial to ensure they meet pedagogical goals.
Abstract
This study explores the utilization of ChatGPT for evaluating English oral presentations by Korean university students. By comparing assessments made by an instructor and ChatGPT, this study aims to determine the efficacy and reliability of AI-assisted evaluation in language education. The findings indicate that ChatGPT can effectively learn and apply the evaluation criteria provided by instructors, particularly in aspects such as delivery and time management. However, limitations exist, including the AI’s tendency to overlook critical areas emphasized by human evaluators, such as content depth and the logical cohesion of ideas. Additionally, non-linguistic elements like body posture and eye contact were less accurately assessed by the AI. These discrepancies highlight the need for continuous refinement of AI algorithms to better mimic human judgment and incorporate detailed evaluation factors. The study also underscores the potential of AI-generated feedback to support a learner-centric approach, enabling students to access tailored feedback and improve their language skills independently. Future research should focus on refining AI evaluation algorithms and exploring the long-term impact of AI-assisted feedback on student performance. The integration of AI tools like ChatGPT into educational practices holds promise, but ongoing evaluation and refinement are crucial to ensure they meet pedagogical goals.
참고문헌 (25)
- Alwi, N. F. B., "Oral Presentation : Self-perceived Competence and Actual Performance among UiTM Business Faculty Students", Procedia-Social and Behavioral Sciences, 90, pp.98~106 (2013)
- Burgoon, J. K., "Nonverbal Communication" (2016), Routledge
- Chan, V., "Teaching Oral Communication in Undergraduate Science : Are We Doing Enough and Doing it Right", Journal of Learning Design, 4(3), pp.71~79 (2011)
- Chapelle, C. A., "Computer Applications in Second Language Acquisition" (2001), Cambridge University Press
- Coates, H., "A Critical Examination of the Effects of Learning Management Systems on University Teaching and Learning", Tertiary Education and Management, 11(1), pp.19~36 (2005)
- Choi, W., "Process-oriented Speaking Assessment of Primary English Using AI Chatbots : Possibilities and Limitations", Primary English Education, 26(1), pp.131~152 (2020)
- "TOEFL iBT Speaking Rubrics"[URL]
- Harmer, J., "The Practice of English Language Teaching" (2007), Pearson Longman
- Kim, H. -S., "Analysis of the Effectiveness of an AI-based English Speaking Program", Journal of Educational Methodology, 33(3), pp.563~588 (2021)[DOI]
- Knight, S., "AcaWriter : A Learning Analytics Tool for Formative Feedback on Academic Writing", Journal of Writing Research, 12(1), pp.141~186 (2020)
- Lee, J., "Investigating ChatGPT’s Reliability as an Automated Scoring Tool for Small Classes in University English Education", Modern English and American Literature, 42(1), pp.189~212 (2024)[DOI]
- Levy, M., "Computer-assisted Language Learning: Context and Conceptualization" (1997), Oxford University Press
- Magin, D., "Peer and Teacher Assessments of Oral Presentation Skills : How Reliable Are They", Studies in Higher Education, 26(3), pp.287~298 (2001)
- Murphy, H. A., "Effective Business Communication" (1997), McGraw-Hill
- Paré, A, "The Routledge Doctoral Student’s Companion", pp.125~133 (2010), Routledge
- Park, M., "A Study on the Comparison of AI PengTalk’s and English Native Speakers’ Assessment of Elementary Students’ English Pronunciation", Journal of the Korea English Education Society, 21(4), pp.67~90 (2022)[DOI]
- Qu, W, "Research on the Application of Automatic Scoring System in College English Writing", Proceedings of the 2016 International Conference on Economics, Social Science, Arts, Education and Management Engineering, pp.694~698 (2016)
- Smith, C. M., "Integrating Lecture Capture as a Teaching Strategy to Improve Student Presentation Skills through Self-assessment", Active Learning in Higher Education, 12(3), pp.151~162 (2011)
- van Ginkel, S., "The Impact of the Feedback Source on Developing Oral Presentation Competence", Studies in Higher Education, 42(9), pp.1671~1685 (2017)
- Xiao, Y., "An Exploratory Study of EFL Learners’ Use of ChatGPT for Language Learning Tasks : Experience and Perceptions", Languages, 8(3), pp.212~ (2023)
- Yang, H., "A Study on Adopting AI-based Chatbot in Elementary English-speaking Classes", Multimedia-Assisted Language Learning, 22(4), pp.184~205 (2019)
- Yoo, I., "Developing an AI-based Sentence-generating Web Service for Writing Activities in Elementary Language Education", Journal of Curriculum and Instructional Studies, 27(2), pp.210~221 (2023)[DOI]