ChatGPT 4.0의 중등교원 임용시험 전공영어 기출문제 풀이수행능력 탐색 연구
Exploring ChatGPT 4.0’ Ability to Solve Secondary School English Teacher Certification Exam Questions
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In recent years, ChatGPT has integrated itself into applied linguistics and language education research, with studies proposing different ways to use this new technology in second language teaching. As part of these efforts, the present study aimed to investigate ChatGPT’s ability to solve the domain-specific questions of high-stakes exams. To this end, the dataset for this study consisted of multiple-choice questions taken from the teacher certification exams for prospective English teachers, developed by the Korea Institute for Curriculum and Evaluation between 2009 and 2013. The authors then asked ChatGPT to solve each question by entering them as prompts. The results showed that ChatGPT’s average percentage of correct answers was 66%, with the percentage slightly lower for the English Literature and Linguistics subsections than for the English Education Theories and General English subsections. The qualitative analysis also revealed four different patterns in the errors made by ChatGPT when answering the target questions, which we illustrate with specific examples. Based on these findings, we have provided implications for pedagogy and future research.
Abstract
In recent years, ChatGPT has integrated itself into applied linguistics and language education research, with studies proposing different ways to use this new technology in second language teaching. As part of these efforts, the present study aimed to investigate ChatGPT’s ability to solve the domain-specific questions of high-stakes exams. To this end, the dataset for this study consisted of multiple-choice questions taken from the teacher certification exams for prospective English teachers, developed by the Korea Institute for Curriculum and Evaluation between 2009 and 2013. The authors then asked ChatGPT to solve each question by entering them as prompts. The results showed that ChatGPT’s average percentage of correct answers was 66%, with the percentage slightly lower for the English Literature and Linguistics subsections than for the English Education Theories and General English subsections. The qualitative analysis also revealed four different patterns in the errors made by ChatGPT when answering the target questions, which we illustrate with specific examples. Based on these findings, we have provided implications for pedagogy and future research.
참고문헌 (20)
- Ahn, Y. Y., "Performance of ChatGPT 3. 5 on CSAT : Its potential as a language learning and assessment tool", Journal of the Korea English Education Society, 22(2), pp.119~145 (2023)
- Chaudron, C, "Second language classroom: Research on teaching and learning" (1988), Cambridge University Press
- Chomsky, N., "Knowledge of language: Its nature, origin and use" (1986), Praeger
- Gilson, A., "How does ChatGPT perform on the medical licensing exams? The implications of large language models for medical education and knowledge assessment", MedRxiv: The Preprint Server for Health Sciences, pp.1~9 (2022)[DOI]
- Jang, H., "The analysis of research trends and topics about the educational use of ChatGPT", Journal of Research in Curriculum & Instruction, 27(4), pp.387~401 (2023)[DOI]
- Jeon, J., "Large language models in education : A focus on the complementary relationship between human teachers and ChatGPT", Education and Information Technologies, 28, pp.15873~15892 (2023)[DOI]
- Kohnke, L., "ChatGPT for language teaching and learning", RELC Journal, 54(2), pp.537~550 (2023)[DOI]
- Korea Institute for Curriculum and Evaluation, "2009 Teacher certification exams: English" (2008), Korea Institute for Curriculum and Evaluation
- Korea Institute for Curriculum and Evaluation, "2010 Teacher certification exams: English" (2009), Korea Institute for Curriculum and Evaluation
- Korea Institute for Curriculum and Evaluation, "2011 Teacher certification exams: English" (2010), Korea Institute for Curriculum and
- Korea Institute for Curriculum and Evaluation, "2012 Teacher certification exams: English" (2011), Korea Institute for Curriculum and Evaluation
- Kwon, S-K., "Investigating the performance of generative AI ChatGPT’s reading comprehension ability", Journal of the Korea English Education Society, 22(2), pp.147~172 (2023)
- Lee, J. H., "Artificial intelligence-based content generator technology for young English-as-a-Foreign-Language learners’ reading enjoyment", RELC Journal, 54(2), pp.508~516 (2023)[DOI]
- Lee, S., "Exploration of domestic research trends on educational utilization of generative artificial intelligence", The Journal of Korean Association of Computer Education, 26(6), pp.15~27 (2023)[DOI]
- Li, J., "Exploring the Use of ChatGPT in Chinese Language Classrooms", International Journal of Chinese Language Teaching, 4(3), pp.36~55 (2023)[DOI]
- Saeed, J. I., "Semantics" (2009), Wiley-Blackwell
- Shin, D., "A case study on English test item development training for secondary school teachers using AI tools : Focusing on ChatGPT", Language Research, 59(1), pp.21~42 (2023)[DOI]
- Shin, D., "Can ChatGPT make reading comprehension testing items on par with human experts?", Language Learning & Technology, 27(3), pp.27~40 (2023)
- Yi, M., "Predicting the difficulty of college scholastic ability test based on ChatGPT and corpus linguistic features", Multimedia-Assisted Language Learning, 26(4), pp.29~50 (2023)[DOI]