과학 교과의 학생 평가에서 ChatGPT의 활용 가능성 및 교사 인식 탐색
Exploration on the Feasibility of Utilization and Teacher Perceptions of Using ChatGPT for Student Assessment in Science
1
조회
0
다운로드
7
인용
0
참고문헌
키워드
초록
본 연구는 과학 교과의 학생 평가에서 생성형 인공지능인 ChatGPT 의 활용 가능성을 탐색하는 데 목적이 있다. 이를 위해 평가 문항을개발하여 학생들로부터 문항에 대한 응답 자료를 수집하였고, 응답결과를 ChatGPT에 입력하여 평가의 과정을 수행하도록 하였다. 또한교사들에게 ChatGPT로부터 얻은 평가 결과를 공유하고, 교사들의평가 과정과 비교하여 학생 평가에서의 ChatGPT의 활용 가능성을탐색해 보았다. 연구 결과, 채점 기준의 설정 측면에서 ChatGPT가생성해 내는 채점 기준이 전반적으로 타당성을 갖추고 있는 것으로볼 수 있었다. 그러나 ChatGPT가 수행한 채점 결과는 교사들 채점한결과와 비교하였을 때 일관성이 다소 낮았고, 특히 문항에 포함된평가 요소가 많고, 평가 요소별 배점 기준이 복잡할수록 채점 결과의일치도가 더 낮게 나타났다. 학생 응답에 대한 피드백 측면에서는학생 응답의 과학적 타당성을 평가하는 과정에서 일부 잘못된 내용을제공하거나 교육과정을 넘어서는 수준의 피드백이 생성되는 경우도있었으나, 문항에 대한 올바른 답안을 알려주고, 추가적인 사례들을제공하는 등의 긍정적인 부분도 확인할 수 있었다. 이러한 결과들을바탕으로 교사들로부터 ChatGPT의 활용 가능성에 대한 인식을 살펴보았을 때, 학생 평가에서 중요하게 인식되는 ‘신뢰성’ 측면에서 부족한 면이 있기는 하지만, 교사들의 평가를 지원하는 측면에서 활용가능성을 발견할 수도 있었다. 마지막으로, 이러한 연구 결과를 종합하여 학생 평가에서의 ChatGPT의 활용에 대한 시사점을 제언하였다.
Abstract
This study explores the possibility of using a generative artificial intelligence, ChatGPT, for student assessment in science subjects. In order to achieve our goal, we developed assessment items, collected students’ responses, and input them into ChatGPT to implement the assessment procedures. Subsequently, we shared the assessment results from ChatGPT with science teachers and compared them to the teachers’ assessment process to investigate the use of ChatGPT in student assessment. Regarding the results, in terms of setting the scoring rubric, we found the rubric generated by ChatGPT to be generally appropriate. However, the consistency between the scoring results obtained from ChatGPT and those determined by the teachers was relatively low. This inconsistency was more pronounced in items with additional assessment components and a more intricate rubric. In regard to feedback on student responses, there were some instances where the feedback generated was scientifically incorrect or beyond the scope of the curriculum, but there were also some positives, such as the provision of exemplary answers to questions and additional examples that helped students learn further. From these results, the teachers perceived limitations in using ChatGPT to conduct assessment in terms of reliability, which is considered crucial in student assessment, but suggested that it could be used to support assessment. Finally, synthesizing these findings, implications for utilizing ChatGPT in student assessment were suggested.
참고문헌 (24)
- Bhat, S., "Towards automated generation and evaluation of questions in educational domains", Paper presented at the 15th International Conference on Educational Data Mining (2022)
- Byeon, J. -H., "An investigation of generative AI in educational application: Focusing on the usage of ChatGPT for learning biology", Brain, Digital, & Learning, 13(1), pp.1~17 (2023)
- Chang, J., "An analysis on the trends of education research related to ‘artificial intelligence chatbot’ in Korea : Focusing on implications for use in science education", Journal of Learner-Centered Curriculum and Instruction, 21(13), pp.729~743 (2021)
- Choi, S. -W., "The use of AI chatbot as an assistant tool for SW education", Journal of the Korea Institute of Information and Communication Engineering, 23(12), pp.1693~1699 (2019)
- González-Calatayud, V., "Artificial intelligence for student assessment : A systematic review", Applied Sciences, 11(12), pp.5467~5482 (2021)
- Holmes, W., "Artificial intelligence in education: Promise and implications for teaching and learning" (2019), Center for Curriculum Redesign
- Artificial Intelligence and EduTech in School Education, "Hong, S.", Korea Institute for Curriculum and Evaluation, ~, pp.RRI 2020-2 (2020)
- Huh, M., "Domestic research trends of learning with AI", Journal of The Korean Association of Information Education, 25(6), pp.973~985 (2021)
- Jang, H., "The analysis of research trends and topics about the educational use of ChatGPT", Journal of Research in Curriculum & Instruction, 27(4), pp.387~401 (2023)
- Jho, H., "Understanding of generative artificial intelligence based on textual data and discussion for its application in science education", Journal of the Korean Association for Science Education, 43(3), pp.307~319 (2023)
- Kang, D., "The advent of ChatGPT and the response of Korean language education", Korean Language and Literature, (82), pp.469~496 (2023)
- Kim, H., "The artificial intelligence era and science education-With a focus on the autonomy and relatedness of artificial intelligence-", The Journal of Yeolin Education, 29(6), pp.1~23 (2021)
- Kim, H., "Teachers’ perceptions of AI in school education", Journal of Educational Technology, 26(3), pp.905~930 (2020)
- Kim, J. I., "Exploring applications of ChatGPT for physics education : Focusing on high school and general physics class", School Science Journal, 17(3), pp.216~239 (2023)
- Moore, S., "Assessing the quality of student-generated short answer questions using GPT-3", Paper presented at the 17th European Conference on Technology Enhanced Learning (2022)
- "ChatGPT: Optimizing language models for dialogue" (2023)[URL]
- A Study on the Development of Automated Scoring Method for Computer-based Essay and Short Answer Question Type Assessment(Ⅰ), "Park, J. -I.", Korea Institute for Curriculum and Evaluation, ~, pp.RRE 2022-6 (2022)
- Park, S. -Y., "Exploring the possibility of science-inquiry competence assessment by ChatGPT-4 : Comparisons with human evaluators", Korean Journal of Educational Research, 61(4), pp.299~332 (2023)
- Shin, D., "Feasibility and constraints in applying an AI chatbot to english education", Brain, Digital, & Learning, 9(2), pp.29~40 (2019)
- Shin, D., "Exploring the potential of using ChatGPT as a content-based English learning and teaching tool", Journal of the Korea English Education Society, 22(1), pp.171~192 (2023)
- Smutny, P., "Chatbots for learning : A review of educational chatbots for the Facebook Messenger", Computers &Education, 151, pp.1~11 (2020)
- Son, T., "Exploring the possibility of using ChatGPT in mathematics education : Focusing on student product and pre-service teachers’ discourse related to fraction problems", Education of Primary School Mathematics, 26(2), pp.99~113 (2023)
- Zhai, X., "A review of Artificial Intelligence(AI)in education from 2010 to 2020", Complexity, 2021, pp.1~18 (2021)
- "A comprehensive survey on pretrained foundation models: A history from bert to chatgpt" (2023)[URL]