생성형 AI를 활용한 현직 국어교사의 서‧논술형 평가 문항 개발 양상 분석
Analysis of the Development Pattern of Constructive Assessment Items by Korean Language Teachers Using Generative AI
1
조회
0
다운로드
7
인용
0
참고문헌
키워드
초록
본 연구에서는 ChatGPT를 활용한 10명의 현직 국어 교사의 서논술형 평가 개발 사례를 바탕으로 생성형 AI를 활용한 서논술형 평가 문항 개발의 가능성과 활용 방안을 제안하였다. 본 연구의 결과를 정리하면 다음과 같다. 첫째, 서‧논술형 평가 문항 생성 시 효과적인 프롬프트 구성이 매우 중요하다. 1) 명시적이고 구체적인 정보의 제공(역할, 대상 학년, 성취기준 등)과 2) 성취기준에 대한 해설 및 서논술형 평가의 개발 요건, 3) 문두, 지문, 채점을 위한 정답, 정답 해설, 채점 루브릭 등에 대한 산출 결과에 명시적인 요구를 제시해 줄 수 있어야 한다. 둘째, ChatGPT에 입력하는 프롬프트를 반복적으로 수정하고 구체화하는 과정이 중요하다. 초기에 원하는 답변을 얻지 못해도 단계별(step by step)로 접근하며 프롬프트를 개선해 나갈 필요가 있다. 셋째, AI가 지문을 직접 생성하는 방식보다는 기존의 텍스트를 제시해 주고, AI가 텍스트를 재가공하여 문항을 생성하도록 하는 것이 효과적이다. 넷째, ChatGPT는 신속하고 효과적인 채점 자료의 생성과 활용이 용이하지만 이를 그대로 수용하기보다는 비판적 재구성을 거쳐 활용할 필요가 있다.
Abstract
In this study, we proposed the potential and utilization methods for developing constructive assessment items using generative AI based on 10 Korean language teachers' cases of developing assessment using ChatGPT. Firstly, it is crucial to effectively compose prompts when generating constructive assessment items.In particular, it is essential to provide explicit and specific information(such as roles, target grade, achievement criteria, etc.) for constructive assessment items. Additionally, prompts should explicitly specify what elements should be included in the generated items. Secondly, the process of iteratively refining and specifying the prompts entered into ChatGPT is crucial. Even if the desired responses are not obtained initially, it is necessary to approach the improvement of prompts step by step, gradually modifying them to achieve the desired outcomes. Thirdly, it is more effective to provide existing text to AI and have it generate assessment items by reprocessing the text, rather than relying on AI to directly generate passages. Fourthly, while ChatGPT facilitates the rapid and efficient generation and utilization of scoring materials, it is important to critically reconstruct and adapt these materials rather than accepting them as they are.
참고문헌 (36)
- 강동훈, "챗지피티(ChatGPT)의 등장과 국어교육의 대응", 국어문학, 82, pp.469~496 (2023)
- 권태현, "인공지능 시대의 글쓰기와 작문 교육의 방향 탐색-생성형 인공지능의 교육적 활용을 중심으로-", 한민족문화연구, 83(83), pp.137~174 (2023)
- 신주은, "프롬프트 엔지니어링 교육이 초등학생의 인공지능 관련 태도에 미치는 영향", 정보교육학회논문지, 27(5), pp.635~644 (2023)
- 장성민, "챗GPT가 바꾸어 놓은 작문교육의 미래-인공지능 시대, 작문교육의 대응을 중심으로-", 작문연구, 56, pp.7~34 (2023)
- 최숙기, "인공지능 시대의 작문 평가를 위한 ChatGPT 활용 방안연구", 청람어문교육, 95, pp.65~109 (2023)
- Adamopoulou, E., "Chatbots: History, technology, and applications", Machine Learning with Applications, 2 (2020)[DOI]
- Barrot, J. S., "Using ChatGPT for second language writing : Pitfalls and potentials", Assessing Writing, 57, pp.100745~ (2023)[DOI]
- Conijn, R., "The Effects of Explanations in Automated Essay Scoring Systems on Student Trust and Motivation", Journal of Learning Analytics, 10(1), pp.37~53 (2023)
- "Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT" (2023)[URL]
- Dergaa, I., "From human writing to artificial intelligence generated text : examining the prospects and potential threats of ChatGPT in academic writing", Biology of sport, 40(2), pp.615~622 (2023)[DOI]
- Duri Long, "What is AI Literacy? Competencies and Design Considerations", Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20), pp.1~16 (2020), Association for Computing Machinery
- Elkins, K., "Can GPT-3 pass a writer’s Turing Test", Journal of Cultural Analytics, 2371, pp.4549~ (2020)
- Floridi, Luciano, "AI as Agency Without Intelligence : on ChatGPT, Large Language Models, and Other Generative Models", Philosophy and Technology, 36(1), pp.1~7 (2023)
- Gilson, A., "How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment", JMIR medical education, 9, pp.e45312~ (2023)[DOI]
- Godwin-Jones, R., "Partnering with AI : Intelligent writing assistance and instructed language learning", Language Learning & Technology, 26(2), pp.5~24 (2022)[DOI]
- Jeon, J., "Large language models in education: a focus on the complementary relationship between human teachers and ChatGPT", Education and Information Technologies (2023)[DOI]
- Katz, Daniel, "GPT-4 Passes the Bar Exam", SSRN Electronic Journal, 10 (2023)[DOI]
- Krueger, R., "Focus Groups: A Practical Guide for Applied Research" (2009), Sage Publications
- Kung TH, "Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models", PLOS Digit Health, 2(2), pp.e0000198~ (2023)[DOI]
- McGarrell, H., "Motivating revision of drafts through formative feedback", ELT Journal, 61(3), pp.228~236 (2007)[DOI]
- "Exploring the Potential of Using an Ai Language Model for Automated Essay Scoring"[URL]
- O'Connor, S., "Open Artificial Intelligence Platforms in Nursing Education: Tools for Academic Progress or Abuse?", Nurse Education in Practice, 66, pp.103537~ (2023)[DOI]
- "OpenAI homepage" (2023)[URL]
- Lee, S., H., "Prompt engineering to improve the performance of teaching and learning materials Recommendation of Generative Artificial Intelligence", Journal of the Korea Society of Computer and Information, 28(8), pp.195~204 (2023)[DOI]
- "Comparing the Quality of Human and ChatGPT Feedback on Students’ Writing" (2023)[URL]
- Su, Y., "Collaborating with ChatGPT in argumentative writing classrooms", Assessing Writing (2023)[DOI]
- Thorp H. H., "ChatGPT is fun, but not an author", Science (New York, N.Y.), 379(6630), pp.313~ (2023)[DOI]
- Wang, L., "ChatGPT's capabilities in providing feedback on undergraduate students’argumentation : A case study", Thinking Skills and Creativity, 51, pp.101440~ (2024)
- Zawacki-Richter, O., "Systematic review of research on artificial intelligence applications in higher education – where are the educators?", Int J Educ Technol High Educ, 16, pp.39~ (2019)[DOI]
- Zhai, X., "A review of artificial intelligence(AI)in education from 2010 to 2020", Complexity, 2021, pp.1~18 (2021)