ChatGPT의 평가와 피드백에 나타난 프레이밍 효과: 탐색적 혼합 연구

Abstract

This study examined how ChatGPT’s evaluation and feedback patterns varied when identical text was framed as ‘student text’ or ‘AI revision.’ An exploratory mixed-methods design was employed, in which 66 argumentative essays written by university students were assessed under two framing conditions, and ChatGPT’s rubric-based scores and feedback were collected for comparison. The quantitative analysis indicated that ChatGPT assigned higher scores when an essay was presented as an ‘AI revision’ rather than as a ‘student text,’ with the largest differences observed in grammar and language use. The qualitative analysis further revealed distinct differences in tone, evaluative orientation, feedback function, and meta-evaluation. Under the ‘AI revision’ framing, ChatGPT provided more encouraging and reinforcement-oriented feedback, whereas under the ‘student text’ framing, the feedback was more critical and evaluative. These findings suggest that ChatGPT’s responses are influenced not only by textual features but also by contextual framing information, with its role shifting from an evaluator under the ‘student text’ framing to a facilitator under the ‘AI revision’ framing. Pedagogically, the study highlights the need for framing literacy, encouraging teachers and learners to recognize the contextual dependence and potential biases of AI feedback and to engage with it more critically.