효율적 AI 피드백 설계를 위한 LLM 성능 비교 연구: 대학생의 설득적 글쓰기 과제를 중심으로
A comparative evaluation of Large Language Models for efficient AI feedback design: Evidence from undergraduate persuasive writing tasks
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
.
Abstract
This study compares the feedback profiles produced by ChatGPT and Gemini on the same undergraduate persuasive-writing task under an identical prompt. Using an RTPR-based LLM-as-a-Judge framework with counter-balancing, we tested seven rubric criteria and the overall mean via independent two-sample t-tests. Significant differences emerged for most criteria and for the overall mean (p<.001); Lexical Accuracy favored ChatGPT over Gemini (p=.017), whereas Sentence Clarity was not significant (p=.084). These findings indicate systematic, model-specific strengths and weaknesses even under identical inputs, suggesting instructionally targeted model choice (e.g., argument/evidence/organization vs. lexical refinement). The study was limited to two models and a single genre; no multiple-comparison correction was applied.