＂아무튼 high level은 아닌 것 같아요＂: 채점방식 재설계 사례연구

Abstract

Task-based English language speaking tests draw attention in different contexts. Among them, Sookmyung Women’s University students are required to take a speaking test, MATE, and pass a standard for graduation. The use of ACTFL-SOPI has been revised in MATE certification process for graduation. The test has been validated in a specific testing mandate. One problem, however, was identified in a rating phase. Raters struggle to give ‘moderate high’ level. They avoid variability in their rating ranges. The purpose of this paper is to explore the ‘overfit’ rating pattern in speaking. The undesirable rating was tracked down through FACETS data analyses, raters’ self-report, and relevant document analyses. This paper also proposes empirically driven rating schemes, which include task-based dichotomous rating techniques. It was inferred that overfit rating patterns can be corrected, and that testing professionals need to step in where implementation concerns are identified. Test reliability doesn’t exist in itself. It is the one to be secured through research. Other tips, like using parsimonious rating checklist, are discussed and further research of rating is proposed.