Comparison of Grading Task Between ESL and Field Raters on Content-Field Essays

Abstract

The current study investigates how ESL raters as a group would differ from content- field raters in grading field-specific essays in terms of rater consistency, rater severity and grading perceptions. A total of 124 essays written on four disciplinary areas such as humanities/social sciences, business, technology and life sciences were graded into four scales regarding five features and one holistic mark. Each essay was read by two content-field teaching assistants and by a random pair of ESL teachers among six. The results revealed a good deal of discrepancy between field and ESL raters. First, across all the features, ESL raters were less consistent with each other than field raters, and this phenomenon was more salient on the content-related features. Second, in general, a group of ESL raters were less severe, and showed more variability in their severity than field TAs. Finally, ESL raters expressed great difficulty of judgment of the content feature for their holistic score decision. Based on the study outcomes, guidelines for future ESL rater training and directions for further studies are suggested.