Corrective Feedback that an Automatic Writing Evaluation System Can and Cannot Provide

Abstract

The purpose of the study is to report and share a reflection on using the ETS Criterion online writing evaluation system in a university setting and evaluate if this system can be a suitable teaching/learning tool for English language learners (ELLs) in an English as a Foreign Language (EFL) context to facilitate their writing ability. Participants were given a writing prompt to write about in class and entered their writing into the Criterion system to receive feedback. Both quantitative and interpretive analyses were conducted for 129 student writing samples and Trait Feedback Analysis reports. The study revealed that the sentence-level feedback appears to be in urgent need for EFL writers, but Criterion did not provide this kind of feedback. Errors in such aspects as basic sentence structures, subject-verb relationship, awkward meaning construction, and formulaic expressions could not be treated by Criterion at all. In addition, the comparison of teacher feedback and Criterion feedback were discussed in details. This study alerts users to contemplate before use what the purpose of using Criterion is, whether for diagnosis of writing proficiency or as a teaching/learning tool for developing students’ writing ability. Plus, educators should be aware of on which language aspects and features EFL writers necessitate feedback to facilitate their writing development, different from first language (L1) writers.