The Potential of Automated Writing Evaluation Feedback in the Korean University Context
2
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
In this study, a popular online writing evaluation service, Criterion, developed by ETS, was integrated into writing instruction in the university context. This study examined the pedagogical effectiveness of Criterion feedback in that context, specifically the extent to which students’ writing accuracy increased over the course of two papers and students’ perceptions of the benefits of feedback were also examined. The results showed that normalized error frequency significantly decreased from paper 1 to paper 2 overall and in the categories of mechanics and style. Students perceived that Criterion feedback, especially grammar, was very useful in revising essays, and it gave them the opportunity to examine their own writing weaknesses. Although normalized frequency of errors in the category of grammar did not decrease from paper 1 to paper 2, students nevertheless perceived grammar feedback as quite useful, as indicated by coding references to “grammar” in journal entries by NVivo analysis. The findings serve to indicate the potential value of automated writing evaluation use in the EFL context, but need to be supplemented by more research with a larger sample in a similar context.
Abstract
In this study, a popular online writing evaluation service, Criterion, developed by ETS, was integrated into writing instruction in the university context. This study examined the pedagogical effectiveness of Criterion feedback in that context, specifically the extent to which students’ writing accuracy increased over the course of two papers and students’ perceptions of the benefits of feedback were also examined. The results showed that normalized error frequency significantly decreased from paper 1 to paper 2 overall and in the categories of mechanics and style. Students perceived that Criterion feedback, especially grammar, was very useful in revising essays, and it gave them the opportunity to examine their own writing weaknesses. Although normalized frequency of errors in the category of grammar did not decrease from paper 1 to paper 2, students nevertheless perceived grammar feedback as quite useful, as indicated by coding references to “grammar” in journal entries by NVivo analysis. The findings serve to indicate the potential value of automated writing evaluation use in the EFL context, but need to be supplemented by more research with a larger sample in a similar context.
참고문헌 (18)
- Attali, Y., "Exploring the feedback and revision features of Criterion", Journal of Second Language Writing, 14, pp.191~205 (2004)
- Biber, D., "Variation across speech and writing" (1988), Cambridge University Press
- Burstein, J., "CriterionSM online essay evaluation: An application for automated evaluation of student essays", Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence, pp.3~10 (2003), Association for the Advancement of Artificial Intelligence
- Burstein, J., "Handbook of automated essay evaluation: Current applications and new directions", pp.2013, Routledge
- EFL teacher’s course, "The grammar book: An ESL", pp.1998, Celce-Murcia, M.
- Chen, C. -F., "Beyond the design of automated writing evaluation : Pedagogical practices and perceived learning effectiveness in EFL writing classes", Language Learning &Technology, 12(2), pp.94~112 (2008)
- Dikli, S., "Automated essay scoring feedback for second language writers: How does it compare to instructor feedback?", Assessing Writing, 22, pp.1~17 (2014)
- Elliot, N., "Handbook of automated essay evaluation: Current applications and new directions", pp.2013, Routledge
- Grimes, D., "Utility in a fallible tool: A multi-site case study of automated writing evaluation", Journal of Technology, Learning, and Assessment, 8(6), pp.1~42 (2010)
- 김태은, "Reflection on Using the Criterion Online Writing Evaluation Service", 멀티미디어 언어교육, 13(3), pp.59~83 (2010), 한국멀티미디어언어교육학회[DOI]
- 김지은, "The Effectiveness of Automated Essay Scoring in an EFL College Classroom", 멀티미디어 언어교육, 17(3), pp.11~36 (2014), 한국멀티미디어언어교육학회[DOI]
- Li, J., "Rethinking the role of automated writing evaluation feedback in ESL writing instruction", Journal of Second Language Writing, 27, pp.1~18 (2015)
- Li, Z., "The role of automated writing evaluation holistic scores in the ESL classroom", System, 44(1), pp.66~78 (2014)
- Liao, H. -C., "Using automated writing evaluation to reduce grammar errors in writing", ELT Journal, 70(3), pp.308~319 (2016)
- Link, S., "Towards best ESL practices for implementing automated writing evaluation", CALICO Journal, 31(3), pp.323~344 (2014)
- 문영인, "Short-term Effects of Automated Writing Feedback and Users’ Evaluation of Criterion", 응용언어학, 27(4), pp.125~150 (2011), 한국응용언어학회
- Warschauer, M., "Automated writing assessment in the classroom", Pedagogies : An International Journal, 3(1), pp.22~36 (2008)