The Effectiveness of Automated Essay Scoring in an EFL College Classroom
3
조회
0
다운로드
9
인용
0
참고문헌
키워드
초록
This study explores two English classes taught in English at a four-year university to investigate theeffectiveness of using the Automated Essay Scoring (AES) system on the improvement of essay writingabilities of both high level and low level EFL students. The instruments involved in this study includestudents’ essays, Criterion, an automated essay scoring system, and a survey. In order to examine theinfluence of the feedback provided by the Criterion system on the participants’ essays, their first and seconddraft were each evaluated and given holistic scores by the teacher-researcher. Following the evaluation, thescores on the first draft and the revised draft were compared. According to the statistical results, both highand low level students all showed overall improvement in their writing after using Criterion. Following anexamination of their writings, a questionnaire was used to measure the strengths and weaknesses of usingCriterion. Additionally, the students’ perception towards face-to-face human feedback, which was providedthrough an individual writing conference, was investigated. The findings of this study can offer pedagogicalsuggestions for incorporating AES in L2 writing class and implications for future research in EFL context.
Abstract
This study explores two English classes taught in English at a four-year university to investigate theeffectiveness of using the Automated Essay Scoring (AES) system on the improvement of essay writingabilities of both high level and low level EFL students. The instruments involved in this study includestudents’ essays, Criterion, an automated essay scoring system, and a survey. In order to examine theinfluence of the feedback provided by the Criterion system on the participants’ essays, their first and seconddraft were each evaluated and given holistic scores by the teacher-researcher. Following the evaluation, thescores on the first draft and the revised draft were compared. According to the statistical results, both highand low level students all showed overall improvement in their writing after using Criterion. Following anexamination of their writings, a questionnaire was used to measure the strengths and weaknesses of usingCriterion. Additionally, the students’ perception towards face-to-face human feedback, which was providedthrough an individual writing conference, was investigated. The findings of this study can offer pedagogicalsuggestions for incorporating AES in L2 writing class and implications for future research in EFL context.
참고문헌 (32)
- Attali, Y., "Exploring the feedback and revision features of Criterion", Paper presented at the National Council on Measurement in Education (NCME) (2004)
- Barkaoui, K., "Teaching writing to second language learners: Insights from theory and research", TESL Reporter, 40(1), pp.35~48 (2007)
- Beuningen, C. V., "Corrective feedback in L2 writing : Theoretical perspectives, empirical insights, and future directions", International Journal of English Studies, 10(2), pp.1~27 (2010)
- Burstein, J., "Automated essay scoring for nonnative English speakers (PDF)", Proceedings of the ACL99 Workshop on Computer-Mediated Language Assessment and Evaluation of Natural Language Processing (1999)
- Burstein, J., "CriterionSM online essay evaluation: An application for automated evaluation of student essays", Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence (2003)
- Chandler, J., "The efficacy of various kinds of error feedback for improvement in the accuracy and fluency of L2 student writing", Journal of Second Language Writing, 12, pp.267~296 (2003)
- Chen, C. F., "The use of a computer-based writing program: Facilitationor frustration?", Paper presented at the 23rd international conference on English teaching and learning (2006)
- Chen, C. F., "Beyond the design of automated writing evaluation:Pedagogical practices and perceived learning effectiveness in EFL writing classes", Language Learning & Technology, 12(2), pp.94~112 (2008)
- Methodological approaches to online scoring of essays, "Chung, K. W. K.", University of California, Center for the Study of Evaluation. (ERIC Document Reproduction Service No. ED 418 101), ~, pp.CSE Tech. Rep. No. 461 (1997)
- Dikli, S., "Automated Essay Scoring", Turkish Online Journal of Distance Education, 7(1) (2006)
- Dikli, S., "An overview of automated scoring of essays", Journal of Learning, and Assessment, 5(1) (2006)[DOI]
- Dikli, S., "The nature of automated essay scoring feedback", CALICO Journal, 28(1), pp.99~134 (2010)
- Ferris, D., "The “grammar correction” debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime...?)", Journal of Second Language Writing, 13, pp.49~62 (2004)
- Ferris, D., "Error feedback in L2 writing classes : How explicit does it need to be?", Journal of Second Language Writing, 10, pp.161~184 (2001)
- Huang, S. J., "Automated versus human scoring: A case study in an EFL context", Electronic Journal of Foreign Language Teaching, 11(1), pp.149~164 (2014)[DOI]
- Hyland, F., "ESL writers and feedback : Giving more autonomy to students", Language Teaching Research, 4(1), pp.33~54 (2000)
- Hyland, K., "Feedback in second language writing: Contexts and issues", pp.1~20 (2006), Cambridge University Press
- Hyland, K., "Feedback on second language students’ writing", Language Teaching, 39(2), pp.83~101 (2006)
- Jiang, J., "Automated scoring research over 40 years : Looking back and ahead", Journal of Artificial Intelligence, 5(1), pp.56~63 (2012)
- Krashen, S., "Principles and practice in second language acquisition" (1982), Pergamon Press
- Lee, I., "L2 writing teachers’ perspectives, practices and problems regarding error feedback", Assessing Writing, 8, pp.216~237 (2003)
- Liu, Y., "The effects of error feedback in second language writing", Arizona Working Papers in SLA & Teaching, 15, pp.65~79 (2008)
- Montague, N., "The process oriented approach to teaching writing to second language learners", New York State Association for Bilingual Education Journal, 10, pp.13~24 (1995)
- Othman, S. B., "Student response to teacher feedback on multiple-draft compositions in ESL classroom", Proceedings of the 2nd international conference of teaching and learning (2009)
- Pasand, P. G., "Process-product approach to writing : The effect of model essays on EFL learners’ writing accuracy", International Journal of Applied Linguistics & English Literature, 2(1), pp.75~79 (2013)
- Raimes, A., "Techniques in teaching writing" (1983), Oxford University Press
- Shermis, M. D., "The impact of automated essay scoring on writing outcomes", Paper presented at the annual meetings of the national council on measurement in education (2008)
- Toranj, S., "Automated versus human essay scoring : A comparative study", Theory and Practice in Language Studies, 2(4), pp.719~725 (2012)
- Toranj, S., "Automated versus human essay scoring", International Conference on Language Medias and Culture, 33, pp.31~35 (2012)
- Valenti, S., "An overview of current research on automated essay grading", Journal of Information Technology Education, 2, pp.319~330 (2003)
- Vokic, G., "Feedback in L2 writing : The students’ perspective", Porta linguarum, 9, pp.139~155 (2008)
- Zhang, J., "A comprehensive review of studies on second language writing", Applied Language Studies, 12, pp.89~123 (2008)