Criterion Feedback in College Writing Instruction: Its Effectiveness and Student Perception
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
The use of computer technologies by writing students has increased, but language teachers and researchers have expressed doubts about their effectiveness and feasibility. In the field of writing assessment, this study examined the effectiveness of Criterion as an automated essay evaluation (AEE) system while addressing two research questions: (a) What types of feedback did Criterion provide to Korean EFL learners? and (b) With respect to their English writing study, how did Korean EFL learners perceive Criterion feedback? Analyses on engineering students' essays and perception survey data were mainly conducted quantitatively. Results showed that Criterion feedback on the students' essays was effective for correcting missing articles and number/subject-verb agreement errors and reducing repetitive words. The overall student perception of Criterion feedback was positive. However, vague feedback on organization and development and repeating words; and inaccurate feedback on mechanical errors, mostly spelling errors and articles; and the lack of feedback on errors need to be dealt with as they decrease the reliability of the AEE system as an instructional tool. The results and findings of this study were discussed while offering implications for future avenues of research.
Abstract
The use of computer technologies by writing students has increased, but language teachers and researchers have expressed doubts about their effectiveness and feasibility. In the field of writing assessment, this study examined the effectiveness of Criterion as an automated essay evaluation (AEE) system while addressing two research questions: (a) What types of feedback did Criterion provide to Korean EFL learners? and (b) With respect to their English writing study, how did Korean EFL learners perceive Criterion feedback? Analyses on engineering students' essays and perception survey data were mainly conducted quantitatively. Results showed that Criterion feedback on the students' essays was effective for correcting missing articles and number/subject-verb agreement errors and reducing repetitive words. The overall student perception of Criterion feedback was positive. However, vague feedback on organization and development and repeating words; and inaccurate feedback on mechanical errors, mostly spelling errors and articles; and the lack of feedback on errors need to be dealt with as they decrease the reliability of the AEE system as an instructional tool. The results and findings of this study were discussed while offering implications for future avenues of research.
참고문헌 (36)
- Attali, Y., "Exploring the feedback and revision features of Criterion", Paper presented at the meeting of the National Council on Measurement in Education (2004)
- Bai, L., "In the face of fallible AWE feedback: How do students respond?", Educational Psychology, 37(1), pp.67~81 (2017)
- Burstein, J., "AES for nonnative English speakers", Paper presented at the meeting of the ACL99 Workshop on Computer-Mediated Language Assessment and Evaluation of Natural Language Processing (1999)
- Burstein, J., "Automated essay evaluation: The Criterion online writing service", AI Magazine, 25(3), pp.27~27 (2004)
- Chen, C-F. E., "Beyond the design of automated writing evaluation : Pedagogical practices and perceived learning effectiveness in EFL writing classes", Language Learning & Technology, 12(2), pp.94~112 (2008)
- Chen, H. H. J., "Analyzing the grammar feedback of two automated writing evaluation systems : My Access and Criterion", English Learning and Teaching, 33(2), pp.1~43 (2009)
- Han, S., "Teaching Google search techniques in an L2 academic writing context", Language Learning & Technology, 21(3), pp.172~194 (2017)
- "CriterionSM: Promoting the standard. Race and Writing Assessment" (2008)[URL]
- Higuchi, A., "A study of automated L2 writing evaluation by Japanese college students : Eva Text Analysis", Studies in Education, 64, pp.1~9 (2012)
- Huang, S. J., "Automated versus human scoring : A case study in an EFL context", Electronic Journal of Foreign Language Teaching, 11(1), pp.149~164 (2014)
- 진경애, "영작문 자동 채점 시스템 개발 연구", 영어어문교육, 13(1), pp.235~259 (2007), 한국영어어문교육학회
- 김지은, "The Effectiveness of Automated Essay Scoring in an EFL College Classroom", 멀티미디어 언어교육, 17(3), pp.11~36 (2014), 한국멀티미디어언어교육학회[DOI]
- 김지은, "Automatic Adverb Error Correction in Korean Learners’ EFL Writing", International Journal of Contents, 5(3), pp.65~70 (2009), 한국콘텐츠학회
- 김태은, "Reflection on Using the Criterion Online Writing Evaluation Service", 멀티미디어 언어교육, 13(3), pp.59~83 (2010), 한국멀티미디어언어교육학회[DOI]
- 이은희, "Korean EFL College Writers' Perceptions of Criterion", 영어어문교육, 21(1), pp.199~216 (2015), 한국영어어문교육학회
- 이영주, "한국 대학생의 사례연구를 통해 살펴본 자동 쓰기 평가 피드백의 장기적인 효과", 영어교과교육, 16(4), pp.105~130 (2017), 한국영어교과교육학회[DOI]
- 이용원, "Investigating the feasibility of generic scoring models of e-rater for TOEFL iBT independent writing tasks", 영어교육연구, 28(1), pp.101~122 (2016), 팬코리아영어교육학회[DOI]
- Li, Z., "Automated writing evaluation (AWE) software in ESL writing classes: An activity theory perspective", Paper presentation at the meeting of the American Association for Applied Linguistics (2014)
- Li, J., "Rethinking the role of automated writing evaluation(AWE)feedback in ESL writing instruction", Journal of Second Language Writing, 27, pp.1~18 (2015)
- Liao, H. C., "Enhancing the grammatical accuracy of EFL writing by using an AWE-assisted process approach", System, 62, pp.77~92 (2016)
- Lim, H., "Review of Criterion for English language learning", Language Learning &Technology, 16(2), pp.38~45 (2012)
- Liu, S., "Investigating the application of automated writing evaluation to Chinese undergraduate English majors: A case study of “WriteToLearn.”", CALICO Journal, 33(1), pp.71~91 (2016)
- Luo, Y., "Comparison between peer feedback and automated feedback in college English writing : A case study", Open Journal of Modern Linguistics, 7(4), pp.197~215 (2017)
- Ma, K., "Improving EFL graduate students' proficiency in writing through an online automated essay assessing system", English Language Teaching, 6(7), pp.158~167 (2013)
- 맹은경, "The Effect and Teachers' Perception of Using an Automated Essay Scoring System in L2 Writing", 영어학연구, 16(1), pp.247~275 (2010), 한국영어학학회[DOI]
- 문영인, "Short-term Effects of Automated Writing Feedback and Users’ Evaluation of Criterion", 응용언어학, 27(4), pp.125~150 (2011), 한국응용언어학회
- Otoshi, J., "An analysis of the use of Criterion in a writing class in Japan", The JALT CALL Journal, 1(1), pp.30~38 (2005)
- Page, E. B., "The imminence of grading essays by computer", Phi Delta Kappan, 48, pp.238~243 (1966)
- Lorena Parra G., "Automated Writing Evaluation Tools in the Improvement of the Writing Skill", International Journal of Instruction, 12(2), pp.209~226 (2019), Modestum Limited[DOI]
- Shermis, M. D., "Handbook of automated essay evaluation: Current Applications and New Directions" (2013), Routledge
- Stevenson, M., "The effects of computer-generated feedback on the quality of writing", Assessing Writing, 19, pp.51~65 (2014)
- Weigle, S., "Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability", Language Testing, 27(3), pp.335~353 (2010)
- Wheeler, E., "Writing in engineering courses", Journal of Engineering Education, 89(4), pp.481~486 (2000)
- Zemach, D. E., "Writing essays : From paragraph to essay" (2013), Macmillan
- Zhang, Z. V., "Student engagement with teacher and automated feedback on L2writing", Assessing Writing, 36, pp.90~102 (2018)
- Zhang, Z., "Automated writing evaluation system : Tapping its potential for learner engagement", IEEE Engineering Management Review, 46(3), pp.29~33 (2018)