영어작문의 과정적 글쓰기에서 자동작문평가(Automated Essay Scoring)의 효과
Integration of Automated Essay Scoring (AES) in the Process of Writing in ESL/EFL Classes
2
조회
0
다운로드
4
인용
0
참고문헌
키워드
초록
영어 학습자의 증가와 국제화된 의사소통은 영어작문 능력에 대한 필요성을 증대시키고 있다. 실증적 연구의 결과와 언어 학습 이론들이 효과적인 작문 수업을 위한 피드백의 중요성을 강조하고 있지만, 효과적인 피드백을 위해 교사가 들여야 하는 시간과 노력을 고려해볼 때, 현실적 적용의 어려움이 많다. 이에 따라, 교사의 피드백을 도울 수 있는 도구로 Automated Essay Scoring(AES)가 점차 관심을 받게 되었다. AES는 학생과 교사에게 작문을 돕는 형성적 평가 도구로 활용될 수 있다. 이 연구는 AES가 영어 작문의 질과 정확성의 향상에 미치는 영향을 AES 통합의 유형과 영어 학습 환경에 따라 비교 연구하였다. AES program으로는 ETS에서 개발한 Criterion을 사용하였으며, AES 통합 유형은 3가지(No-AES, 선택적 AES, 통합적 AES)로 구분하고 영어 학습 환경은 미국(ESL)과 한국(EFL)로 나누었다. 한국과 미국의 대학에서 영어작문 수업을 듣는 대학생 172명이 참여하였으며, 학생들의 작문자료를 수집하여 분석하였다. 학생들의 작문은 전체적인 질(質)과 정확성 향상의 측면에서 평가되었다. 분석 결과, ESL과 EFL의 영어학습 환경 모두에서 통합적 AES 집단이 다른 두 집단보다 작문의 질과 정확성의 향상에 더욱 우수한 결과를 보였다. 하지만 영어 학습 환경이 작문의 질과 정확성에 미치는 영향은 통계적 유의성을 보이지 않았다. 연구 결과는 자동 작문 평가 시스템이 과정적 글쓰기와 통합적으로 활용될 때, 작문의 질과 정확성의 향상에 이바지할 수 있음을 밝혔다. 자동 작문 평가 시스템의 통합적 활용은 연구에서 제시되 피드백 모형의 효과를 실증적으로 보여준 것으로 향후 자동 평가 시스템과 같은 교육 프로그램의 활용 모델을 제시하였다.
Abstract
The growing number of English Language Learners(ELLs) and globalized communications are factors making English writing instruction a critical educational concern in many countries around the world. While empirical research findings and theories related to language learning have suggested instructive interactions, the considerable amount of time and efforts required for instructive interactions have hindered effective writing instruction. As a result, Automated Essay Scoring(AES) had gain attention of educators as a formative assessment tool, which provides assistive writing resources to teachers and students. This study examined the effects of types of AES integration-No-AES, Optional-AES, and Integrated-AES-and English learning contexts-ESL and EFL-in improving English writing quality. One hundred seventy two ELL students participated in this study from an ESL program at a U. S. university and an EFL program at a Korean university. Students received writing instruction under different types of AES integration settings and English learning contexts for the 8-week intervention period. Students’ essays were collected and assessed for analysis. The analysis results revealed that the types of AES integration significantly influenced writing quality of each writing assignments in terms of holistic quality and accuracy. The Integrated-AES group received significantly higher holistic scores of the first drafts and the final revisions than the Optional-AES and the No-AES groups. In addition, the Integrated-AES group improved holistic scores and reduced errors significantly more than the Optional-AES and No-AES groups. The findings suggest that AES is an effective instructional tool as a formative assessment tool when it is integrated with writing instruction and process.
참고문헌 (38)
- Bransford, J. D, "How People Learn: Brain, Mind, Experience, and School" (2000), National Academies Press
- Chen, C. E, "Beyond the Design of Automated Writing Evaluation: Pedagogical Practices and Perceived Learning Effectiveness in EFL Writing Classes", Language Learning & Technology, 12(2), pp.94~112 (2008)
- Feedback on compositions: Teacher and student verbal reports, "Cohen, A. D", 155~177 (1990)
- Dikli, S, "An overview of automated scoring of essays", Journal of Technology, Learning, and Assessment, 5(1), pp.1~36 (2006)
- Dikli, S, "Automated essay scroing in an English as a second language setting", 박사, pp.2007
- Elliot, S, "The impact of MY Access!™ use on student writing performance: A technology overview and four studies" (2004), Annual Meeting of the American Educational Research Association
- Ferris, D, "Student reactions to teacher response in multiple-draft composition classrooms", TESOL Quarterly, 29(1), pp.33~53 (1995)
- Ferris, D, "The case for grammar correction in L2 writing classes: A response to Truscott (1996)", Journal of Second Language Writing, 8(1), pp.1~11 (1996)
- Ferris, D, "The “grammar correction” debate in L2 writing: Where are we, and where do we go from here? (and what do we do in the meantime…?)", Journal of Second Language Writing, 13(1), pp.49~62 (2004)
- Ferris, D, "Does error feedback help student writers? New evidence on the short-and between-pre-and-post-test effects of written error correction, in Feedback in second language writing", pp.81~104 (2006), Contexts and issues
- Ferris, D, "Teaching ESL composition: Purpose, process, and practice" (2005), Routledge
- Ferris, D, "Teacher commentary on student writing: Descriptions & implications", Journal of Second Language Writing, 6(2), pp.155~182 (1997)
- Freedman, S. W, "The Evaluation of, and Response to Student Writing: A Review", The 68th Annual Meeting of the American Educational Research Association (1984)
- Gass, S. M, "Input, interaction, and output in second language acquisition, in Theories in Second Language Acquisition", pp.175~199 (2007), Mahwah
- Hedgcock, J, "Feedback on Feedback: Assessing Learning Receptivity to Teacher Response in L2 Composing", Journal of Second Language Writing, 3(2), pp.141~163 (1994)
- Hyland, F, "Feedback in second language writing: Contexts and issues" (2006), Cambridge University Press
- Kachru, B. B, "The alchemy of English" (1986), Pergamon Press
- Krashen, S, "Writing: Research, Theory, and Applications" (1984), Pergamon Press
- Leki, I, "Coaching from the margins", pp.57~68 (1990), Research Insights for the Classroom
- Lee, I, "ESL Learners' performance in error correction in writing: Some implications for teaching", System, 25(4), pp.465~477 (1997)
- Lightbown, P. M, "How languages are learned" (1999), Oxford University Press
- Long, M. H, "Focus on form: Theory, research, and practice, in Focus on form in classroom second language acquisition", pp.15~41 (1998), Cambridge University Press
- Nayar, P. B, "ESL", EFL dichotomy today: Language politics or pragmatics?, 1997, pp.31(1), 9~37
- Palincsar, A. S, "Social constructivist perspectives on teaching and learning", Annual Review of Psychology, 49(1), pp.345~375 (1998)
- Schulz, R. A, "Cultural Differences in Student and Teacher Perceptions concerning the Role of Grammar Instruction and Corrective Feedback", The Modern Language Journal, 85(2), pp.244~258 (2001)
- Shermis, M. D, "The impact of automated essay scoring on high stakes writing assessments", Annual Meeting of the National Council on Measurement in Education (2004)
- Shermis, M. D, "The Impact of Automated Essay Scoring on Writing Outcomes Online Submission", The Annual Meetings of the National Council on Measurement in Education, pp.25~27 (2008)
- Silva, T, "Toward an understanding of the distinct nature of L2 writing: The ESL research and its implications", TESOL Quarterly, 27(4), pp.657~677 (1993)
- Sommers, N, "Responding to student writing", pp.148~156 (1982), College Composition and Communication
- Truscott, J, "The case against grammar correction in L2 writing classes", Language Learning, 46, pp.327~369 (1996)
- Vygotsky, L. S, "Mindinsociety" (1978), Harvard University Press
- Warden, C, "Improving feedback while decreasing teacher burden in ROC ESL business English writing classes. Explorations in English for professional communications", pp.125~137 (1995), City University of Hong Kong
- Ware, P, "Electronic feedback and second language writing, in Feedback in second language writing: Contexts and issues", pp.105~122 (2006), Cambridge University Press
- Warschauer, M, "The Changing Global Economy and the Future of English Teaching", TESOL Quarterly, 34(3), pp.511~535 (2000)
- Warschauer, M, "Automated writing evaluation: defining the classroom research agenda", Language Teaching Research, 10(2), pp.157~180 (2006)
- Wood, D, "The role of tutoring in problem solving", Journal of Child Psychology and Psychiatry, 17(2), pp.89~100 (1976)
- Zamel, V, "Writing: The process of discovering meaning", TESOL Quarterly, 16(2), pp.195~209 (1982)
- Zamel, V, "Responding to Student Writing", TESOL Quarterly, 19(1), pp.79~101 (1985)