Interactions of Writers’ and Raters’ Argument Positions in Scoring L2 Argumentative Writing
2
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
The present study examines the interactions of writer’s and rater’s argument positions on the prompt in scoring L2 argumentative writing. Four Korean EFL raters scored essays composed by sixty eight Korean EFL university students. Their scores were statistically observed and analyzed according to the argument positions of writers and raters. Results of the study revealed that the majority of the students wrote supporting argument essays and yet, their scores were lower than those who wrote counter-argument essays. Although there were no statistically significant variations observed from the argument positions of writers, the raters assigned higher scores to the latter, assuming higher task difficulty and greater proficiency of the writers. Rater group variations were found depending on the assessment dimensions and the essay’s argument positions, though no statistically significant interactional effects were noted by the argument positions of writers and raters. Score variations were further revealed within each of the rater groups when they rated essays written from the same position as theirs. The study shows the interplay of writer’s and rater’s positions in scoring EFL argumentative essays and concludes that the rater’s positions can be the important sources of rater variation which influence their perceived task difficulty and final scores with implications and suggestions for further research.
Abstract
The present study examines the interactions of writer’s and rater’s argument positions on the prompt in scoring L2 argumentative writing. Four Korean EFL raters scored essays composed by sixty eight Korean EFL university students. Their scores were statistically observed and analyzed according to the argument positions of writers and raters. Results of the study revealed that the majority of the students wrote supporting argument essays and yet, their scores were lower than those who wrote counter-argument essays. Although there were no statistically significant variations observed from the argument positions of writers, the raters assigned higher scores to the latter, assuming higher task difficulty and greater proficiency of the writers. Rater group variations were found depending on the assessment dimensions and the essay’s argument positions, though no statistically significant interactional effects were noted by the argument positions of writers and raters. Score variations were further revealed within each of the rater groups when they rated essays written from the same position as theirs. The study shows the interplay of writer’s and rater’s positions in scoring EFL argumentative essays and concludes that the rater’s positions can be the important sources of rater variation which influence their perceived task difficulty and final scores with implications and suggestions for further research.
참고문헌 (29)
- Chapelle, C. A., "Validity arguments for diagnostic assessment using automated writing evaluation", Language Testing, 32, pp.385~405 (2015)
- Choi, Y. H., "GENOVA and FACETS analysis of an English writing test : Tasks, raters, and scales", English Teaching, 56(2), pp.125~142 (2001)
- 최연희, "Roles of Receptive and Productive Vocabulary Knowledge in L2 Writing through the Mediation of L2 Reading Ability", 영어교육, 72(1), pp.3~24 (2017), 한국영어교육학회
- Cumming, A., "Decision making while rating ESL", EFL writing tasks : A descriptive framework, 2002, pp.86, 67~96
- Hamp-Lyons, L., "Second language writing:Research insights for the classroom", pp.69~87 (1990), Cambridge University Press
- Hamp-Lyons, L., "Examining expert judgments of task difficulty on essay tests", Journal of Second Language Writing, 3, pp.49~68 (1994)
- Jeong, H., "Narrative and expository genre effects on students, raters, and performance criteria", Assessing Writing, 31, pp.113~125 (2017)
- Johnson, J. S., "The influence of rater language background on writing performance assessment", Language Testing, 26, pp.485~509 (2009)
- Susie Kim, "Exploring Rater Behaviors During a Writing Assessment Discussion", 영어교육, 70(1), pp.97~121 (2015), 한국영어교육학회[DOI]
- Leckie, G., "Rater effects on essay scoring : A multilevel analysis of severity drift, central tendency, and rater experience", Journal of Educational Measurement, 48, pp.399~418 (2011)
- 이은희, "Influences of Writing Genres and Proficiency on Lexical Diversity", 교과교육학연구, 22(2), pp.139~147 (2018), 교과교육연구소[DOI]
- 이성용, "Rating of Korean Students’ L2 Writing: Similarities and Differences Between Native and Non-Native Raters", 교과교육학연구, 16(3), pp.629~655 (2012), 교과교육연구소[DOI]
- Lim, G. S., "Prompt and rater effects in second language writing performance assessment", doctoral, pp.2009
- Lumley, T., "Assessment criteria in a large-scale writing test: What do they really mean to the raters", Language Testing, 19, pp.246~276 (2002)
- Lumley, T., "Rater characteristics and rater bias: Implications for training", Language Testing, 12, pp.54~71 (1995)
- Messick, S., "Validity of psychological assessment : Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning", American Psychologist, 50, pp.741~749 (1995)
- 박수경, "The Interplay of Task, Rating Scale, and Rater Background in the Assessment of Korean EFL Students’ Writing", 영어교육, 70(2), pp.55~82 (2015), 한국영어교육학회[DOI]
- Schaefer, E., "Rater bias patterns in an EFL writing assessment", Language Testing, 25, pp.465~493 (2008)
- Shi, L., "Native-and nonnative-speaking EFL teachers’evaluation of Chinese students’ English writing", Language Testing, 18, pp.303~325 (2001)
- Shohamy, E., "The effect of raters’ background and training on the reliability of direct writing tests", The Modern Language Journal, 76, pp.27~33 (1992)
- Staples, S., "Understanding first-year L2writing : A lexico-grammatical analysis across L1s, genres, and language ratings", Journal of Second Language Writing, 32, pp.17~35 (2016)
- Warschauer, M., "Automated writing evaluation : Defining the classroom research agenda", Language Teaching Research, 10, pp.157~180 (2006)
- Weigle, S. C., "Investigating rater", prompt interactions in writing assessment : Quantitative and qualitative approaches, 1999, pp.6, 145~178
- Weigle, S. C., "English language learners and automated scoring of essays : Critical considerations", Assessing Writing, 18, pp.85~99 (2013)
- Weigle, S. C., "Effects of task and rater background on the evaluation of ESL student writing : A pilot study", TESOL Quarterly, 37, pp.345~354 (2003)
- Wiseman, C. S., "Rater effects : Ego engagement in rater decision-making", Assessing Writing, 17, pp.150~173 (2012)
- Yeh, S. S., "Validation of a scheme for assessing argumentative writing of middle school students", Assessing Writing, 5, pp.123~150 (1998)
- Yun, W., "How raters’ and writers’ perception of a topic affect the scoring of compositions", College Teaching, 51, pp.115~118 (2003)
- Zhang, J., "Same text different processing? Exploring how raters’ cognitive and meta-cognitive strategies influence rating accuracy in essay scoring", Assessing Writing, 27, pp.37~53 (2016)