Rating of Korean Students’ L2 Writing: Similarities and Differences Between Native and Non-Native Raters
1
조회
0
다운로드
13
인용
0
참고문헌
키워드
초록
The present study investigates the rating behavior of two native-speaking teachers and two Korean teachers of English when assessing forty essays written by Korean EFL students. Three research questions guided this study: (1) are there any similarities or differences in their rating behavior in holistic scoring? (2) Are there any similarities or differences in their rating behavior in analytic scoring? (3) And what factors are involved in these similarities or differences? Forty college students preparing for their TOEFL were asked to write an essay. Their essays were evaluated by the four teachers based on the TOEFL writing scoring rubric using the six point scale. The teachers read each essay twice. They were first required to give overall scores; then,in the second reading, they provided analytic assessments for four subareas. A paired t-test was performed to examine any discrepancy in holistic and analytic scoring between the two groups. Differences in holistic scoring appeared statistically significant: the Korean teachers gave higher scores than did their native counterparts. With respect to analytic scoring, the Korean raters also gave higher scores in all subareas, and the differences between the two groups were much greater than those in the holistic scoring. Interestingly, both groups gave the highest score in vocabulary, followed by grammar, content, and organization. A qualitative analysis indicated that all the teachers prioritized the logical flow of ideas and emphasized rhetorical features rather than linguistic features. Native teachers tended to score linguistic features based on native intuition. They also followed the Western writing convention in rating, while the Korean teachers applied the Korean rhetorical style.
Abstract
The present study investigates the rating behavior of two native-speaking teachers and two Korean teachers of English when assessing forty essays written by Korean EFL students. Three research questions guided this study: (1) are there any similarities or differences in their rating behavior in holistic scoring? (2) Are there any similarities or differences in their rating behavior in analytic scoring? (3) And what factors are involved in these similarities or differences? Forty college students preparing for their TOEFL were asked to write an essay. Their essays were evaluated by the four teachers based on the TOEFL writing scoring rubric using the six point scale. The teachers read each essay twice. They were first required to give overall scores; then,in the second reading, they provided analytic assessments for four subareas. A paired t-test was performed to examine any discrepancy in holistic and analytic scoring between the two groups. Differences in holistic scoring appeared statistically significant: the Korean teachers gave higher scores than did their native counterparts. With respect to analytic scoring, the Korean raters also gave higher scores in all subareas, and the differences between the two groups were much greater than those in the holistic scoring. Interestingly, both groups gave the highest score in vocabulary, followed by grammar, content, and organization. A qualitative analysis indicated that all the teachers prioritized the logical flow of ideas and emphasized rhetorical features rather than linguistic features. Native teachers tended to score linguistic features based on native intuition. They also followed the Western writing convention in rating, while the Korean teachers applied the Korean rhetorical style.
참고문헌 (37)
- Canagarajah, A. S., "Interrogating the ‘native speaker fallacy’: Non-linguistic roots, non-pedagogical results, In Non-Native Educators in English Language Teaching", pp.77~92 (1999), Lawrence Erlbaum
- Cheskey, J. A., "The effects of prior knowledge and audience on high school students' writing", Journal of Educational Research, 80, pp.304~313 (1987)
- Choi, Y., "Textual Coherence in English and Korean: An Analysis of Argumentative Writing by American and Korean Students", 박사, pp.1988
- Connor-Linton, J., "Cross-cultural comparison of writing standards. American ESL and Japanese EFL", World Englishes, 14(1), pp.99~115 (1995)
- Crusan, D., "An assessment of ESL writing placement assessment", Assessing Writing, 8, pp.17~30 (2002)
- Elbow, P., "On the nature of holistic scoring : An inquiry composed on email", Assessing Writing, 1, pp.91~107 (1994)
- Gordon, B. L., "Another look : Standardized tests for placement in college composition courses", Writing Program Administration, 10(3), pp.29~38 (1987)
- Greenberg, K., "Validity and reliability issues in direct assessment of writing", Writing Program Administration, 16, pp.7~22 (1992)
- Hamp-Lyons, L., "Raters respond to rhetoric in writing, In Interlingual Processes", pp.229~244 (1989), Gunter Narr
- Hamp-Lyons, L., "Issues and directions in assessing second language writing instruction in academic contexts, In Assessing Second Language Writing in Academic Contexts", pp.323~329 (1991), Ablex
- Harrington, S., "New visions of authority in placement test rating", Writing Program Administration, 22(1, pp.2) (1998), 53~84
- Homburg, T. J., "Holistic evaluation of ESL composition: Can it be validated objectively?", TESOL Quarterly, 18, pp.87~107 (1984)
- Huang, J., "How accurate are ESL students' holistic writing scores on large-scale assessments? A generalizability theory approach", Assessing Writing, 13(3), pp.201~218 (2008)
- Jenkins, J., "ELF at the gate: the position of English as a Lingua Franca", Humanising Language Teaching, 7(2) (2005)[DOI]
- Johns, A. M., "Faculty assessment of ESL student literacy skills: Implications for writing assessment, In Assessing Second Language Writing in Academic Contexts", pp.167~179 (1991), Ablex
- Kobayashi, H., "Factors affecting composition evaluation in an EFL context: Cultural rhetorical pattern and readers' background", Language Learning, 46(3), pp.397~433 (1996)
- Kobayashi, T., "Native and nonnative reactions to ESL compositions", TESOL Quarterly, 26(1), pp.81~112 (1992)
- Langer, J. A., "The effects of available information on responses to school writing tasks", Research in the Teaching of English, 18, pp.27~44 (1984)
- 이희경, "Native and nonnative rater behavior in grading Korean students' English essays", Asia Pacific Education Review, 10(3), pp.387~397 (2009), 교육연구소
- Perkins, K., "Using objective methods of attained writing proficiency to discriminate among holistic evaluations", TESOL Quarterly, 14, pp.61~69 (1980)
- Piazza, L., "French tolerance for grammatical errors made by Americans", Modern Language Journal, 64, pp.422~427 (1980)
- Ruetten, M. K., "Evaluating ESL students' performance on proficiency exams", Journal of Second Language Writing, 3(2), pp.85~96 (1994)
- Ha Ju Ryu, "Evaluation of Korean EFL students' English writing : a comparison of native- and non-native-speaking teachers' evaluation", 석사, pp.2011
- Santos, T., "Professors' reactions to the academic writing of nonnative-speaking students", TESOL Quarterly, 22(1), pp.69~90 (1988)
- Sheorey, R., "Error perceptions of native-speaking and non-native-speaking teachers of ESL", ELT Journal, 40(4), pp.306~312 (1986)
- Shi, L., "Native-and nonnative-speaking EFL teachers' evaluation of Chinese students' English writing", Language Testing, 18(3), pp.303~325 (2001)
- Snow, M. A., "Teacher Training for English as a Lingua Franca", Annual Review of Applied Linguistics, 26, pp.261~281 (2006)
- Song, B., "Using portfolios to access the writing of ESL students: a powerful alternative?", Journal of Second Language Writing, 11, pp.49~72 (2002)
- Song, B., "Do English and ESL faculty differ in evaluating the essays of native-English speaking and ESL students?", Journal of Second Language Writing, 5(2), pp.163~182 (1996)
- Sweedler-Brown, C. O., "ESL essay evaluation: The influence of sentence-level and rhetorical features", Journal of Second Language Writing, 2(1), pp.3~17 (1993)
- Tedick, D. J., "ESL writing assessment: subject-matter knowledge and its impact on performance", English for Specific Purposes, 9, pp.123~143 (1990)
- Thompson, R. W., "Application of assessment methods to instruction in a high school writing program", Evaluation and Program Planning, 16, pp.153~157 (1993)
- "What is Nonsul?" (2002)[URL]
- Vaughan, C., "Holistic assessment: what goes on in the rater's mind?, In Assessing Second Language Writing in Academic Contexts", pp.111~125 (1991), Ablex
- Westcott, W., "Holistic scoring as a teaching device", Teaching English in the Two year College, 11, pp.35~39 (1984)
- Yancey, K. B., "Looking back as we look forward: Historicizing writing assessment", College Composition and Communication, 50(3), pp.483~503 (1999)
- Zhang, W. X., "The Rhetorical Patterns Found in Chinese EFL Student Writers' Examination Essays in English and the Influence of These Patterns on Rater Response", 박사, pp.1999