Investigating the Construct Validity of L2 Writing Performance Assessment Developed for Young Learners
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The study aimed to investigate the construct validity of a second language writing performance assessment designed for Korean elementary school children. Specifically, the validity of the test construct was established by examining the main effects of students’ writing ability, raters’ severity, and difficulty of analytic scoring criteria, along with the interaction effects of these elements. A total of 98 elementary school students took a writing test, which required them to write a short self-introduction. Their responses were scored by two raters based on an analytic scoring rubric. The ratings were analyzed using many-facet Rasch measurement, and the results indicated that the three facets (examinee, rater, and analytic component) functioned appropriately; moreover, there was no systematic interaction effect found from the bias analysis. The study provides both validity evidence for a writing achievement test and further implications for performance assessments in the elementary school English classrooms.
Abstract
The study aimed to investigate the construct validity of a second language writing performance assessment designed for Korean elementary school children. Specifically, the validity of the test construct was established by examining the main effects of students’ writing ability, raters’ severity, and difficulty of analytic scoring criteria, along with the interaction effects of these elements. A total of 98 elementary school students took a writing test, which required them to write a short self-introduction. Their responses were scored by two raters based on an analytic scoring rubric. The ratings were analyzed using many-facet Rasch measurement, and the results indicated that the three facets (examinee, rater, and analytic component) functioned appropriately; moreover, there was no systematic interaction effect found from the bias analysis. The study provides both validity evidence for a writing achievement test and further implications for performance assessments in the elementary school English classrooms.
참고문헌 (36)
- Bachman, L. F, "Some Reflections on Task-based Language Performance Assessment", Language Testing, 19, pp.453~476 (2002)
- Bachman, L. F., "Statistical Analyses for Language Assessment" (2004), Cambridge University Press
- Bachman, L. F, "Language Testing in Practice" (1996), Oxford
- Brown, J. D., "Essay Prompts and Topics: Minimizing the Effect of Mean Differences", Written Communication, 8, pp.533~556 (1991)
- Choi, C., "On How to Improve Foreign Language Teaching" (1986), Korean Educational Development Institute
- Cumming, A., "Decision Making While Rating ESL", EFL Writing Tasks: A Descriptive Framework, 2002, pp.86, 67~96
- Development Committee of the Sixth Curriculum for High School English, "The Report on the Revision of the English Curriculum for High School" (1992)
- 도명기, "초등영어 수행평가 루브릭의 개발과 적용의 방향", 초등영어교육, 9(2), pp.39~54 (2003), 한국초등영어교육학회
- Eckes, T, "Examining Rater Effects in Test DaF Writing and Weaking Performance Assessments: A Many-facet Rasch Analysis", Language Assessment Quarterly, 2, pp.197~221 (2005)
- Engelhard, G., Jr, "Examining Rater Errors in the Assessment of Written Composition with a Many-faceted Rasch Model", Journal of Educational Measurement, 31, pp.93~112 (1994)
- Ferris, D., "Teaching ESL Composition" (1998), Lawrence Erlbaum Associates
- Gyeonggi Provincial Office of Education, "Gyeonggi-do Curriculum" (2012), Gyeonggi Provincial Office of Education
- Johnson, J. S, "The Influence of Rater Language Background on Writing Performance Assessment", Language Testing, 26(4), pp.485~505 (2009)
- 김현정, "Developing a Rating Scale for Assessing Young Language Learners' Speaking Performance", Foreign Languages Education, 19(4), pp.1~25 (2012), 한국외국어교육학회
- 김정숙, "초등영어 듣기·말하기 수행평가 도구 개발과 실행", 초등영어교육, 8(1), pp.97~137 (2002), 한국초등영어교육학회
- Lee, W., "A Guide to Primary English Language Testing" (2009), JYbooks
- Lewkowicz, J., "Writing from Sources: Does Source Material Help or Hinder Students' Performance?", Paper Presented at the Annual International Language in Education Conference (1994)
- Lim, G. S, "The Development and Maintenance of Rating Quality in Performance Writing Assessment: A Longitudinal Study of New and Experienced Raters", Language Testing, 28, pp.543~560 (2011)
- Lumley, T, "Perceptions of Language-trained Raters and Occupational Experts in a Test of Occupational English Language Proficiency", English for Specific Purposes, 17, pp.347~367 (1998)
- Lumley, T., "Assessing Second Language Writing: The Rater’s Perspective" (2005), Peter Lang
- Lumley, T., "Rater Characteristics and Rater Bias: Implications for Training", Language Testing, 12, pp.54~71 (1995)
- McNamara, T. F., "Modelling Performance: Opening Pandora’s Box", Applied Linguistics, 16, pp.159~179 (1995)
- McNamara, T. F., "Measuring Second Language Performance" (1996), Longman
- Ministry of Education, Science and Technology, "The National English Language Curriculum (I)" (2008), Ministry of Education, Science and Technology
- Ministry of Education, Science and Technology., "An Introduction to the Elementary English National Curriculum" (2011), Ministry of Education, Science and Technology
- Park, T., "Investigating the Construct Validity of the Community Language Program (CLP) English Writing Test", Ed.D., Teachers College, pp.2007
- Pollitt, A., "Calibrating Graded Assessments: Rasch Partial Credit Analysis of Performance in Writing", Language Testing, 4, pp.72~92 (1987)
- Powers, D. E., "Test Takers’ Judgments of Essay Prompts: Perceptions and Performance", Educational Assessment, 6, pp.3~22 (1999)
- Sakyi, A. A., "Validation of Holistic Scoring for ESL Writing Assessment: How Raters Evaluate Compositions", pp.131~152 (2000), Cambridge University Press
- Schaefer, E, "Rater Bias Patterns in an EFL Writing Assessment", Language Testing, 25(4), pp.465~493 (2008)
- Shin, H. W., "Linguistic Differences in the Writing Performance of Adolescent EFL Learners: The Influence of Independent and Integrated Tasks", The Journal of Asia TEFL, 11(1), pp.125~148 (2014)
- Skehan, P., "A Cognitive Approach to Language Learning" (1998), Oxford University Press
- Schoonen, R., "The Assessment of Writing Ability: Expert Readers Versus Lay Readers", Language Testing, 14(2), pp.157~184 (1997)
- Smith, D., "Rater Judgments in the Direct Assessment of Competency Based Second Language Writing Ability", pp.159~189 (2000), National Centre for English Language Teaching and Research, Macquarie University
- Turner, C., "The Need for Impact Studies of L2 Performance Testing and Rating: Identifying Areas of Potential Consequences at all Levels of the Testing Cycle", pp.128~139 (2001), Cambridge University Press, University of Cambridge Local Examinations Syndicate
- Wiseman, C. S., "Investigating Selected Facets in Measuring Second Language Writing Ability Using Holistic and Analytic Scoring Methods", Ed.D., Teachers College, pp.2008
