Investigating the Construct Validity of Two Types of Second Language Writing Tests
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This study examined the construct validity of two types of expository essay writing–compare/contrast writing and problem/solution writing–which were administered as part of a high- stakes admissions test. The two types of writing were presented for each administration, from which examinees were allowed to choose one type. A total of 143 examinees took the writing test across five administrations, which were scored by two raters based on an analytic rubric. Examinee performance on each type of writing was separately analyzed using many-facet Rasch measurement. An analysis of the main effects of examinee ability, rater severity, and difficulty of the rating scales provided validity evidence in both types of writing. The two types differed in the difficulty of rating scales; specifically, the examinees in the compare/contrast writing group showed different levels of difficulty across the three rating components. This study provides implications for using two different types of writing for a high-stakes language test.
Abstract
This study examined the construct validity of two types of expository essay writing–compare/contrast writing and problem/solution writing–which were administered as part of a high- stakes admissions test. The two types of writing were presented for each administration, from which examinees were allowed to choose one type. A total of 143 examinees took the writing test across five administrations, which were scored by two raters based on an analytic rubric. Examinee performance on each type of writing was separately analyzed using many-facet Rasch measurement. An analysis of the main effects of examinee ability, rater severity, and difficulty of the rating scales provided validity evidence in both types of writing. The two types differed in the difficulty of rating scales; specifically, the examinees in the compare/contrast writing group showed different levels of difficulty across the three rating components. This study provides implications for using two different types of writing for a high-stakes language test.
참고문헌 (20)
- American Educational Research Association, "Standards for Educational and Psychological Testing" (1999), American Educational Research Association
- Bachman, L. F., "Statistical Analyses for Language Assessment" (2004), Cambridge University Press
- Eckes, T, "Examining Rater Effects in TestDaF Writing and Speaking Performance Assessments: A Many-facet Rasch Analysis", Language Assessment Quarterly, 2, pp.197~221 (2005)
- Ferris, D., "Teaching ESL Composition" (1998), Lawrence Erlbaum Associates
- Fulcher, G., "Testing Second Language Speaking" (2003), Longman
- Kim, H. J., "Investigating Raters’ Development of Rating Ability on a Second Language Speaking Assessment", Ed.D., Teachers College, pp.2011
- Lewkowicz, J., "Writing from Sources: Does Source Material Help or Hinder Students' Performance?", Paper Presented at the Annual International Language in Education Conference (1994)
- Linacre, J. M., "FACETS Rasch Measurement Computer Program, Version 3.66.3" (2010), Winsteps.com
- Lumley, T., "Rater Characteristics and Rater Bias: Implications for Training", Language Testing, 12, pp.54~71 (1995)
- McNamara, T. F., "Measuring Second Language Performance" (1996), Longman
- Milanovic, M., "Performance Testing, Cognition and Assessment: Studies in Language Testing 3", pp.1~17 (1996), University of Cambridge Local Examinations Syndicate
- Myford, C. M., "Building a Facets Specification File: The “Basics”"
- O’Sullivan, B., "Using Observation Checklists to Validate Speaking-test Tasks", Language Testing, 19(1), pp.33~56 (2002)
- Park, T., "Investigating the Construct Validity of the Community Language Program (CLP) English Writing Test", Ed.D., Teachers College, pp.2007
- Pollitt, A., "Calibrating Graded Assessments: Rasch Partial Credit Analysis of Performance in Writing", Language Testing, 4, pp.72~92 (1987)
- Shin, H. W., "Linguistic Differences in the Writing Performance of Adolescent EFL Learners: The Influence of Independent and Integrated Tasks", The Journal of Asia TEFL, 11, pp.125~148 (2014)
- Spaan, M., "A New Decade of Language Testing Research", pp.98~122 (1993), TESOL
- Weigle, S. C., "Investigating Rater", prompt Interactions in Writing Assessment: Quantitative and Qualitative Approaches, 1999, pp.6(2), 145~178
- Weigle, S. C., "Assessing Writing" (2002), Cambridge University Press
- Wiseman, C. S., "Investigating Selected Facets in Measuring Second Language Writing Ability Using Holistic and Analytic Scoring Methods", Ed.D., Teachers College, pp.2008