The Effects of Test Method on L2 Reading and Listening Performance
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This paper examines how different test methods may tap different aspects of second language knowledge. It employs multiple-choice (MC) and constructed response (CR) items which yield distinct or convergent information in the computer delivered testing of English in its presentation of this factor. In order to examine the effects of test method, a CFA approach to a multitraitmultimethod design was adopted to examine the convergent as well as discriminant validity of the two skill area traits and the two test methods. After comparing the hypothesized model with a nested series of more restrictive models using 2 difference tests (Byrne, 2006; Widaman, 1985), based on the information from the path coefficients, and also using principal factor analyses, it was found that: 1) the skills are factorially separable, yet, highly correlated, 2) MC and CR methods are separable, and very weakly correlated, 3) there is thus a method effect, and 4) the MC method may not be suitable in sampling the unique ability characteristics of reading and listening skills.
Abstract
This paper examines how different test methods may tap different aspects of second language knowledge. It employs multiple-choice (MC) and constructed response (CR) items which yield distinct or convergent information in the computer delivered testing of English in its presentation of this factor. In order to examine the effects of test method, a CFA approach to a multitraitmultimethod design was adopted to examine the convergent as well as discriminant validity of the two skill area traits and the two test methods. After comparing the hypothesized model with a nested series of more restrictive models using 2 difference tests (Byrne, 2006; Widaman, 1985), based on the information from the path coefficients, and also using principal factor analyses, it was found that: 1) the skills are factorially separable, yet, highly correlated, 2) MC and CR methods are separable, and very weakly correlated, 3) there is thus a method effect, and 4) the MC method may not be suitable in sampling the unique ability characteristics of reading and listening skills.
참고문헌 (28)
- Ackerman, T. A., "A comparison of the information provided by essay, multiple-choice, and free-response tests", Applied Psychological Measurement, 12(2), pp.117~128 (1988)
- Bachman, L. F., "The construct validation of the FSI oral interview", Language Learning, 31(1), pp.67~85 (1981)
- Bae, J., "A latent variable approach to listening and reading : testing factorial invariance across two groups of children in the English", English two-way immersion program, 1998, pp.15(3), 380~414
- Bennett, R. E., "Construction versus choice in cognitive measurement: issues in constructed response, performance testing, and portfolio assessment", pp.1~27 (1993), Lawrence Erlbaum Associates, Inc., Publishers
- Bennett, R. E., "Construction versus choice in cognitive measurement: issues in constructed response, performance testing, and portfolio assessment" (1993), Lawrence Erlbaum Associates, Inc., Publishers
- Bentler, P. M., "EQS 6 structural equations program manual" (2004), Multivariate Software, Inc
- Birenbaum, M., "Open-ended versus multiplechoice response formats – it does make a difference for diagnostic purposes", Applied Psychological Measurement, 11(4), pp.385~395 (1987)
- Byrne, B. M., "Structural equation modeling with EQS and EQS Windows: basic concepts, applications, and programming" (2006), Lawrence Erlbaum Associates, Inc
- Campbell, D. T., "Convergent and discriminant validation by the multitrait-multimethod matrix", Psychological Bulletin, 56(2), pp.81~104 (1959)
- Cheng, H. F., "A Comparison of multiple-choice and open ended formats for the assessment of listening proficiency in English", Foreign Language Annals, 37(4), pp.544~555 (2004)
- Frederikson, N., "The real test bias", American Psychologist, 39(3), pp.193~202 (1984)
- Frederikson, N, "Diagnostic monitoring of skill and knowledge acquisition", pp.ix~xvii (1990), Lawrence Erlbaum Associates
- Frederikson, N., "Development of methods for selection and evaluation in undergraduate medical education" (1981), Educational Testing Service
- Henning, G., "Oral proficiency testing : comparing validities of interview, imitation and completion methods", Language Learning, 33(3), pp.315~332 (1983)
- IBM Corp. Released, "Released 2012. IBM SPSS Statistics for Windows, Version 21.0", IBM Corp, ~ (2012)
- In'nami, Y., "A meta-analysis of test format effects on reading and listening test performance : Focus on multiple-choice and open-ended formats", Language Testing, 26(2), pp.219~244 (2009)
- Kobayashi, M., "Method effects on reading comprehension test performance : text organization and response format", Language Testing, 19(2), pp.193~220 (2002)
- Kline, R. B., "Principles and practice of structural equation modeling" (2005), The Guilford Press
- 이정원, "A Comparison of Constructed Response Formats as Measures of EFL Reading Comprehension", 영어교육, 66(2), pp.149~167 (2011), 한국영어교육학회[DOI]
- Messick, S., "Construction versus choice in cognitive measurement: issues in constructed response, performance testing, and portfolio assessment", pp.61~74 (1993), Lawrence Erlbaum Associates, Inc.,Publishers
- Norris, J., "The handbook of second language acquisition", pp.717~761 (2003), Blackwell
- Shohamy, E., "Does the Testing Method Make a Difference? The Case of Reading Comprehension", Language Testing, 1(2), pp.147~170 (1984)
- Snow, R. E., "Construction versus choice in cognitive measurement: issues in constructed response, performance testing, and portfolio assessment", pp.45~60 (1993), Lawrence Erlbaum Associates, Inc., Publishers
- "Multitrait-multimethod comparisons of selected and constructed response assessments of language ability" (2008)[URL]
- Tono, Y., "CAN-DO list creation and the use of the CEFR-J guidebook" (2013), Taishukan
- Wainer, H., "Combining multiple-choice and constructed-response test scores : toward a Marxist theory of test construction", Applied Measurement in Education, 6(2), pp.103~118 (1993)
- Ward, W. C., "Construct validity of free-response and multiple-choice versions of a test", Journal of Educational Measurement, 17(1), pp.11~29 (1980)
- Widaman, K. F., "Hierarchically tested covariance structure models for multitrait-multimethod data", Applied Psychological Measurement, 9, pp.1~26 (1985)