Creating Student Profiles Based on the Distractor Analysis of a Vocabulary Test
1
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
Despite some criticisms, multiple-choice (MC) tests are widely used in vocabulary tests for English language learners. It was pointed out that distractors affect examinees taking an MC-format vocabulary test. Thus, in this study the distractors of a vocabulary test were analyzed to create student profiles on the distractor categories. A 50-item MC test was administered to 2, 206 secondary school students. Then the distractors of the test were classified into three categories: word meaning,graphemic variation/affix, and contextual. Using these categories, the data sets were constructed to analyze the examinee’s tendency of choosing distractors when they could not find the correct answer. The results reveal that the patterns to choose three distractor types closely relate to examinees’ proficiency: high proficient students are likely to choose more meaning-based distractors but less graphemic variation/affix than low proficient students. By showing the patterns of choosing distractors, a distractor analysis of a vocabulary test will inform the examinees of what to correct and improve in the future as well as what they have mastered, and thus it will contribute to creating a detailed profile for the examinees.
Abstract
Despite some criticisms, multiple-choice (MC) tests are widely used in vocabulary tests for English language learners. It was pointed out that distractors affect examinees taking an MC-format vocabulary test. Thus, in this study the distractors of a vocabulary test were analyzed to create student profiles on the distractor categories. A 50-item MC test was administered to 2, 206 secondary school students. Then the distractors of the test were classified into three categories: word meaning,graphemic variation/affix, and contextual. Using these categories, the data sets were constructed to analyze the examinee’s tendency of choosing distractors when they could not find the correct answer. The results reveal that the patterns to choose three distractor types closely relate to examinees’ proficiency: high proficient students are likely to choose more meaning-based distractors but less graphemic variation/affix than low proficient students. By showing the patterns of choosing distractors, a distractor analysis of a vocabulary test will inform the examinees of what to correct and improve in the future as well as what they have mastered, and thus it will contribute to creating a detailed profile for the examinees.
참고문헌 (22)
- Bock, R. D., "Estimating item parameters and latent ability when the responses are scored in two or more nominal categories", Psychometrika, 37, pp.29~51 (1972)
- Bolt, D. M., "Application of a multidimensional nested logit model to multiple-choice test items", Psychometrika, 77, pp.339~357 (2012)
- Cizek, G. J., "Setting performance standards: Foundations, methods, and innovations" (2012), Routledge
- Cizek, G. J., "Standard setting: A practitioner’s guide to establishing and evaluating performance standards on tests" (2007), Sage
- Cizek, G. J., "Setting performance standards : Contemporary methods", Educational Measurement : Issues and Practice, 23(4), pp.31~50 (2004)
- Crocker, L., "Introduction to classical and modern test theory" (1986), Holt, Rinehart and Winston, Inc
- Cronbach, L. J., "Coefficient alpha and the internal structure of tests", Psychometrika, 16, pp.297~334 (1951)
- de Ayala, R. J., "The theory and practice of item response theory" (2009), Guilford
- Goodrich, H. C., "Distractor efficiency in foreign language testing", TESOL Quarterly, 11(1), pp.69~78 (1977)
- Haladyna, T. M., "A review of multiplechoice item-writing guidelines for classroom assessment", Applied Measurement in Education, 15(3), pp.309~334 (2002)
- Kline, P., "The handbook of psychological testing" (1999), Routledge
- "Winsteps.com", 2009
- McQueen, J., "Rasch scaling: how valid is it as the basis for content-referenced descriptors of test performance?, In The language testing cycle: From inception to washback", pp.137~187 (1996), ANU
- Perkins, K., "A construct definition study of a standardized ESL vocabulary test", Language Testing, 4(2), pp.125~141 (1987)
- "TestGraf: A program for the graphical analysis of multiple choice test and questionnaire data" (2000)[URL]
- Probabilistic models for some intelligence and attainment tests, "Rasch, G.", Danish Institute for Educational Research, ~ (1960)
- "R: A language and environment for statistical computing. R Foundation for Statistical Computing" (2010)[URL]
- Read, J., "Assessing vocabulary" (2000), Cambridge University Press
- Talamas, A., "From form to meaning : Stages in the acquisition of second language vocabulary", Bilingualism : Language and Cognition, 2(1), pp.45~58 (1999)
- Thissen, D., "A response model for multiple choice items", Psychometrika, 49, pp.501~519 (1984)
- Wang, W. C., "Rasch analysis of distractors in multiple-choice items", Journal of Outcome Measurement, 2(1), pp.43~65 (1998)
- Wesche, M., "Assessing second language vocabulary knowledge : Depth & breadth", Canadian Modern Language Review, 53(1), pp.13~40 (1996)