Differential Item Functioning Analysis of an EFL Vocabulary Test
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
Differential item functioning (DIF) has been studied as a statistical method of finding item bias, and DIF studies have been conducted on tests of verbal abilities such as reading comprehension and vocabulary knowledge. Recently, researchers of language testing sought to understand the causes of DIF. The purpose of this study was to find possible sources of explaining DIF on an EFL vocabulary test, which can then benefit EFL learners and teachers as well. This study thus applied three DIF detection methods (the likelihood ratio test, SIBTEST, and the Mantel-Haenszel test), followed by a regression analysis using item categories and difficulties as predictors of explaining DIF. As a result, only the “Academic” category was statistically significant, positively contributing to DIF in favor of male examinees. Male examinees were better at academic lexical words than female learners of EFL at the same vocabulary knowledge level. This also means that female learners were relatively strong at nonacademic words in comparison to male EFL learners. This finding may lead to more effective curriculum of teaching English. The need for more evidence from replicated studies with better item categorization is also warranted.
Abstract
Differential item functioning (DIF) has been studied as a statistical method of finding item bias, and DIF studies have been conducted on tests of verbal abilities such as reading comprehension and vocabulary knowledge. Recently, researchers of language testing sought to understand the causes of DIF. The purpose of this study was to find possible sources of explaining DIF on an EFL vocabulary test, which can then benefit EFL learners and teachers as well. This study thus applied three DIF detection methods (the likelihood ratio test, SIBTEST, and the Mantel-Haenszel test), followed by a regression analysis using item categories and difficulties as predictors of explaining DIF. As a result, only the “Academic” category was statistically significant, positively contributing to DIF in favor of male examinees. Male examinees were better at academic lexical words than female learners of EFL at the same vocabulary knowledge level. This also means that female learners were relatively strong at nonacademic words in comparison to male EFL learners. This finding may lead to more effective curriculum of teaching English. The need for more evidence from replicated studies with better item categorization is also warranted.
참고문헌 (28)
- Abbott, M. L, "A confirmatory approach to differential item functioning on an ESL reading assessment", Language Testing, 24, pp.7~36 (2007)
- Bolt, D. M, "A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods", Applied Measurement in Education, 15, pp.113~141 (2002)
- Camilli, G, "Methods for identifying biased test items" (1994), Sage
- Chen, Z, "Linguistic and cultural bias in language proficiency tests", Language Testing, 2, pp.155~163 (1985)
- Daller, H, "Modelling and assessing vocabulary knowledge" (2007), Cambridge University Press
- Douglas, J, "Item-bundle DIF hypothesis testing: Identifying suspect bundles and assessing their differential functioning", Journal of Educational Measurement, 33, pp.465~484 (1996)
- Embretson, S. E, "Item response theory for psychologists" (2000), Lawrence Erlbaum
- Holland, P. W, "Differential item performance and the Mantel-Haenszel procedure. In Test validity", pp.129~145 (1988), Lawrence Erlbaum
- Holland, P. W, "Differential item functioning" (1993), Lawrence Erlbaum
- Hughes, A, "Testing for language teachers (2nd ed.)" (2003), Cambridge University Press
- Hyde, J, "Gender differences in verbal ability: A meta-analysis", Psychological Bulletin, 104, pp.53~69 (1988)
- Kim, Mikyung, "Detecting DIF across the different language groups in a speaking test", Language Testing, 18, pp.89~114 (2001)
- Maller, S. J, "Differential item functioning in the WISC-III: Item parameters for boys and girls in the national standardization sample", Educational and Psychological Measurement, 61, pp.793~817 (2001)
- Pae, Tae-Il, "DIF for examinees with different academic backgrounds", Language Testing, 21, pp.53~73 (2004)
- Pearson, P. D, "Vocabulary assessment: What we know and what we need to learn", Reading Research Quarterly, 42, pp.282~296 (2007)
- Penfield, R. D, "Assessing differential item functioning in performance assessment: Review and recommendations", Educational Measurement: Issues and Practice, 19, pp.5~15 (2000)
- RAND Reading Study Group, "Reading for understanding: Toward an R&D program in reading comprehension" (2002), RAND
- Read, J, "Assessing vocabulary" (2000), Cambridge University Press
- Read, J, "A framework for second language vocabulary assessment", Language Testing, 18, pp.1~32 (2001)
- Roussos, L. A, "A multidimensionality-based DIF analysis paradigm", Applied Psychological Measurement, 20, pp.355~371 (1996)
- Ryan, K. E, "Differential item functioning on two tests of EFL proficiency", Language Testing, 9, pp.12~29 (1992)
- Sasaki, M, "A comparison of two methods for detecting differential item functioning in an ESL placement test", Language Testing, 8, pp.95~115 (1991)
- Shealy, R, "A model-based standardization approach that separates true bias", DIF from group ability differences and detects test bias, DIF, pp.Psychometrika, 1993[DOI]
- Stout, W. F, "A new item response theory modeling approach with applications to unidimensionality assessment and ability estimation", Psychometrika, 55, pp.293~325 (1990)
- Stout, W. F, "SIBTEST user manual. Unpublished manuscript", University of Illinois at Urbana-Champaign, ~ (1992)
- Takala, S, "Test fairness: A DIF analysis of an L2 vocabulary test", Language Testing, 17, pp.323~340 (2000)
- (2001)[URL]
- Thissen, D, "Use of item response theory in the study of group differences in trace lines. In Test validity", pp.147~169 (1988), Lawrence Erlbaum