Cognitive Diagnostic Assessment of English Grammar for Korean EFL Learners
2
조회
0
다운로드
6
인용
0
참고문헌
키워드
초록
If the purpose of an assessment is diagnosing examinees’ knowledge states to improve their learning, more fine-grained information than the overall level of their ability is necessary. Recent advances in diagnostic assessment triggered the development of cognitive diagnosis models (CDMs), such as the deterministic inputs, noisy “And”gate (DINA) model. Although CDMs for language assessments have been applied to reading or listening test data, a CDM may produce more practical results if the construct to measure has specific and well-defined skill attributes as in a grammar assessment. For this study, a grammar test consisting of 40 multiple-choice items was administered to 3,000 Korean learners of English as a foreign language. From the test items, a Q-matrix, which is an essential tool for CDMs, was constructed based on six grammar skills. As a result, skill profiles were obtained for all examinees. This diagnostic information can be used for tailored instruction. Issues with regard to applications of CDMs to language assessments are also discussed.
Abstract
If the purpose of an assessment is diagnosing examinees’ knowledge states to improve their learning, more fine-grained information than the overall level of their ability is necessary. Recent advances in diagnostic assessment triggered the development of cognitive diagnosis models (CDMs), such as the deterministic inputs, noisy “And”gate (DINA) model. Although CDMs for language assessments have been applied to reading or listening test data, a CDM may produce more practical results if the construct to measure has specific and well-defined skill attributes as in a grammar assessment. For this study, a grammar test consisting of 40 multiple-choice items was administered to 3,000 Korean learners of English as a foreign language. From the test items, a Q-matrix, which is an essential tool for CDMs, was constructed based on six grammar skills. As a result, skill profiles were obtained for all examinees. This diagnostic information can be used for tailored instruction. Issues with regard to applications of CDMs to language assessments are also discussed.
참고문헌 (22)
- Birenbaum,M, "Diagnosing knowledge states in algebra using the rule-space model", Journal for Research in Mathematics Education, 24(5), pp.442~459 (1993)
- Buck,G, "Application of the rule-space procedure to language testing:Examining attributes of a free response listening test", Language Testing, 15(2), pp.119~157 (1998)
- Buck,G, "The skills of reading:Rule-space analysis of a multiple-choice test of second language reading comprehension", Language Learning, 47(3), pp.423~466 (1997)
- Cheng,Y, "When cognitive diagnosis meets computerized adaptive testing:CDCAT", Psychometrika, 74, pp.619~632 (2009)
- Cizek,G.J, "An introduction to formative assessment:History,characteristics,and challenges.In Handbook of formative assessment", pp.3~17 (2010), Taylor and Francis
- Cronbach,L.J, "Coefficient alpha and the internal structure of tests", Psychometrika, 16, pp.297~334 (1951)
- DeCarlo,L.T, "On the analysis of fraction subtraction data:The DINA model,classification,latent class sizes,and the Q-matrix", Applied Psychological Measurement, 35(1), pp.8~26 (2011)
- de la Torre,J, "A cognitive diagnosis model for cognitively based multiple-choice options", Applied Psychological Measurement, 33(3), pp.163~183 (2009)
- de la Torre,J, "DINA model and parameter estimation:A didactic", Journal of Educational and Behavioral Statistics, 34(1), pp.115~130 (2009)
- DiBello,L.V, "Review of cognitive diagnostic assessment and a summary of psychometric models.In Handbook of statistics,Vol.26,Psychometrics", pp.979~1030 (2007), Elsevier
- Doornik,J.A, (2002), Timberlake Consultants Press
- Hambleton,R.K, "Fundamentals of item response theory" (1991), Sage
- Huebner,A, "A note on comparing examinee classification methods for cognitive diagnosis models", Educational and Psychological Measurement, 71(2), pp.407~419 (2011)
- Jang,E.E, "Demystifying a Q-matrix for making diagnostic inferences about L2 reading skills", Language Assessment Quarterly, 6(3), pp.210~238 (2009)
- Junker,B.W, "Cognitive assessment models with few assumptions,and connections with nonparametric item response theory", Applied Psychological Measurement, 25(3), pp.258~272 (2001)
- Koizumi,R, "Development and validation of a diagnostic grammar test for Japanese learners of English", Language Assessment Quarterly, 8(1), pp.53~72 (2011)
- Lee,J, "Diagnosis of subtraction bugs using Bayesian networks", Applied Psychological Measurement, 35(1), pp.27~47 (2011)
- Lee,Y.-W, "Application of three cognitive diagnosis models to ESL reading and listening assessments", Language Assessment Quarterly, 6(3), pp.239~263 (2009)
- Leighton,J.P, "The attribute hierarchy method for cognitive assessment:A variation on Tatsuoka’s rule-space approach", Journal of Educational Measurement, 41(3), pp.205~237 (2004)
- Sawaki,Y, "Q-matrix construction:Defining the link between constructs and test items in large-scale reading and listening comprehension assessments", Language Assessment Quarterly, 6(3), pp.190~209 (2009)
- Tatsuoka,K.K, "Cognitive assessment:An introduction to the rule space method" (2009), Routledge
- von Davier,M, "A general diagnostic model applied to language testing data", British Journal of Mathematical and Statistical Psychology, 61(2), pp.287~307 (2008)