Estimation of item parameters and examinees’ mastery probability in each domain of the Korean Medical Licensing Examination using a deterministic inputs, noisy “and” gate (DINA) model
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Purpose: The deterministic inputs, noisy “and” gate (DINA) model is a promising statistical method for providing useful diagnosticinformation about students’ level of achievement, as educators often want to receive diagnostic information on how examinees did oneach content strand, which is referred to as a diagnostic profile. The purpose of this paper was to classify examinees of the Korean Medical Licensing Examination (KMLE) in different content domains using the DINA model. Methods: This paper analyzed data from the KMLE, with 360 items and 3,259 examinees. An application study was conducted to estimate examinees’ parameters and item characteristics. The guessing and slipping parameters of each item were estimated, and statisticalanalysis was conducted using the DINA model. Results: The output table shows examples of some items that can be used to check item quality. The probabilities of mastery of eachcontent domain were also estimated, indicating the mastery profile of each examinee. The classification accuracy and consistency for 8content domains ranged from 0.849 to 0.972 and from 0.839 to 0.994, respectively. As a result, the classification reliability of the diagnostic classification model was very high for the 8 content domains of the KMLE. Conclusion: This mastery profile can provide useful diagnostic information for each examinee in terms of each content domain of theKMLE. Individual mastery profiles allow educators and examinees to understand which domain(s) should be improved in order tomaster all domains in the KMLE. In addition, all items showed reasonable results in terms of item parameters.
Abstract
Purpose: The deterministic inputs, noisy “and” gate (DINA) model is a promising statistical method for providing useful diagnosticinformation about students’ level of achievement, as educators often want to receive diagnostic information on how examinees did oneach content strand, which is referred to as a diagnostic profile. The purpose of this paper was to classify examinees of the Korean Medical Licensing Examination (KMLE) in different content domains using the DINA model. Methods: This paper analyzed data from the KMLE, with 360 items and 3,259 examinees. An application study was conducted to estimate examinees’ parameters and item characteristics. The guessing and slipping parameters of each item were estimated, and statisticalanalysis was conducted using the DINA model. Results: The output table shows examples of some items that can be used to check item quality. The probabilities of mastery of eachcontent domain were also estimated, indicating the mastery profile of each examinee. The classification accuracy and consistency for 8content domains ranged from 0.849 to 0.972 and from 0.839 to 0.994, respectively. As a result, the classification reliability of the diagnostic classification model was very high for the 8 content domains of the KMLE. Conclusion: This mastery profile can provide useful diagnostic information for each examinee in terms of each content domain of theKMLE. Individual mastery profiles allow educators and examinees to understand which domain(s) should be improved in order tomaster all domains in the KMLE. In addition, all items showed reasonable results in terms of item parameters.
참고문헌 (11)
- Jimmy de la Torre, "DINA Model and Parameter Estimation: A Didactic", Journal of Educational and Behavioral Statistics, 34(1), pp.115~130 (2009), American Educational Research Association (AERA)[DOI]
- Jimmy de la Torre, "Higher-order latent trait models for cognitive diagnosis", Psychometrika, 69(3), pp.333~353 (2004), Springer Science and Business Media LLC[DOI]
- Yoon Soo Park, "An Extension of the DINA Model Using Covariates", Applied Psychological Measurement, 38(5), pp.376~390 (2014), SAGE Publications[DOI]
- Rupp AA, "Diagnostic assessment: theory, methods, and applications" (2010), Guilford Press
- Yigal Attali, "Validating Classifications From Learning Progressions: Framework and Implementation", ETS Research Report Series, 2019(1), pp.1~20 (2019), Wiley[DOI]
- Yoon Soo Park, "Reporting Subscore Profiles Using Diagnostic Classification Models in Health Professions Education", Evaluation & the Health Professions, 43(3), pp.149~158 (2020), SAGE Publications[DOI]
- Dong Gi Seo, "Usefulness of the DETECT program for assessing the internal structure of dimensionality in simulated data and results of the Korean nursing licensing examination", Journal of Educational Evaluation for Health Professions, 14, pp.32~ (2017), Korea Health Personnel Licensing Examination Institute[DOI]
- Richard A. Duschl, "Learning progressions: framing and designing coherent sequences for STEM education", Disciplinary and Interdisciplinary Science Education Research, 1(1) (2019), Springer Science and Business Media LLC[DOI]
- "CDM: cognitive diagnosis modeling: R package version 7.5-15"[URL]
- Robert C. MacCallum, "Power analysis and determination of sample size for covariance structure modeling.", Psychological Methods, 1(2), pp.130~149 (1996), American Psychological Association (APA)[DOI]
- Ying Cui, "Estimating Classification Consistency and Accuracy for Cognitive Diagnostic Assessment", Journal of Educational Measurement, 49(1), pp.19~38 (2012), Wiley[DOI]