국가수준 학업성취도 평가 결과 분석에 적합한 인지진단모형 탐색: Fusion 모형과 DINA 모형의 비교
An Exploration of the Appropriate Cognitive Diagnostic Model for NAEA(National Assessment of Educational Achievement): A Comparison between Fusion and DINA Models
0
조회
0
다운로드
19
인용
0
참고문헌
키워드
초록
본 연구에서는 인지진단모형을 활용하여 국가수준 학업성취도 평가 결과를 분석하고, 학생들에게 프로파일 형태의 상세한 피드백을 줄 수 있는 방안을 마련하고자 자료 특성에 가장 적합한 모형을 탐색하였다. 선행연구를 통해 대규모 평가 분석에 자주 활용되는 것으로 파악된 인지진단모형인 Fusion Model과 DINA Model 중에서 학업성취도 평가 자료에 더 적합한 모형을 탐색하고자 하였다. 연구 결과, 약 60만명의 대단위 학생 자료를 포함하고 있는 학업성취도 평가 자료에 인지진단모형 분석을 위한 프로그램을 적용하여 개별 학생의 세부 인지요소 숙달 수준에 대한 프로파일 정보를 산출할 수 있는 가능성을 확인하였다. 또한, Fusion Model과 DINA Model을 비교한 결과 인지요소 숙달수준을 숙달/미숙달의 2단계가 아닌 상/중/하의 3단계로 분류하는 경우, DINA Model은 ‘상’이 0% 또는 ‘하’가 0%에 가깝게 나타나는 등 인지요소 숙달 수준이 편중되는 경향을 보였다. 따라서, 국가수준 학업성취도 평가 자료를 인지진단모형을 통해 2단계 이상의 숙달수준으로 분류하는 경우 Fusion Model이 더 적합한 것으로 나타났다.
Abstract
This study sought the way of providing assessment results as the type of elaborate profile by applying cognitive diagnostic model to NAEA(National Assessment of Educational Assessment). Because any cognitive diagnostic study applied to census data is unprecedented in Korea and abroad so far, it is aimed to find cognitive diagnostic model that is available for processing census data of a large-scale assessment whose size of data is more than 600,000 people. Prior studies identified that Fusion model and DINA model were often used for analyzing a large-scale assessment and there needs to search for more appropriate model for NAEA data among two models. As a result of comparing Fusion model and DINA model, if mastery probability of cognitive attribute is classified into three levels of high, middle and low, not two levels of mastery and non-mastery, DINA model showed that mastery probability of cognitive attribute tended to be unequally distributed such that ‘high’ or ‘low’ is close to 0%. Therefore, if NAEA data is analyzed through cognitive diagnostic model and mastery probability of cognitive attributes is classified into the mastery levels more than two, Fusion model is revealed as more appropriate.
참고문헌 (31)
- 김성훈, "인지구조모형에 근거한 학생의 지식상태의 진단-규칙장이론(Rule-space theory)의 적용-", 교육학연구, 43(1), pp.81~107 (2005), 한국교육학회
- 김성훈, "DINA 모형을 활용한 대규모 학업성취도 평가 결과 분석", 교육과정평가연구, 14(1), pp.177~200 (2011), 한국교육과정평가원
- 김수진, "Fusion Model에 의한수학 능력 진단을 위한 Q-행렬의 정교화", 교육평가연구, 21(2), pp.115~139 (2008), 한국교육평가학회
- 인지진단모형을 적용한 학업성취 프로파일 분석 및 결과보고 방안, "김희경", 한국교육과정평가원, ~, pp.연구보고서 RRE-2012-7 (2012)
- 인지진단모형을 통한 국가수준 학업성취도 평가 결과 분석 및 성적 보고 방법 탐색, "송미영", 한국교육과정평가원, ~, pp.연구보고서. RRE-2011-8 (2011)
- Buck, G., "Application of the rule-space procedure to language testing : Examining attributes of a free response listening test", Language Testing, 15(2), pp.119~157 (1998)
- Chiu, C., "Cluster analysis for cognitive diagnosis: An application to the 2001 PIRLS reading assessment" (2011)
- "College Board" (2012)[URL]
- de la Torre, J., "An empirically-based method of Q-matrix validation for the DINA model : Development and applications", Journal of Educational Measurement, 45, pp.343~362 (2008)
- de la Torre, J., "DINA model parameter estimation : A didactic", Journal of Educational Behavioral Statistics, 34, pp.115~130 (2009)
- "Applied Informative Assessment Research Enterprises", 2010
- psychometric diagnostic assessment likelihood-based classification techniques, In Cognitively diagnostic assessment, "Unified cognitive", pp.1995, Dibello, L.
- "Timberlake Consultants Press and Oxford", 2003
- Fu, J., "A polytomous extension of the Fusion Model and its Bayesian parameter estimation", 박사, pp.2005
- Fu, J., "Cognitively diagnostic psychometric models: An integrative review", Paper presented at the annual meeting of the Natonal Council on Measurement in Education (2007)
- Haertel, E. H., "Continuous and discrete structure models to map skill structure of achievement items", Psychometrika, 55, pp.477~494 (1990)
- Hartz, S., "A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality", 박사, pp.2002
- Cluster-wise assessment of cluster stability, "Hennig, C.", Department of Statistical Science, University of College London, ~, pp.Unpublished research report (2004)
- Modifications of the Arpeggio algorithm to permit analysis of NAEP, "Henson, R. A.", ~, pp.Unpublished ETS project report (2004)
- Huebner, A., "Practical issues concerning the application of the DINA model to CAT data", Paper presented at 2009 GMAC conference on Computerized Adaptive Testing (2009)
- Jang, E. E., "Demystifying a Q-Matrix for making diagnostic inferences about L2 reading skills", Language Assessment Quarterly, 6, pp.210~238 (2009)
- Junker, B. W., "Cognitive assessment models with few assumptions and connections with nonparametric item response theory", Applied Psychological Measurement, 25, pp.258~272 (2001)
- Landis, J. R., "The measurement of observer agreement for categorical data", Biometrics, 33, pp.159~174 (1977)
- Lee, Y-W., "Application of three cognitive diagnosis models to ESL reading and listening", Assessment : Language Assessment Quarterly, 6(3), pp.239~263 (2009)
- Li, H., "A cognitive diagnostic analysis of the MELAB reading test", Spaan fellow working papers in second or foreign language assessment, 9, pp.17~46 (2011)
- Park, C., "A multilevel IRT model for group-level diagnostic assessment with application to TIMSS", 박사, pp.2008
- Roussos, L. A., "The Fusion Model skill diagnosis system, In Cognitive diagnostic assessment for education", pp.275~318 (2007), Cambridge University Press
- Rupp, A. A., "The effects of Q-matrix misspecification on parameter estimates and misclassification rates in the DINA model", Educational and Psychological Measurement, 68, pp.78~96 (2008)
- "MRC Biostatistics Unit", 2003
- Tatsuoka, K. K., "Rule space : An approach for dealing with misconceptions based on item response theory", Journal of Educational Measurement, 20, pp.345~354 (1983)
- Tatsuoka, K. K., "A probabilistic model for diagnosing the misconceptions by the pattern classification approach", Journal of Educational Statistics, 10(1), pp.55~73 (1985)