차별기능의 원인파악을 위한 차별기능문항군 기법의 적용
Application of Differential Bundle Functioning Techniques for Understanding Sources of Differential Functioning
1
조회
0
다운로드
2
인용
0
참고문헌
키워드
초록
본 연구의 목적은 혼합문항 유형으로 구성된 평가도구에서 발생 가능한 차별기능의 원인을 파악하기 위하여 ‘차별기능문항군(Differential Bundle Functioning: DBF)’ 기법을 적용하는 것이다. 본 연구의 분석 자료는 PISA 2006 읽기 소양 검사를 응시한 한국과 핀란드 자료이며, 차별기능 분석을 위해서는 비모수적 접근 중 공통 로그 승산비 기법을 활용한 L-A LOR 검증(Liu & Agresti, 1996)과 DIF 효과 분산 추정치(Penfield & Algina, 2006) 및 PSIBTEST(Shealy & Stout, 1993)를 사용하였다. 문항 수준의 차별기능과 더불어서 PISA 평가틀의 하위차원인 읽기상황, 지문유형, 인지과정 및 문항유형에 따라 문항군(item bundle)을 구성하고, 전체 검사 및 이들 문항군 수준에서 집단 간 차별기능이 파악되는지를 검증하였다. 연구 결과, 문항수준에서는 약 59%의 DIF가 파악되었고, 전체 검사는 한국과 핀란드 집단 간에 ‘큰’ 수준의 차별기능을 보였다. 한편 DBF 분석 결과에서는 읽기상황, 지문유형 및 문항유형 등이 차별기능의 원인으로 탐색되었다. 마지막으로 본 연구 결과로부터 차별기능문항군 및 검사 분석이 평가도구 개발 및 검토 단계에 주는 유용성 및 시사점을 논의하였다.
Abstract
The purpose of this study is to demonstrate the applicability of differential bundle or test functioning(DBF or DTF) analyses to understand possible sources of differential functioning. The process is described on the reading literacy test from PISA 2006 that includes 27 mixed format items. Comparisons were made between Korea and Finland. Differential item functioning(DIF) analyses were conducted using L-A LOR tests(Liu & Agresti, 1996), whereas DBF or DTF analyses were conducted using variance-based DTF measures suggested by Penfield and Algina(2006) and PSIBTEST(Shealy & Stout, 1993). Bundles were created by using the sub-dimensions of assessment framework developed for PISA 2006 as well as item formats. About 59% of the items were identified as DIF, while some item bundles for text format or item types exhibited DBF. Implications of the results, as well as suggestions for future research related to test development or validation, are discussed.
참고문헌 (23)
- 손원숙, "국제학업성취도 평가의 DIF 원인 탐색을 위한 로지스틱 회귀분석의 적용", 교육평가연구, 23(2), pp.371~390 (2010), 한국교육평가학회
- PISA 2006결과 분석 연구:과학적 소양,읽기 소양,수학적 소양 수준 및 배경 변수 분석, "이미경", 한국교육과정평가원, ~, pp.연구보고.RRE 2007-1 (2007)
- Allalouf,A., "Identifying the causes of DIF in translated verbal items", Journal of Educational Measurement, 36(3), pp.185~198 (1999)
- Banks,K., "Using DDF in a Post Hoc Analysis to understand sourcesofDIF", EducationalMeasurement, 14(2), pp.103~118 (2009)
- Camilli,G., "Methodforidentifying biased test items(Vol.4)" (1994), Sage
- Dorans,N.J., "DIF detection and description : Mantel-Haenszeland standardization, In Differentialitem functioning" (1993), Erlbaum
- Douglas,J.A., "Item-bundle DIF hypothesis testing : Identifying suspect bundles and assessing their differentialfunctioning", JournalofEducationalMeasurement, 33, pp.465~484 (1996)
- Ercikan, K., "Effects of adaptations on comparabilityoftestitemsandtestscores, In SecondaryanalysisoftheTIMSSResults;A synthesisofcurrent research", pp.391~307 (2002), Kluwer
- Liu,I.-M, "Mantel-Haenszel-type inference for cumulativeoddsratiowith astratified ordinalresponse", Biometrics, 52, pp.1223~1234 (1996)
- Mantel,N., "Statisticalaspectsoftheanalysisof datafrom retrospectivestudiesofdisease", JournaloftheNationalCancer Institute, 22, pp.719~748 (1959)
- McCarty,F.A., "Identifyingpossible sourcesofdifferentialfunctioning using differentialbundle functioning with polytomously scored data", Applied Measurement in Education, 20(2), pp.205~225 (2007)
- Nandakumar,R., "SimultaneousDIF amplification and cancellation: Shealy-Stout'stestforDIF", JournalofEducationalMeasurement, 30(4), pp.293~311 (1993)
- OECD, "Assessing scientific,reading and mathematicalliteracy:A frameworkforPISA 2006" (2006), OECD
- OECD, "PISA 2006:ScienceCompetenciesforTommorow'sWorld. Volume1:Analysis" (2007), OECD
- Oshima,T.C., "DifferentialbundlefunctioningusingtheDFIT Framework:Procedures for identifying possible sources of differential functioning", Applied MeasurementinEducation, 11(4), pp.353~369 (1998)
- Penfield,R.D., "DIFAS : Differentialitem functioning analysis system", AppliedPsychologicalMeasurement, 29, pp.150~151 (2005)
- Penfield,R.D., "Assessingdifferentialstepfunctioninginpolytomous items using a common odds ratio estimator", Journalof Educational Measurement, 44(3), pp.187~210 (2007)
- Penfield,R.D., "Applying theLiu-Agrestiestimator ofthe cumulativecommon oddsratioto DIF detection in polytomous items", JournalofEducationalMeasurement, 40, pp.353~370 (2003)
- Penfield,R.D., "A generalized DIF effectvariance estimatorformeasuring unsigned differentialtestfunctioning in mixed formattests", JournalofEducationalMeasurement, 43(4), pp.295~312 (2006)
- Raju,N.S., "IRT-based internalmeasuresofdifferentialfunctioning ofitemsand tests", Applied PsychologicalMeasurement, 19, pp.353~368 (1995)
- Roussos,L.A., "A multidimensionality-basedDIF analysisparadigm", Applied Psychological Measurement, 20, pp.353~371 (1996)
- Shealy, R., "A model-based standardization approachthatseparates true bias", DIF from group differences and detects test bias, DIF, pp.Psychometrika, 1993[DOI]
- Wainer,H., "Differential testlet functioning", Journal of Educational Measurement, 28, pp.197~220 (1991)