본문 바로가기
KERD
논문

Empirical Comparisons of Analytic Strategies for MIMIC DIF Analysis: A Potential Solution for Biased Anchor Set

0

조회

0

다운로드

0

인용

0

참고문헌

키워드

주제분류심리과학

초록

The purpose of this Monte Carlo study was to evaluate the performance of the multiple indicators and multiple causes (MIMIC) confirmatory factor analysis (CFA) for detecting differential item functioning (DIF). Specifically, this study compared different application strategies including two conventional testing approaches (forward-inclusion, backward-elimination) and five test statistic values (uncorrected or Bonferroni-corrected LR, △CFI of 0.01 or 0.002, △SRMR of 0.005) across conditions of different item type, test length, sample size, impact, and DIF type and DIF size in a target item and an anchor set. In addition, the author proposed an alternative testing approach (effects-coded backward-elimination) as a potential solution for arbitrary choice of a DIF-free anchor set. Simulation results indicated that when an anchor set was truly biased, only the proposed approach performed adequately under several conditions. False positive rates were controlled at the nominal alpha level (with Bonferroni-corrected LR) or slightly inflated (with uncorrected LR) as the DIF contamination rate in a scale decreased.

Abstract

The purpose of this Monte Carlo study was to evaluate the performance of the multiple indicators and multiple causes (MIMIC) confirmatory factor analysis (CFA) for detecting differential item functioning (DIF). Specifically, this study compared different application strategies including two conventional testing approaches (forward-inclusion, backward-elimination) and five test statistic values (uncorrected or Bonferroni-corrected LR, △CFI of 0.01 or 0.002, △SRMR of 0.005) across conditions of different item type, test length, sample size, impact, and DIF type and DIF size in a target item and an anchor set. In addition, the author proposed an alternative testing approach (effects-coded backward-elimination) as a potential solution for arbitrary choice of a DIF-free anchor set. Simulation results indicated that when an anchor set was truly biased, only the proposed approach performed adequately under several conditions. False positive rates were controlled at the nominal alpha level (with Bonferroni-corrected LR) or slightly inflated (with uncorrected LR) as the DIF contamination rate in a scale decreased.

참고문헌 (54)

  1. American Educational Research Association, "Standards for educational and psychological testing" (1999), American Educational Research Association
  2. Barrett,P, "Structural equation modeling: Adjusting model fit", Personality and Individual Differences, 42, pp.815~824 (2007)
  3. Bollen,K.A, "Structural equations with latent variables" (1989), Wiley
  4. Brannick,M.T, "Critical comments on applying covariance structure modeling", Journal of Organizational Behavior, 16, pp.201~213 (1995)
  5. Byrne, B. M, "The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process", Structural Equation Modeling, 13, pp.287~321 (2006)
  6. Camilli, G, "Measurement methods for the social sciences series: Methods for identifying biased test items (Vol. 4)" (1994), Sage
  7. Chen,F.F, "Sensitivity of goodness of fit indexes to lack of measurement invariance", Structural Equation Modeling, 14, pp.464~504 (2007)
  8. Cheung, G. W, "Testing factorial invariance across groups: A reconceptualization and proposed new method", Journal of Management, 25, pp.1~27 (1999)
  9. Cheung, G. W., "Evaluating goodness-of-fit indexes for testing measurement invariance", Structural Equation Modeling, 9, pp.233~255 (2002)
  10. Christensen, H, "The “common cause hypothesis” of cognitive aging: Evidence for not only a common factor but also specific associations of age with vision and grip strength in a cross-sectional analysis", Psychology and Aging, 16, pp.588~599 (2001)
  11. Dorans, N. J, "DIF detection and description: Mantel Haenszel and standardization, in Differential Item Functioning", pp.35~66 (1993), Lawrence Erlbaum
  12. Drasgow,F, "Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables are central issues", Psychological Bulletin, 95, pp.134~135 (1984)
  13. Everson, H. T, "Beyond individual differences: Exploring school effects on SAT scores", Educational Psychologist, 39, pp.157~172 (2004)
  14. Finch,H, "The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST and the IRT likelihood ratio test", Applied Psychological Measurement, 29, pp.278~295 (2005)
  15. Finch, H, "Estimation of MIMIC model parameters with multilevel data", Structural Equation Modeling, 18, pp.229~252 (2011)
  16. "Using MIMIC models to assess the influence of differential item functioning"[URL]
  17. Fleishman, J. A, "Impact of differential item functioning on age and gender differences in functional disability", Journal of Gerontology: Social Sciences, 57, pp.275~283 (2002)
  18. Gallo, J. J, "Age differences in the symptoms of depression: A latent trait analysis", Journal of Gerontology: Psychological Sciences, 49, pp.251~264 (1994)
  19. González-Romá, V, "Power and Type I error of the mean and covariance structure analysis model for detecting differential item functioning in graded response items", Multivariate Behavioral Research, 41, pp.29~53 (2006)
  20. Jones,R.N, "Identification of measurement differences between English and Spanish language versions of the Mini-Mental State Examination: Detecting differential item functioning using MIMIC modeling", Medical Care, 44, pp.124~133 (2006)
  21. Jöreskog, K. G, "Estimation of a model with multiple indicators and multiple causes of a single latent variable", Journal of the American Statistical Association, 10, pp.631~639 (1975)
  22. Kamata, A., "A note on the relation between factor analytic and item response theory models", Structural Equation Modeling, 15, pp.136~153 (2008)
  23. Lee,J, "Type I error and power of the MACS CFA for DIF detection: Methodological issues and resolutions", 박사, pp.2009
  24. Lee, J, "Methodological issues in using structural equation models for testing differential item functioning, in Cross-cultural data analysis: Methods and applications", pp.57~86 (2010), Routledge
  25. Little,T.D, "Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues", Multivariate Behavioral Research, 32, pp.53~76 (1997)
  26. Little, T. D., "A non-arbitrary method of identifying and scaling latent variables in SEM and MACS models", Structural Equation Modeling, 13, pp.59~72 (2006)
  27. MacIntosh, R., "Variance estimation for converting MIMIC model parameters to IRT parameters in DIF analysis", Applied Psychological Measurement, 27, pp.372~379 (2003)
  28. Mackinnon, A., "A short form of the Positive and Negative Affect Schedule: Evaluation of factorial validity and invariance across demographic variables in a community sample", Personality and Individual Differences, 27, pp.405~416 (1999)
  29. Maydeu-Olivares, A, "A cautionary note on using G2 (dif) to assess relative model fit in categorical data analysis", Multivariate Behavioral Research, 41, pp.55~64 (2006)
  30. McDonald,R.P, "Test theory: Unified treatment" (1999), Lawrence Erlbaum Associates
  31. Meade, A. W, "A Monte-Carlo study of confirmatory factor analytic tests of measurement equivalence", invariance, 2004, pp.11, 60~72
  32. Meade, A. W, "Power and sensitivity of alternative fit indices in test of measurement invariance", Journal of Applied Psychology, 93, pp.568~592 (2008)
  33. Mehta, P. D, "People are variables too: Multilevel structural equations modeling", Psychological Methods, 10, pp.259~284 (2005)
  34. Mellenbergh, G. J, "A unidimensional latent trait model for continuous item responses", Multivariate Behavioral Research, 29, pp.223~237 (1994)
  35. Millsap, R. E, "Four unresolved problems in studies of factorial invariance, in Contemporary psychometrics", pp.153~172 (2005), Lawrence Erlbaum Associates, Inc
  36. Muthén, B. O, "Some uses of structural equation modeling in validity studies: Extending IRT to external variables, in Test Validity", pp.213~238 (1988), Lawrence Erlbaum
  37. Muthén, B. O, "Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus" (2002), University of California and Muthén & Muthén
  38. Muthén, B. O., "Instructionally sensitive psychometrics: Application of a new IRT-based detection technique to mathematics achievement test items", Journal of Educational Measurement, 28, pp.1~22 (1991)
  39. Muthén, L.K, "Mplus user’s guide. (6th Ed.)" (1998), Muthén & Muthén
  40. Navas-Ara, M. J, "Effects of ability scale purification on identification of DIF", European Journal of Psychological Assessment, 18, pp.9~15 (2002)
  41. Raju, N. S., "Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory", Journal of Applied Psychology, 87, pp.517~529 (2002)
  42. Raudenbush, S. W, "Hierarchical linear models: Applications and data analysis methods (2nd ed.)" (2002), Sage
  43. Reise, S. P, "Confirmatory Factor Analysis and item response theory: Two approaches for exploring measurement invariance", Psychological Bulletin, 114, pp.552~566 (1993)
  44. STAT 9.2 user's guide, "SAS", pp.2002, SAS Institute
  45. Samejima,F, "Estimation of latent ability using a response pattern of graded scores", Psychometrika Monographs, No. 17, ~ (1969)
  46. Sörbom,D, "A general method for studying differences in factor means and factor structure between groups", British Journal of Mathematical and Statistical Psychology, 27, pp.229~239 (1974)
  47. Stark, S, "Examining the effects of differential item", test functioning (DIF, Journal of Applied Psychology, pp.2004, 89[DOI]
  48. Stark, S., "Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy", Journal of Applied Psychology, 91, pp.1292~1306 (2006)
  49. Teresi,J.A, "Overview of quantitative measurement methods: Equivalence, invariance, and differential item functioning in health applications", Medical Care, 44, pp.39~49 (2006)
  50. Vandenberg, R. J, "A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research", Organizational Research Methods, 3, pp.4~69 (2000)
  51. Woods,C.M, "Evaluation of MIMIC-model methods for DIF testing with comparison to two-group anlaysis", Multivariate Behavioral Research, 44, pp.1~27 (2009)
  52. Woods,C.M, "Testing for differential item functioning with measures of partial association", Applied Psychological Measurement, 33, pp.538~554 (2009)
  53. Woods, C. M., "Testing for nonuniform differential item functioning with multiple indictor multiple cause models", Applied Psychological Measurement, 35, pp.339~361 (2011)
  54. Woods, C. M, "Illustration of MIMIC-Model DIF Testing with the Schedule for Nonadaptive and Adaptive Personality", Journal of Psychopathology and Behavioral Assessment, 31, pp.320~330 (2009)

관련 논문AI추천

이 간행물의 다른 논문 (15건)