본문 바로가기
KERD
논문

반복측정 자료에 기반한 변화의 집단차 분석 방법: 차이점수 모형과 공분산분석 모형 비교

How to analyze group difference in change: Comparing difference score model and analysis of covariance model

석혜원|이영수
한국심리학회 (The Korean Psychological Association)|2024.09|한국심리학회지:일반 (Korean Journal of Psychology: General)|제43권 제3호|pp. 231-260|DOI 10.22257/kjp.2024.9.43.3.231

0

조회

0

다운로드

0

인용

0

참고문헌

키워드

공분산분석ANCOVA인과추론causal inferencetreatment effect차이점수처치효과Lord의 역설difference scoreLord’s paradox
주제분류심리과학

초록

심리학 여러 분야에서 사전, 사후 시점에 반복측정한 자료에 기반하여 처치집단과 통제집단 간 변화의 차이를 살펴보는 연구를 자주 볼 수 있다. 이때 연구자들이 가장 널리 사용하는 분석 모형은 차이점수 모형과 공분산분석 모형이다. 하지만, 이 두 모형은 때로 상이한 결과를 산출하기 때문에, 많은 연구자들은 언제 어떠한 방법을 사용해야 하는지 혼란을 겪고 있다. 이에, 본 연구는 두 모형을 이론적, 경험적으로 비교한 연구를 개관하고, 이에 기반하여 언제 어느 모형을 사용하는 것이 적절한지 가이드라인을 제시하고자 하였다. 이를 위해, 우선 두 모형을 각각 소개하고, 예시 자료를 통해 두 모형이 서로 다른 분석 결과를 산출할 수 있음을 보였다. 다음으로, 차이점수 사용과 관련된 논쟁을 살펴보고, 차이점수에 대한 전통적인 비판이 지나치게 단순화된 가정과 잘못된 믿음에 근거한 것임을 확인하였다. 이어서, 인과추론의 맥락에서 두 방법이 어떤 숨겨진 가정을 내포하고 있는지 이론적으로 살펴보고, 이러한 가정 및 시뮬레이션 연구 결과들에 기반하여, 실험집단에 참여자를 할당하는 방법과 분석 목적에 따라 어떤 방법을 사용하는 것이 적절한지 가이드라인을 제시하였다. 본 연구를 통해 연구자들이 보다 적절한 분석 방법을 선택하고, 엄밀하고 효과적으로 분석을 수행하는 데 도움을 제공할 수 있을 것으로 기대된다.

Abstract

In various fields of psychology, researchers commonly investigate difference in changes between treatment and control groups by analyzing data gathered before and after interventions. The most widely used analytical methods used in such cases are the difference score model and the analysis of covariance model. However, since these models may produce conflicting outcomes, researchers often get confused when determining the most appropriate method for their studies. Therefore, this study aims to offer an in-depth examination of the theoretical and empirical difference between these models, aiming to furnish guidelines on when to use which method. Initially, we introduce and illustrate each model using an example dataset to showcase their potential divergent analytical outcomes. Subsequently, we scrutinize the debate on the use of difference scores, debunking traditional criticisms grounded in oversimplified assumptions and misunderstandings. We then delve into the implicit assumptions of both models within the framework of causal inference and, drawing upon these assumptions and findings from simulation studies, furnish recommendations for selecting the appropriate method under different participant allocation methods and analytical purposes. This study endeavors to empower researchers in making informed decisions regarding their choice of analytical methods, thereby enhancing the rigor and efficacy of their investigations.

참고문헌 (50)

  1. Castro-Schilo, L., "Using residualized change versus difference scores for longitudinal research", Journal of Social and Personal Relationships, 35(1), pp.32~58 (2018)[DOI]
  2. Chiou, J., "The Reliability of Difference Scores: A Re-Examination", Journal of Consumer Satisfaction, Dissatisfaction &Complaining Behavior, 9, pp.158~167 (1996)
  3. correlation analysis for the behavioral sciences, "Applied multiple regression", pp.2003, Cohen, J.
  4. Cronbach, L. J., "How should we measure"change"-or should we", Psychological Bulletin, 74(1), pp.68~80 (1970)[DOI]
  5. Culpepper, S. A., "Using analysis of covariance(ANCOVA)with fallible covariates", Psychological Methods, 16(2), pp.166~178 (2011)[DOI]
  6. Furby, L., "Interpreting regression toward the mean in developmental research", Developmental Psychology, 8(2), pp.172~179 (1973)[DOI]
  7. Galton, F., "Regression Towards Mediocrity in Hereditary Stature", The Journal of the Anthropological Institute of Great Britain and Ireland, 15, pp.246~263 (1886)[DOI]
  8. Gollwitzer, M., "Individual differences make a difference : On the use and the psychometric properties of difference scores in social psychology", European Journal of Social Psychology, 44(7), pp.673~682 (2014)[DOI]
  9. Gulliksen, H., "Theory of mental tests" (1950), John Wiley & Sons Inc
  10. Holland, P. W., "Statistics and Causal Inference", Journal of the American Statistical Association, 81(396), pp.945~960 (1986)[DOI]
  11. Holland, P. W., "Principals of modern psychological measurement", pp.3~25 (1983), Laurence Erlbaum Associates
  12. Huck, S. W., "Using a repeated measures ANOVA to analyze the data from a pretest-posttest design : A potentially confusing task", Psychological Bulletin, 82, pp.511~518 (1975)[DOI]
  13. Jamieson, J., "Correlates of reactivity : Problems with regression based methods", International Journal of Psychophysiology, 17(1), pp.73~78 (1994)[DOI]
  14. Jamieson, J., "Dealing with baseline differences : Two principles and two dilemmas", International Journal of Psychophysiology, 31(2), pp.155~161 (1999)[DOI]
  15. Jennings, M. A., "Comparing Pre-Post Change Across Groups : Guidelines for Choosing between Difference Scores, ANCOVA, and Residual Change Scores", Journal of Data Science, 14(2), pp.205~230 (2016)[DOI]
  16. Kenny, D. A., "A quasi-experimental approach to assessing treatment effects in the nonequivalent control group design", Psychological Bulletin, 82(3), pp.345~362 (1975)[DOI]
  17. Kim, H., "Propensity Score Analysis in Non-Randomized Experimental Designs : An Overview and a Tutorial Using R Software", New Directions for Child and Adolescent Development, 2019(167), pp.65~89 (2019)[DOI]
  18. Kim, S., "An Exploratory Study on the Effectiveness of Educational Donation Programs for Middle School Students", Korean Journal of Psychology : General, 38(3), pp.301~322 (2019)[DOI]
  19. Kisbu-Sakarya, Y., "A Monte Carlo Comparison Study of the Power of the Analysis of Covariance, Simple Difference, and Residual Change Scores in Testing Two-Wave Data", Educational and Psychological Measurement, 73(1), pp.47~62 (2013)[DOI]
  20. Köhler, C., "Deciding between the Covariance Analytical Approach and the Change-Score Approach in Two Wave Panel Data", Multivariate Behavioral Research, 56(3), pp.447~458 (2021)[DOI]
  21. Lee, M-H., "Development and Construct Validation of the Basic Psychological Needs Scale for Korean Adolescents : Based on the Self-Determination Theory", Korean Journal of Social and Personality Psychology, 22(4), pp.157~174 (2008)[DOI]
  22. Linn, R. L., "The Determination of the Significance of Change Between Pre-and Posttesting Periods", Review of Educational Research, 47(1), pp.121~150 (1977)[DOI]
  23. Lord, F. M., "The Measurement of Growth", Educational and Psychological Measurement, 16(4), pp.421~437 (1956)[DOI]
  24. Lord, F. M., "Problems in Measuring Change", pp.21~38 (1963), The University of Wisconsin Press
  25. Lord, F. M., "A paradox in the interpretation of group comparisons", Psychological Bulletin, 68, pp.304~305 (1967)[DOI]
  26. Lord, F. M., "Statistical theories of mental test scores" (1968), Addison-Wesley
  27. Maris, E., "Covariance adjustment versus gain scores—Revisited", Psychological Methods, 3, pp.309~327 (1998)[DOI]
  28. McArdle, J. J., "Latent variable modeling of differences and changes with longitudinal data", Annual Review of Psychology, 60, pp.577~605 (2009)[DOI]
  29. Miyazaki, Y., "Bias for Treatment Effect by Measurement Error in Pretest in ANCOVA Analysis", Educational and Psychological Measurement, 82(6), pp.1130~1152 (2022)[DOI]
  30. Nesselroade, J. R., "Regression toward the mean and the study of change", Psychological Bulletin, 88(3), pp.622~637 (1980)[DOI]
  31. Overall, J. E., "Unreliability of difference scores : A paradox for measurement of change", Psychological Bulletin, 82(1), pp.85~86 (1975)[DOI]
  32. Petscher, Y., "A Simulation Study on the Performance of the Simple Difference and Covariance-Adjusted Scores in Randomized Experimental Designs", Journal of Educational Measurement, 48(1), pp.31~43 (2011)[DOI]
  33. Rausch, J. R., "Analytic Methods for Questions Pertaining to a Randomized Pretest, Posttest, Follow-Up Design", Journal of Clinical Child and Adolescent Psychology, 32(3), pp.467~486 (2003)[DOI]
  34. Rogosa, D., "The Analysis of Change", pp.3~66 (1995), Lawrence Erlbaum Associates
  35. Rogosa, D., "A growth curve approach to the measurement of change", Psychological Bulletin, 92(3), pp.726~748 (1982)[DOI]
  36. Rogosa, D. R., "Demonstrating the Reliability of the Difference Score in the Measurement of Change", Journal of Educational Measurement, 20(4), pp.335~343 (1983)[DOI]
  37. Rogosa, D. R., "Understanding correlates of change by modeling individual differences in growth", Psychometrika, 50(2), pp.203~228 (1985)[DOI]
  38. Roh, Y., "The Psychological Impact of Perceived Overqualification in College Graduates : A Longitudinal Analysis", Korean Journal of Industrial and Organizational Psychology, 19(1), pp.59~84 (2006)
  39. Rosenbaum P. R., "The central role of the propensity score in observational studies for causal effects", Biometrika, 70(1), pp.41~55 (1983)[DOI]
  40. Shadish, W. R., "Experimental and quasi-experimental designs for generalized causal inference" (2002), Wadsworth Cengage Learning
  41. Thomas, D. R., "Difference Scores From the Point of View of Reliability and Repeated-Measures ANOVA : In Defense of Difference Scores for Data Analysis", Educational and Psychological Measurement, 72(1), pp.37~43 (2012)[DOI]
  42. Valente, M. J., "Comparing Models of Change to Estimate the Mediated Effect in the Pretest–Posttest Control Group Design", Structural Equation Modeling : A Multidisciplinary Journal, 24(3), pp.428~450 (2017)[DOI]
  43. Van Breukelen, G. J. P., "ANCOVA versus change from baseline had more power in randomized studies and more bias in nonrandomized studies", Journal of Clinical Epidemiology, 59(9), pp.920~925 (2006)[DOI]
  44. Van Breukelen, G. J. P., "ANCOVA Versus CHANGE From Baseline in Nonrandomized Studies : The Difference", Multivariate Behavioral Research, 48(6), pp.895~922 (2013)[DOI]
  45. Wainer, H., "Adjusting for differential base rates : Lord’s paradox again", Psychological Bulletin, 109(1), pp.147~151 (1991)[DOI]
  46. Werts, C. E., "A general linear model for studying growth", Psychological Bulletin, 73, pp.17~22 (1970)[DOI]
  47. West, S. G., "Propensity scores as a basis for equating groups : Basic principles and application in clinical treatment outcome research", Journal of Consulting and Clinical Psychology, 82(5), pp.906~919 (2014)[DOI]
  48. Wright, D. B., "Comparing groups in a before–after design : When t test and ANCOVA produce different results", British Journal of Educational Psychology, 76(3), pp.663~675 (2006)[DOI]
  49. Zimmerman, D. W., "Gain Scores in Research Can Be Highly Reliable", Journal of Educational Measurement, 19(2), pp.149~154 (1982)[DOI]
  50. Zimmerman, D. W., "Reliability of gain scores under realistic assumptions about properties of pre-test and post-test scores", British Journal of Mathematical and Statistical Psychology, 51(2), pp.343~351 (1998)[DOI]

관련 논문AI추천

사전-사후 통제집단 설계에서 차이점수 분석과 공분산분석의 수행 비교: 시뮬레이션 연구석혜원, 이영수한국심리학회 · 한국심리학회지:일반2025 · 제44권 제3호반복측정한 임상 데이터의 분석방법에 대한 선발의 연구송정대한임상검사과학회 · 대한임상검사과학회지2013 · 제45권 제2호혼합 설계에서 공분산분석의 주의점: KwakStat과 SPSS를 사용하여곽호완한국인지및생물심리학회 · 한국심리학회지: 인지 및 생물2011 · 제23권 제2호반복측정 자료를 다루는 교육 연구 실태 분석유진은한국열린교육학회 · 열린교육연구2014 · 제22권 제4호선형 혼합효과 모형에서 실험조건 효과 검정을 위한 절차 비교신유철, 이우열한국심리학회 · 한국심리학회지:일반2022 · 제41권 제5호다지표 잠재성장모형에서 요인동일성과 요인척도 설정의 문제: 모형 적합도와 모수 추정값에 미치는 영향박송이, 장승민한국심리학회 · 한국심리학회지:일반2018 · 제37권 제1호

이 간행물의 다른 논문 (3건)

자립준비청년 상담 효과에 관한 혼합연구최해연, 백 지성pp. 173-197구조방정식 모형의 전반적인 평가 및 효과크기와 연속성에 대한 숙고유 소현, 김수영pp. 199-230번안 심리검사 타당화 작업에 대한 체계적 검토: 검수와 보고 관행에 대한 검토와 제언임예지, 김미림pp. 261-295
원문 보기