실험자료 분석 시 무선효과 구조 선택을 위한 우도비 검정에서의 영가설 분포의 모양

Abstract

When using a multilevel model to analyze experimental data, the results of hypothesis testing can vary depending on the random effects structure. Previous studies have shown that the likelihood ratio test (LRT) for selecting the random effect structure does not match the shape of the null distribution with the theoretical chi-squared distribution due to the parameter space constraints of the variance. In this study, we investigate the shape, the type 1 error rate, and convergence failure rate of null distribution of LRT to select random effects structures through Monte Carlo simulations. Before proceeding with this study, the validity of the methodology was secured by checking the shape of null distribution of LRT for testing fixed effects. We conducted simulations by manipulating the experimental design, number of participants, and number of items, and found that the mean, variance, 95th and 99th percentiles of null distribution of LRT for selecting random effects structures were smaller than the theoretical chi-squared distribution under all conditions. When LRT was performed using the theoretical chi-squared distribution, the type 1 error rate was smaller than .05 significance level. The convergence failure occurred frequently in models with complex experimental designs and correlation structure in the structure of the random effects. The caveats of choosing the model’s random effects structure were discussed for empirical researchers when using multilevel models to test for experimental condition effects.