대규모 교육자료에서 표본 통계치의 표준오차 산출 비교:BRR 방법과 Fay 조정계수
An Evaluation of Standard Errors for Large-scale Educational Data: The BRR Method with Fay’s Adjustment Factors
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
이 연구의 목적은 대규모 교육자료 분석에서 BRR방법(Balanced Repeated Replication method)의 Fay 조정계수에 따른 표본 통계치의 표준오차 추정 결과를 비교하는 것에 있다. 구체적으로 0과 1사이 값으로 설정된 Fay의 조정계수가 표본의 크기에 따라, 선형식으로 정의되는 평균(척도점수)과 비선형식으로 표현되는 중앙값의 표준오차에 미치는 영향을 검증하였다. 두 가지 대규모 교육자료 분석결과(PISA 2012와 2011 국가수준 학업성취도평가), 표본의 표집설계를 반영하지 않고 임의표집을 가정한 표준오차는 과소추정되며, 이는 Type I 오류의 가능성을 높인다. Fay 조정계수는 평균, 비율과 같은 선형통계치의 표준오차 추정에 큰 영향을 미치지 않으며, 이에 대한 BRR방법의 결과는 Jackknife방법과 유사하게 나타난다. 그러나 중앙값으로 대표되는 비선형 통계치에 대한 조정계수의 영향은 매우 민감하게 나타난다. 이 연구에서 논의된 Fay의 BRR방법은 표집절차를 적절히 반영한다는 측면에서 우리나라 대규모 교육자료 분석에 적용될 때, 교육적 의사결정을 위한 통계적 불확실성(표준오차)이 보다 과학적으로 검증될 수 있을 것이다.
Abstract
Purpose: This study sought to evaluate the effects of Fay’s adjustment factors in BRR (balanced repeated replication) method on standard errors of sample statistics, which were estimated from the large-scale educational data. Design/methodology/data/approach: Two large-scale educational data sets, PISA 2012 Korea data set and National Assessment of Educational Achievement 2011 data set (Elementary school), were analyzed. The standard errors (or sampling variances) of linear statistics (e.g., mean and ratio) and nonlinear statistics (e.g., median) were compared according to various Fay’s adjustment factors, such as 0.0, 0.2, 0.5, 0.7 and 0.9. Finding: Standard errors ignoring sampling designs (i.e., assuming a random sampling design) were found to have been underestimated and then Type I error rates increased as well. Fay’s adjustment factors had little impact on the standard errors of linear statistics and the BRR method produced standard errors similar to the Jackknife method. The standard errors of non-linear statistics, however, varied considerably according to Fay’s adjustment factors especially from small samples and skewed distributions. Values: The Fay’s BRR method, discussed in this paper, were useful to analyze large-scale education data with complex sampling designs since it reflected sampling designs accordingly and then generated sampling errors properly for parameter inferences.
참고문헌 (32)
- 국가수준의 기초학력 점검을 위한 초등학교 학업성취도 평가 방안, "김경희", 한국교육과정평가원, ~ (2014)
- 김경희, "2011 국가수준 학업성취도 평가 결과 : 초등학교 학업성취도 변화 추이 분석" (2012), 한국교육과정평가원
- 김경희, "2012년 국가수준 학업성취도 평가 결과 : 학교교육의 성과와 변화" (2013), 한국교육과정평가원
- 민경석, "국가수준 학업성취도 평가의 초등학교 표집방법 비교", 교육과정평가연구, 17(2), pp.49~70 (2014), 한국교육과정평가원
- 조지민, "PISA 표집 설계에 따른 모수 및 분산 추정", 교육평가연구, 26(4), pp.875~896 (2013), 한국교육평가학회
- Ardilly, P., "Sampling methods : Exercises and solutions" (2006), Springer
- An empirical comparison of BRR and linearization variance estimators, "Boonstra, H. J.", Statistics Netherlands, ~ (2003)
- Carsey, M. T., "Monte carlo simulation and resampling methods for social science" (2014), SAGE
- Chen, H., "Comparison of variance estimation methods using PPI data. Proceedings of the Section on Government Statistics", Journal of Statistical Methods (2010)
- A comparison of tayler linearization and balanced repeated replication methods for variance estimation in Medical Expenditure Panel Survey, "Chowdhury, S. R.", Agency for Healthcare Research and Quality, ~, pp.13004 (2013)
- Cochran, W. G., "Sampling technique" (1977), John Wiley & Sons
- Fay, R. E., "Some properties of estimate of variance based on replication methods", Proceedings of the Social Statistics Session, American Statistical Association (1984)
- Fay, R. E., "Theory and application of replicate weighting for variance calculations. Proceedings of the Social Statistics Session", American Statistical Association (1989)
- Coming of age in the 1990s: The eighth-grade class of 1988 12 years later(NCES 2002-321), "Ingels, S. J.", U.S. Department of Education, National Center for Education Statistics, ~ (2002)
- TIMSS 2007 sample design, "Joncas, M.", TIMSS, ~ (2008)
- TIMSS 2007 sample weights and participation rates, "Joncas, M.", TIMSS, ~ (2008)
- Judkins, D. R., "Fay’s method of variance estimation", Journal of Official Statistics, 6(3), pp.223~239 (1990)
- Kish, L., "Statistical design for research" (1987), John Wiley & Sons
- Kish, L., "Balanced repeated replications for standard errors", Journal of the American Statistical Association, 65(331), pp.1071~1094 (1970)
- Linn, R. L., "Accountability : Responsibility and reasonable expectations", Educational Researcher, 32(7), pp.3~13 (2003)
- Linn, R. L., "Accountability systems : Implications of requirements of the No Child Left Behind Act of 2001", Educational Researcher, 31(6), pp.3~16 (2002)
- Lohr, S. L., "Sampling : Design and analysis", pp.2010, Brooks
- PISA 2009 technical report, "OECD", OECD, ~ (2012)
- OECD, "PISA 2012 results : What students know and can do : Student performance in mathematics, reading and science(Volume I)" (2014), OECD
- Paben, S. P., "Comparison of variance estimation methods for the National Compensation Survey. Proceedings of the section on Survey Research Methods", annual meeting of American Statistical Association (1999)
- Parsons, V. L., "Application of a Fay-modified balanced repeated replication method to the National Health Interview Survey (NHIS)", Proceedings of the Section on Survey Research Methods, presented at 2010 Joint Statistical Meeting in Vancouver (2010)
- Rao, J. N. K., "Modified balanced repeated replication for complex survey data", Biometrika, 86(2), pp.403~415 (1999)
- Rutkowski, L., "International large-scale assessment data : Issues in secondary analysis and reporting", Educational Researcher, 39(2), pp.142~151 (2010)
- Shao, J., "The jackknife and bootstrap" (1995), Springer
- Slavin, R. E., "Evidence-based education policies : Transforming educational practice and research", Educational Researcher, 31(7), pp.15~21 (2002)
- Slavin, R. E., "What works? Issues in synthesizing educational program evaluations", Educational Researcher, 37(1), pp.5~14 (2008)
- Wolter, K. M., "Introduction to variance estimation" (2007), Springer