Inappropriate Survey Design Analysis of the Korean National Health and Nutrition Examination Survey May Produce Biased Results
1
조회
0
다운로드
17
인용
0
참고문헌
키워드
초록
Objectives: The inherent nature of the Korean National Health and Nutrition Examination Survey (KNHANES) design requires special analysis by incorporating sample weights, stratification, and clustering not used in ordinary statistical procedures. Methods: This study investigated the proportion of research papers that have used an appropriate statistical methodology out of the research papers analyzing the KNHANES cited in the PubMed online system from 2007 to 2012. We also compared differences in mean and regression estimates between the ordinary statistical data analyses without sampling weight and design-based data analyses using the KNHANES 2008 to 2010. Results: Of the 247 research articles cited in PubMed, only 19.8% of all articles used survey design analysis, compared with 80.2% of articles that used ordinary statistical analysis, treating KNHANES data as if it were collected using a simple random sampling method. Means and standard errors differed between the ordinary statistical data analyses and design-based analyses, and the standard errors in the design-based analyses tended to be larger than those in the ordinary statistical data analyses. Conclusions: Ignoring complex survey design can result in biased estimates and overstated significance levels. Sample weights, stratification, and clustering of the design must be incorporated into analyses to ensure the development of appropriate estimates and standard errors of these estimates.
Abstract
Objectives: The inherent nature of the Korean National Health and Nutrition Examination Survey (KNHANES) design requires special analysis by incorporating sample weights, stratification, and clustering not used in ordinary statistical procedures. Methods: This study investigated the proportion of research papers that have used an appropriate statistical methodology out of the research papers analyzing the KNHANES cited in the PubMed online system from 2007 to 2012. We also compared differences in mean and regression estimates between the ordinary statistical data analyses without sampling weight and design-based data analyses using the KNHANES 2008 to 2010. Results: Of the 247 research articles cited in PubMed, only 19.8% of all articles used survey design analysis, compared with 80.2% of articles that used ordinary statistical analysis, treating KNHANES data as if it were collected using a simple random sampling method. Means and standard errors differed between the ordinary statistical data analyses and design-based analyses, and the standard errors in the design-based analyses tended to be larger than those in the ordinary statistical data analyses. Conclusions: Ignoring complex survey design can result in biased estimates and overstated significance levels. Sample weights, stratification, and clustering of the design must be incorporated into analyses to ensure the development of appropriate estimates and standard errors of these estimates.
참고문헌 (32)
- Second workshop for the data analysis of the Korea National Health & Nutrition Examination Survey and online survey of adolescent health behavior, "Korea Centers for Disease Control and Prevention", Korea Centers for Disease Control and Prevention, 1~60 (2012)
- Jacobson AF, "Obtaining and evaluating data sets for secondary analysis in nursing research", West J Nurs Res, 15(4), pp.483~494 (1993)
- Korn EL, "Analysis of large health surveys: accounting for the sampling design", J R Stat Soc Ser A Stat Soc, 158(2), pp.263~295 (1995)
- Magee T, "Generating new knowledge from existing data: the use of large data sets for nursing research", Nurs Res, 55(2-1), pp.50~S56 (2006)
- Levy PS, "Sampling of populations: methods and applications", pp.333~363 (1999), Wiley-Interscience Publication
- Lee ES, "Analyzing complex survey data", pp.11~20 (2006), SAGA Publications
- A comparison of designbased and model based analysis of sample survey in geography, "Wheeler DC", Rollings School of Public Health, Emory University, 1~34 (2007)
- Lohr SL, "Sampling: design and analysis", pp.2010, Brooks
- Korn EL, "Analysis of health surveys", pp.8~50 (1999), Wiley-Interscience Publication
- The National Health and Nutrition Examination Survey (NHANES): analytic and reporting guidelines, "National Center for Health Statistics", Centers for Disease Control and Prevention, 1~14 (2012)
- Guideline for the evaluation of the fourth Korea National Health and Nutrition Examination Survey, "Korea Centers for Disease Control and Prevention", Korea Centers for Disease Control Prevention, 19~34 (2009)
- Lee BK, "Iron deficiency is associated with increased levels of blood cadmium in the Korean general population: analysis of 2008-2009 Korean National Health and Nutrition Examination Survey data", Environ Res, 112, pp.155~163 (2012)
- 박선민, "Vitamin D deficiency is an independent risk factor for cardiovascular disease in Koreans aged ≥ 50 years: results from the Korean National Health and Nutrition Examination Survey", Nutrition Research and Practice, 6(2), pp.162~168 (2012), 한국영양학회
- 홍상모, "Characteristics of Body Fat, Body Fat Percentage and Other Body Composition for Koreans from KNHANES IV", Journal of Korean Medical Science, 26(12), pp.1599~1605 (2011), 대한의학회
- Hahs-Vaughn DL, "A primer for using and understanding weights with national datasets", J Exp Educ, 73(3), pp.221~248 (2005)
- National Research Council, "Methods for analysis of complex surveys, In Understanding American agriculture: challenges for the agricultural resource management survey", pp.127~148 (2007), National Academies Press
- The impact of complex survey design on prevalence estimates of intakes of food groups in the Australian National Children’s Nutrition and Physical Activity Survey, "Burden S", Centre for Statistical and Survey Methodology, University of Wollongong, 1~2 (2010)
- Kalton G, "Models in the practice of survey sampling (revisited)", J Off Stat, 18(2), pp.129~154 (2002)
- Rodgers-Farmer AY, "Analyzing complex survey data", Soc Work Res, 25(3), pp.185~192 (2001)
- Brogan D, "Software for sample survey data, misuse of standard packages, In Encyclopedia of biostatistics", pp.5057~5064 (2005), Estados Unidos: John Wiley & Sons
- West BT, "Statistical and methodological issues in the analysis of complex sample survey data: practical guidance for trauma researchers", J Trauma Stress, 21(5), pp.440~447 (2008)
- Lemeshow S, "Illustration of analysis taking into account complex survey considerations: the association between wine consumption and dementia in the PAQUID study", Am J Epidemiol, 148(3), pp.298~306 (1998)
- Korn EL, "Examples of differing weighted and unweighted estimates from a sample survey", Am Stat, 49(3), pp.291~295 (1995)
- Cho GJ, "The relationship between blood mercury level and osteoporosis in postmenopausal women", Menopause, 19(5), pp.576~581 (2012)
- Heeringa SG, "Applied survey data analysis", pp.13~50 (2005), CRC Press
- Park I, "Design effects for the weighted mean and total estimators under complex survey sampling", Surv Methodol, 30(2), pp.183~193 (2004)
- "Specifying weighting parameters" (2011)[URL]
- "National Health and Nutrition Examination Survey, 2007-2008" (2012)[URL]
- Lee BK, "Proper sample-weighted data analysis is required to confirm the association between blood cadmium concentration and lipid profile in Korean adults: re: Kim K. Blood cadmium concentration and lipid profile in Korean adults", Environ. Res (2011)
- Lee BK, "Without the application of appropriate statistical methods, it is difficult to reach the proper conclusion", Menopause, 19(6), pp.710~ (2012)
- Platt RW, "Survey data with sampling weights: is there a “best” approach?", Environ Res, 120, pp.143~144 (2013)
- Hahs-Vaughn DL, "Methodological considerations in using complex survey data: an applied example with the Head Start Family and Child Experiences Survey", Eval Rev, 35(3), pp.269~303 (2011)