Overreliance on Statistical Testing Logic in the Empirical Testing of Theories and Hypotheses
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This article explicates some conceptual and methodological problems involved in the tendency of overreliance on statistical testing logic, logic of disproof. Although statistical hypothesis testing is only a subset of the whole process of empirical testing, a number of researchers tend to misconceive that statistical testing logic is universally applicable to the whole process of empirical theory testing. The authors show that this overreliance on logic of disproof leads to many problems: (1) conceptual problems such as confusion between the research hypothesis and the statistical hypothesis, and oversight of the necessity of logic of proof for accepting the “substantive” null hypothesis containing the researcher’s assertion per se, and (2) methodological problems such as a non-rigorous way of conducting empirical research (i.e., the breakdown of the boundary and the proper sequence between the deductive route and the inductive route in empirical theory testing), an incorrect interpretation of test results associated with the testing of the “substantive” null hypothesis, and confusion between logic of disproof (for testing the null hypothesis) and falsificationism (for testing theory). To remove such conceptual and methodological points of confusion caused by overreliance on logic of disproof, the authors have proposed a seven-step model of empirical theory testing: Step 1 (Theory) → Step 2 (Setting up the research hypothesis: RHEF [i.e., the research hypothesis in existential form]/RHNF [i.e., the research hypothesis in non-existential form]) → Step 3 (Setting up the statistical hypothesis: Translating RHEF or RHNF into H0 and H1) → Step 4 (Testing the statistical hypothesis by logic of disproof/proof) → Step 5 (Testing the research hypothesis: RHEF or RHNF is supported/not supported) → Step 6 (Testing theory empirically: The theory is empirically supported/not supported) → Step 7 (Interpretation of this empirical test result from the researcher’s scientific standpoint, such as falsificationism or logical empiricism) (for better understanding, see Figure 1 in the conclusion section). If the research hypothesis is inductively derived from observations rather than theory, then Step 1, Step 6, and Step 7 become irrelevant. With regard to the application of the seven step model, the authors have noted the following four points. Firstly, researchers should not intend to apply this seven step model to cases where statistical hypothesis testing is not valid (e.g., “H0: The Sun revolves around the Earth” and “H1: The Earth revolves around the Sun” [Johnson, 1999]; “H0: The defendant is innocent” and “Ha: The defendant is guilty” [Anderson et al., 1999]). Secondly, a theory (T), the research hypothesis (RH) derived from the theory, and the statistical hypothesis (H0 and H1) translated from the research hypothesis should be distinguished from each other clearly. Thirdly, even in cases of empirically testable theories or models, two competing theories or models should not be regarded as being the direct objects of statistical testing such as “H0: model A vs. H1: model B” (e.g., “H0: the hypothesized or simpler model” and “H1: the more general model” [e.g., Arora 1982]; “H0: the random-walk model” and “H1: the Markov switching model” (and vice versa) [e.g., Cheung and Erlandsson, 2005]). To test two competing theories or models, they usually have to use two separate seven-step or six-step models. Finally, the former Steps 1-3 correspond to the deductive route ruled by theory and the later Steps 4-6 stand for the inductive route ruled by empirical sample data. Thus, the boundary and the proper sequence between the deductive route and the inductive route must be observed in that order. If researchers observe these four points faithfully, unnecessary confusion can be avoided. The authors believe that the seven step model of empirical theory testing can play a role in accelerating the sound development of scientific knowledge with regard to the empirical testing of theories and hypotheses.
Abstract
This article explicates some conceptual and methodological problems involved in the tendency of overreliance on statistical testing logic, logic of disproof. Although statistical hypothesis testing is only a subset of the whole process of empirical testing, a number of researchers tend to misconceive that statistical testing logic is universally applicable to the whole process of empirical theory testing. The authors show that this overreliance on logic of disproof leads to many problems: (1) conceptual problems such as confusion between the research hypothesis and the statistical hypothesis, and oversight of the necessity of logic of proof for accepting the “substantive” null hypothesis containing the researcher’s assertion per se, and (2) methodological problems such as a non-rigorous way of conducting empirical research (i.e., the breakdown of the boundary and the proper sequence between the deductive route and the inductive route in empirical theory testing), an incorrect interpretation of test results associated with the testing of the “substantive” null hypothesis, and confusion between logic of disproof (for testing the null hypothesis) and falsificationism (for testing theory). To remove such conceptual and methodological points of confusion caused by overreliance on logic of disproof, the authors have proposed a seven-step model of empirical theory testing: Step 1 (Theory) → Step 2 (Setting up the research hypothesis: RHEF [i.e., the research hypothesis in existential form]/RHNF [i.e., the research hypothesis in non-existential form]) → Step 3 (Setting up the statistical hypothesis: Translating RHEF or RHNF into H0 and H1) → Step 4 (Testing the statistical hypothesis by logic of disproof/proof) → Step 5 (Testing the research hypothesis: RHEF or RHNF is supported/not supported) → Step 6 (Testing theory empirically: The theory is empirically supported/not supported) → Step 7 (Interpretation of this empirical test result from the researcher’s scientific standpoint, such as falsificationism or logical empiricism) (for better understanding, see Figure 1 in the conclusion section). If the research hypothesis is inductively derived from observations rather than theory, then Step 1, Step 6, and Step 7 become irrelevant. With regard to the application of the seven step model, the authors have noted the following four points. Firstly, researchers should not intend to apply this seven step model to cases where statistical hypothesis testing is not valid (e.g., “H0: The Sun revolves around the Earth” and “H1: The Earth revolves around the Sun” [Johnson, 1999]; “H0: The defendant is innocent” and “Ha: The defendant is guilty” [Anderson et al., 1999]). Secondly, a theory (T), the research hypothesis (RH) derived from the theory, and the statistical hypothesis (H0 and H1) translated from the research hypothesis should be distinguished from each other clearly. Thirdly, even in cases of empirically testable theories or models, two competing theories or models should not be regarded as being the direct objects of statistical testing such as “H0: model A vs. H1: model B” (e.g., “H0: the hypothesized or simpler model” and “H1: the more general model” [e.g., Arora 1982]; “H0: the random-walk model” and “H1: the Markov switching model” (and vice versa) [e.g., Cheung and Erlandsson, 2005]). To test two competing theories or models, they usually have to use two separate seven-step or six-step models. Finally, the former Steps 1-3 correspond to the deductive route ruled by theory and the later Steps 4-6 stand for the inductive route ruled by empirical sample data. Thus, the boundary and the proper sequence between the deductive route and the inductive route must be observed in that order. If researchers observe these four points faithfully, unnecessary confusion can be avoided. The authors believe that the seven step model of empirical theory testing can play a role in accelerating the sound development of scientific knowledge with regard to the empirical testing of theories and hypotheses.
참고문헌 (46)
- Anderson, D.R., "Statistics for Business and Economics" (1999), South-Western
- Arora, R., "Validation of an S-O-R model for situation, enduring, and response components of involvement", Journal of Marketing Research, 19(Nov), pp.505~516 (1982)
- Babbie, E., "The Practice of Social Research" (1998), Wadsworth Publishing
- Bagozzi, R.P., "Performance and satisfaction in an industrial sales force: An examination of their antecedents and simultaneity", Journal of Marketing, 44(Spring), pp.65~77 (1980)
- Bagozzi, R.P., "Development and test of a theory of technological learning and usage", Human Relations, 45(7), pp.659~686 (1992)
- Bailar III, J.C., "Guidelines for statistical reporting in articles for medical journals", Annals of Internal Medicine, 108(Feb), pp.266~273 (1988)
- Bhattacharyya, G.K., "Statistical Concepts and Methods" (1977), John Wiley & Sons
- Carnap, R., "Testability and meaning", Philosophy of Science, 4, pp.1~40 (1953)
- Cashen, L.H., "Statistical power and the testing of null hypotheses: A review of contemporary management research and recommendations for future studies", Organizational Research Methods, 7(2), pp.151~167 (2004)
- Cheung, Y.W., "Exchange rates and markov switching dynamics", Economic Statistics, 23(3), pp.314~320 (2005)
- Cohen, J., "The earth is round (p<.05)", American Psychologist, 49, pp.997~1003 (1994)
- Fisher, R.A., "The Design of Experiments" (1951), Hafner
- Frey, LR., "Investigating Communication: An Introduction to Research Methods" (2000), Allyn & Baco
- Frick, R.W., "Accepting the null hypothesis", Memory & Cognition, 23(1), pp.132~138 (1995)
- Harcum, E.R., "Guidance from the literature for accepting a null hypothesis when its truth is expected", Journal of General Psychology, 117(3), pp.325~344 (1990)
- Harnett, D.L., "Statistical Methods for Business and Economics" (1991), Addison- Wesley
- Hays, W.L., "Statistics" (1981), Holt, Rinehart and Winston
- Hunt, S.D., "Modern Marketing Theory: Critical Issues in the Philosophy of Marketing Science" (1991), South-Western Publishing
- Johnson, D.H., "The insignificance of statistical significance testing", Journal of Wildlife Management, 63(3), pp.763~772 (1999)
- Kanagaretnam, K., "Trust and reciprocity with transparency and repeated interactions", Journal of Business Research, 63, pp.241~247 (2010)
- Keiding, N., "The precautionary principle and statistical approaches to uncertainty", Human and Ecological Risk Assessment, 11(1), pp.201~207 (2005)
- Kerlinger, F.N., "Foundations of Behavioral Research" (2000), Harcourt College Publishers
- Jihyun Kim, "The Mediating Effect of Perceived Service Risk on Perceived Value of Internet Apparel Shopping: From the Quality-Risk-Value Approach", Journal of Global Scholars of Marketing Science(마케팅과학연구), 20(4), pp.306~317 (2010), 한국마케팅과학회
- Kirmani, A., "The ownership effect in consumer responses to brand line stretches", Journal of Marketing, 63(Jan), pp.88~101 (1999)
- Lind, D.A, "Statistical Techniques in Business & Economics" (2002), McGraw-Hill
- Mansfield, E., "Statistics for Business and Economics: Methods and Applications" (1983), W.W. Norton & Company
- Maritz, A., "Implementation of service profit Chain initiatives in a franchise system", Journal of Services Marketing, 22(1), pp.13~23 (2008)
- Misinterpreting a failure to disconfirm as a confirmation: A recurrent misreading of significance tests, in Working Paper, "Mayer, T.", ~ (2001)
- Ebrahim Mazaheri, "Investigating the Moderating Impact of Hedonism on Online Consumer Behavior", Journal of Global Scholars of Marketing Science(마케팅과학연구), 20(2), pp.123~134 (2010), 한국마케팅과학회
- Meehl, P.E., "Theory testing in psychology and in physics: A methodological paradox", Philosophy of Science, 34, pp.103~115 (1967)
- Miller, H.T., "The epistemic community", Administration & Society, 32(6), pp.668~685 (2001)
- Mishra, D.P., "Information asymmetry and levels of agency relationships", Journal of Marketing Research, 35(3), pp.277~295 (1998)
- Nickerson, R.S., "Null hypothesis significance testing: A review of an old and continuing controversy", Psychological Methods, 5(2), pp.241~301 (2000)
- Jeonpyo Noh, "The Effect of Forgiveness and Trust on Relational Performance in the Purchasing of Business Services", Journal of Global Scholars of Marketing Science(마케팅과학연구), 20(4), pp.293~305 (2010), 한국마케팅과학회
- Popper, K.R., "The logic of scientific discovery" (1959), Harper and Row
- Roehm, M.L., "When will a brand scandal spill over, and how should competitors respond", Journal of Marketing Research, 43(Aug), pp.366~373 (2006)
- Seiders, K., "Do satisfied customers buy more? Examining moderating influences in a retailing context", Journal of Marketing, 69(Oct), pp.26~43 (2005)
- Spreng, R.A., "A reexamination of the determinants of consumer satisfaction", Journal of Marketing, 60(July), pp.15~32 (1996)
- Sprowls, R., "Statistical Inference, in Handbook of marketing research", pp.2-278~2-294 (1974), McGraw- Hill
- Steiger, J.H., "Noncentrality Interval Estimation and the Evaluation of Statistical Models, in What If There Were No Significance Tests? Hillsdale", pp.221~257 (1997), Lawrence Erlbaum
- Swanstrom, M., "Corporate governance and the abnormal returns and acquisition announcements", Journal of Business Strategies, 23(2), pp.115~129 (2006)
- Swedlow, J.R., "Informatics and quantitative analysis in biological imaging", Science, 300(5616), pp.100~102 (2003)
- Tebes, J.K., "Community science, philosophy of science, and the practice of research", American Journal of Community Psychology, 35(3), pp.213~230 (2005)
- Trafimow, D., "Using epistemic ratios to evaluate hypotheses: An imprecision penalty for imprecise hypotheses", Genetic, Social, and General Psychology Monographs, 132(4), pp.431~462 (2006)
- Wonnacott, T.H., "Introductory Statistics for Business and Economics" (1984), John Wiley & Sons
- Wolverton, M.L., "Research design, hypothesis testing, and sampling", Appraisal Journal, pp.370~382 (2009)