Secondary Benefits of Manipulation Checks: Three Illustrations From Behavioral Public Administration
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Manipulation checks in behavioral public administration are commonly used and reported to determine if the experimental and control group have received different treatments. This paper uses three experiments to argue that manipulation checks for experimental treatments can have secondary benefits that can be used to improve the quality of behavioral work in the field. The three cases address the importance of using more clear terms in experimental manipulations (government v. public), using different on-line platforms to recruit experimental subjects (Mechanical Turk, Prolific, and Data.Spring), and whether larger payments more produce more attentive subjects.
Abstract
Manipulation checks in behavioral public administration are commonly used and reported to determine if the experimental and control group have received different treatments. This paper uses three experiments to argue that manipulation checks for experimental treatments can have secondary benefits that can be used to improve the quality of behavioral work in the field. The three cases address the importance of using more clear terms in experimental manipulations (government v. public), using different on-line platforms to recruit experimental subjects (Mechanical Turk, Prolific, and Data.Spring), and whether larger payments more produce more attentive subjects.
참고문헌 (41)
- Anna A. Amirkhanyan, "Liberté, Égalité, Crédibilité: An experimental study of citizens' perceptions of government responses to <scp>COVID<", scp>‐19 in eight countries, 2023, pp.83(2), 401~418[DOI]
- An, S., "When the Plane Crashes: Contracting and Blame Avoidance for Failures", ~ (2021)
- Angrist, J. D., "Identification and estimation of local average treatment effects" (1995), National Bureau of Economic Research
- Bachrach, D. G., "Rigor in behavioral experiments : A basic primer for supply chain management researchers", Journal of Supply Chain Management, 47(3), pp.5~8 (2011)
- Bellé, N., "Experimental evidence on the relationship between public service motivation and job performance", Public Administration Review, 73(1), pp.143~153 (2013)
- Avner Ben-Ner, "Does Ownership Matter in the Selection of Service Providers? Evidence From Nursing Home Consumer Surveys", Nonprofit and Voluntary Sector Quarterly, 47(6), pp.1271~1295 (2018), SAGE Publications[DOI]
- Adam J. Berinsky, "Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk", Political Analysis, 20(3), pp.351~368 (2012), Cambridge University Press (CUP)[DOI]
- Buhrmester, M., "Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality data?", Perspectives on Psychological Science, 6(1), pp.3~5 (2011)[DOI]
- 제시 캠벨, "Public Participation and Trust in Government: Results From a Vignette Experiment", Journal of Policy Studies, 38(2), pp.23~31 (2023), 서울대학교행정대학원[DOI]
- Krista Casler, "Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing", Computers in Human Behavior, 29(6), pp.2156~2160 (2013), Elsevier BV[DOI]
- Emma Ejelöv, "“Rarely safe to assume”: Evaluating the use and interpretation of manipulation checks in experimental social psychology", Journal of Experimental Social Psychology, 87, pp.103937~ (2020), Elsevier BV[DOI]
- Marie-Pierre Fayant, "On the Limitations of Manipulation Checks: An Obstacle Toward Cumulative Science", International Review of Social Psychology, 30(1), pp.125~ (2017), Ubiquity Press, Ltd.[DOI]
- Festinger, L., "Research Methods in the Behavioral Sciences", pp.136~172 (1953), Dryden Press
- Andriati Fitriningrum, "GOVERNMENT, STATE-OWNED ENTERPRISE, and LIBERALIZATION of UNIVERSAL POSTAL OBLIGATION SERVICES", Journal of Policy Studies, 38(3), pp.25~40 (2023), 서울대학교행정대학원[DOI]
- Flake, J. K., "Construct validation in social and personality research : Current practice and recommendations", Social Psychological and Personality Science, 8(4), pp.370~378 (2017)
- Kendall Funk, "If the shoe fits: Gender role congruity and evaluations of public managers", Journal of Behavioral Public Administration, 2(1) (2019), Center for Experimental and Behavioral Public Administration[DOI]
- Brian J. Gaines, "The Logic of the Survey Experiment Reexamined", Political Analysis, 15(1), pp.1~20 (2007), Cambridge University Press (CUP)[DOI]
- Anil Kumar Gupta, "Service Users’ Confidence in Accessing Public Services in Nepal: What Makes Differences?", Journal of Policy Studies, 38(1), pp.29~43 (2023), 서울대학교행정대학원[DOI]
- Hauser, D. J., "Are manipulation checks necessary?", Frontiers in Psychology, 9, pp.998~ (2018)[DOI]
- David J. Hauser, "Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants", Behavior Research Methods, 48(1), pp.400~407 (2015), Springer Science and Business Media LLC[DOI]
- Highhouse, S., "Designing experiments that generalize", Organizational Research Methods, 12(3), pp.554~566 (2009)
- Ulrik Hvidman, "Perceptions of Public and Private Performance: Evidence from a Survey Experiment", Public Administration Review, 76(1), pp.111~120 (2015), Wiley[DOI]
- James, O., "Behavioral public performance : How people make sense of government metrics" (2020), Cambridge University Press
- John, P., "How far to nudge?: assessing behavioural public policy" (2018), Edward Elgar Publishing
- Austin P. Johnson, "Can Hierarchy Dodge Bullets? Examining Blame Attribution in Military Contracting", Journal of Conflict Resolution, 63(8), pp.1965~1985 (2019), SAGE Publications[DOI]
- Joanna N. Lahey, "Computerizing audit studies", Journal of Economic Behavior & Organization, 70(3), pp.508~514 (2009), Elsevier BV[DOI]
- Marvel, J. D., "Unconscious bias in citizens’ evaluations of public sector performance", Journal of Public Administration Research and Theory, 26(1), pp.143~158 (2016)
- Kenneth J. Meier, "Sector bias in public programs: US nonprofit hospitals", Journal of Behavioral Public Administration, 3(1) (2020), Center for Experimental and Behavioral Public Administration[DOI]
- Meier, K. J., "Sector bias and the credibility of performance information : An experimental study of elder care provision", Public Administration Review, 82(1), pp.69~82 (2022)
- Ismael Mourifié, "Testing Local Average Treatment Effect Assumptions", The Review of Economics and Statistics, 99(2), pp.305~313 (2017), MIT Press - Journals[DOI]
- Mutz, D. C., "Population-based survey experiments" (2011), Princeton University Press
- Mutz, D. C., "Standards for experimental research : Encouraging a better understanding of experimental methods", Journal of Experimental Political Science, 2(2), pp.192~215 (2015)
- 오민지, "Loyal to the Public: Examining the Relationship Between Chief Executives and the Pursuit of Public Values in Quangos", Journal of Policy Studies, 38(1), pp.1~14 (2023), 서울대학교행정대학원[DOI]
- Byron D’Andra Orey, "Black and Blue: Black Police Officers’ Implicit and Explicit Biases in Split-Second Decisions to Shoot or Not to Shoot Unarmed Black Civilians", Journal of Policy Studies, 38(3), pp.1~13 (2023), 서울대학교행정대학원[DOI]
- Niels Bjørn G Petersen, "Whoever Has Will be Given More: The Effect of Performance Information on Frontline Employees’ Support for Managerial Policy Initiatives", Journal of Public Administration Research and Theory, 30(4), pp.533~547 (2020), Oxford University Press (OUP)[DOI]
- Jaclyn Piatak, "Bureaucratic accountability in third‐party governance: Experimental evidence of blame attribution during times of budgetary crisis", Public Administration, 95(4), pp.976~989 (2017), Wiley[DOI]
- Hal G. Rainey, "Comparing Public and Private Organizations", Public Administration Review, 36(2), pp.233~ (1976), JSTOR[DOI]
- Norma M. Riccucci, "Representative Bureaucracy in Policing: Does It Increase Perceived Legitimacy?", Journal of Public Administration Research and Theory, 24(3), pp.537~551 (2014), Oxford University Press (OUP)[DOI]
- Roethlisberger, F. J., "Management and the Worker" (1939), Harvard University Press
- Using Facebook as a subject recruitment tool for survey-experimental research, "Samuels, D. J.", ~ (2013)
- Simon, H. A., "Administrative behavior" (1947), The Free Press
