Within/Between or Latin Square: Experimental Designs of Acceptability Judgment Testing
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The present study addresses whether and how the design of presenting the pairwise sentences to the participants affects the result of acceptability judgment testing. Because a single choice in experimental design can occasionally lead to a dramatic change at the end, empirical science requires the latent impacts of the parameters in experimental settings to be examined in advance. The present study conducts three designs of acceptability judgment testing, such as the between-group, the within-group, and the Latin Square designs with the same test items. The experiments were administrated on a large scale with two different experimental tasks, such as the Likert scale task and the binary Yes/No task. The comparison across the designs and the tasks demonstrates that the different designs do not necessarily yield different results This indicates that there is no ground for us to regard the Latin Square design as an indispensable process in language experiments.
Abstract
The present study addresses whether and how the design of presenting the pairwise sentences to the participants affects the result of acceptability judgment testing. Because a single choice in experimental design can occasionally lead to a dramatic change at the end, empirical science requires the latent impacts of the parameters in experimental settings to be examined in advance. The present study conducts three designs of acceptability judgment testing, such as the between-group, the within-group, and the Latin Square designs with the same test items. The experiments were administrated on a large scale with two different experimental tasks, such as the Likert scale task and the binary Yes/No task. The comparison across the designs and the tasks demonstrates that the different designs do not necessarily yield different results This indicates that there is no ground for us to regard the Latin Square design as an indispensable process in language experiments.
참고문헌 (21)
- 안희돈, "한국어 수용성판단 자료의 학술지간 비교: ‘생성문법연구’와 ‘국어학’을 중심으로", 생성문법연구, 31(2), pp.343~356 (2021), 한국생성문법학회[DOI]
- Bard, E. G., "Magnitude Estimation of Linguistic Acceptability", Language, 72(1), pp.32~68 (1996)
- Benjamini, Y., "Controlling the False Discovery Rate: a Practical and Powerful Approach to Multiple Testing", Journal of the Royal statistical society: series B (Methodological), 57(1), pp.289~300 (1995)
- Cowart, W., "Experimental Syntax" (1997), Sage Publications
- Featherston, S, "Magnitude Estimation and What it can do for your Syntax: Some wh-constraints in German", Lingua, 115(11), pp.1525~1550 (2005)
- Fukuda, S., "Is Magnitude Estimation Worth the Trouble", Proceedings of the 29th West Coast Conference on formal linguistics (2012)
- Gibson, E., "Using Mechanical Turk to Obtain and Analyze English Acceptability Judgments", Language and Linguistics Compass, 5(8), pp.509~524 (2011)
- 강형진, "한국인 영어 학습자의 영어 생략문 태불일치문장 수용성 판단에 관한 연구", 언어과학, 23(4), pp.1~23 (2016), 한국언어과학회[DOI]
- Mathôt, S., "OpenSesame: An Open-Source, Graphical Experiment Builder for the Social Sciences", Behavior Research Methods, 44(2), pp.314~324 (2012)
- Moskowitz, H. R, "Magnitude Estimation: Notes on What, How, When, and Why to Use it", Journal of Food Quality, 1(3), pp.195~227 (1977)
- Myers, J, "The Design and Analysis of Small-scale Syntactic Judgment Experiments", Lingua, 119(3), pp.425~444 (2009)
- Myers, J., "Syntactic Judgment Experiments", Language and Linguistics Compass, 3(1), pp.406~423 (2009)
- 오현영, "한국인 영어 학습자의 통사적 섬에 대한 수용성 연구", 언어과학, 23(3), pp.157~179 (2016), 한국언어과학회[DOI]
- Petzschner, F. H., "A Bayesian Perspective on Magnitude Estimation", Trends in Cognitive Sciences, 19(5), pp.285~293 (2015)
- 송상헌, "5 or 7: Is the Choice So Important in Acceptability Judgment Testing?", 언어, 41(3), pp.449~480 (2016), 한국언어학회[DOI]
- 송상헌, "What do you mean by contrast in syntax?", 언어연구, 34(3), pp.387~426 (2017), 언어정보연구소[DOI]
- Sorace, A, "Using Magnitude Estimation in Developmental Linguistic Research", Experimental Methods in Language Acquisition Research, 27, pp.57~72 (2010)
- Sprouse, J, "A Test of the Cognitive Assumptions of Magnitude Estimation: Commutativity does not Hold for Acceptability Judgments", Language, 87(2), pp.274~288 (2011)
- Sprouse, J., "A Comparison of Informal and Formal Acceptability Judgments Using a Random Sample from Linguistic Inquiry 2001-2010", Lingua, 134, pp.219~248 (2013)
- Sprouse, J., "Experimental syntax and the variation of island effects in English and Italian", Natural Language and Linguistic Theory, 34, pp.307~344 (2016)
- Wescott, T., "On the Informativity of Different Measures of Linguistic Acceptability", Language, 87(2), pp.249~273 (2011)