Different Ways to Test Acceptability Judgments
1
조회
0
다운로드
5
인용
0
참고문헌
키워드
초록
The testing of acceptability judgments has recently attracted some attention in the study of syntax. This is because many linguists have come to realize that the extensive measuring the correlations between native speakers' intuitions provides a reliable way to investigate the genuine nature of human language. The growing research interest notwithstanding, the methodology of acceptability judgment experiments has not been completely validated. Since experimental syntax is an empirical science, it is important to develop a strong research methodology for a large-scale and in-depth quantitative analysis. In this context, the research question this paper raises is whether or not different methodologies of acceptability judgment testing yield different conclusions. The present study conducts two experiments that share exactly the same test items but are controlled and analyzed in different ways. Looking into the convergence rate using different methods of experimental control and statistical analysis, the present study provides four findings which can lead to a better methodology for observing native speakers’ intuitions: First, randomization is a mandatory process in acceptability judgment testing. Second, the mean in itself does not provide good statistical power in acceptability judgment testing. Third, sociological variables can affect acceptability judgments. Fourth, the size of sample in acceptability judgment testing should be more than 40 in order to yield a trustworthy conclusion.
Abstract
The testing of acceptability judgments has recently attracted some attention in the study of syntax. This is because many linguists have come to realize that the extensive measuring the correlations between native speakers' intuitions provides a reliable way to investigate the genuine nature of human language. The growing research interest notwithstanding, the methodology of acceptability judgment experiments has not been completely validated. Since experimental syntax is an empirical science, it is important to develop a strong research methodology for a large-scale and in-depth quantitative analysis. In this context, the research question this paper raises is whether or not different methodologies of acceptability judgment testing yield different conclusions. The present study conducts two experiments that share exactly the same test items but are controlled and analyzed in different ways. Looking into the convergence rate using different methods of experimental control and statistical analysis, the present study provides four findings which can lead to a better methodology for observing native speakers’ intuitions: First, randomization is a mandatory process in acceptability judgment testing. Second, the mean in itself does not provide good statistical power in acceptability judgment testing. Third, sociological variables can affect acceptability judgments. Fourth, the size of sample in acceptability judgment testing should be more than 40 in order to yield a trustworthy conclusion.
참고문헌 (34)
- Adger, David, "Core syntax: a minimalist approach" (2003), Oxford University Press
- Cho, Euiyon, "Theoretical commitments and the use of the asterisk ‘*’ in functional linguistics", Discourse and Cognition, 5(2), pp.285~292 (1998)
- 조수근, "한국어 내핵관계절의 존재 유무에 대한 연구", 언어정보, (19), pp.189~205 (2014), 언어정보연구소
- Clariana, Roy, "Paper–based versus computer–based assessment: key factors associated with the test mode effect", British Journal of Educational Technology, 33(5), pp.593~602 (2002)
- Eckert, Penelope, "The whole woman: Sex and gender differences in variation", Language variation and change, 1(3), pp.245~267 (1989)
- Eckert, Penelope, "The Handbook of Sociolinguistics" (1997), Blackwell Publisher Ltd
- Forster, Kenneth I., "DMDX: a Window display program with millisecond accuracy. Behavior Research Methods", Instruments and Computers, 35, pp.116~124 (2003)
- Gibson, Edward, "The need for quantitative methods in syntax and semantics research", Language and Cognitive Processes, 28, pp.88~124 (2013)
- Greenbaum, Sidney, "Contextual influence on acceptability judgements", Linguistics, 15(187), pp.5~12 (1977)
- Gries, Stefan Th, "Statistics for linguistics with R: a practical introduction" (2013), WalterDe Gruyter
- 한혜령, "Processing Scrambled Wh-Constructions in Head-Final Languages: Dependency Resolution and Feature Checking", 언어와 정보, 18(2), pp.59~79 (2014), 한국언어정보학회
- Korean Linguistics 19, "Japanese", pp.2011, Han, Chung-hye
- Han, Chung-hye, "On the syntax of relative clauses in Korean", Journal of Linguistics, 58(2), pp.319~347 (2013)
- Hardre, Patricia L., "Testing differential effects of computer-based, web-based and paper-based administration of questionnaire research instruments", British Journal of Educational Technology, 38(1), pp.5~22 (2007)
- Keller, Frank, "Timing accuracy of web experiments: A case study using the Webexp software package", Behavior Research Methods, 41, pp.1~12 (2009)
- 고희정, "A Hybrid Approach to Floating Quantifiers: Some Experimental Evidence", 언어연구, 29(1), pp.69~106 (2012), 언어정보연구소[DOI]
- Labov, William, "Sociolinguistic patterns" (1972), University of Pennsylvania Press
- Labov, William, "What is a linguistic fact?" (1975), Peter De Ridder Press
- 이용훈, "An Experimental Approach to Multiple Case Constructions in Korean", 언어와 정보, 17(2), pp.29~50 (2013), 한국언어정보학회
- 이용훈, "Alienable", Inalienable Possessions and Animacy in the Multiple Case Constructions: An Experimental Approach, 2014, pp.18(2), 31~58[DOI]
- Lim, Erle CH, "Computer-based versus pen-and-paper testing: Students' perception", Anals-Academy of Medicine Singapore, 35(9), pp.599~603 (2006)
- 남윤주, "한국어 문장처리에서부가어적 형용사와 명사의 의미통합", 언어연구, 31(2), pp.381~402 (2014), 언어정보연구소
- Norman, Geoff, "Likert scales, levels of measurement and the “laws” of statistics", Advances in Health Sciences Education, 15(5), pp.625~632 (2010)
- Noyes, Jan M., "Computer- vs. paper-based tasks: Are they equivalent?", Ergonomics, 51(9), pp.1352~1375 (2008)
- 박범식, "Re-identifying Null Arguments with Ellipsis: A Reply to Ahn and Cho (2013)", 생성문법연구, 23(4), pp.797~822 (2013), 한국생성문법학회
- Peirce, Jonathan W, "PsychoPy – psychophysics software in Python", Journal of Neuroscience Methods, 162(1), pp.8~13 (2007)
- R Core Team, ": a language and environment for statistical computing" (2014), R Foundation for Statistical Computing
- "Linger: a flexible platform for language processing experiments" (2003)[URL]
- 송상헌, "FAQ: Do Non-linguists Share the Same Intuition as Linguists?", 어학연구, 50(2), pp.357~386 (2014), 언어교육원
- Spencer, Nancy J, "Differences between linguists and nonlinguists in intuitions of grammaticality-acceptability", Journal of Psycholinguistic Research, 2(2), pp.83~98 (1973)
- Sprouse, Jon, "Continuous Acceptability, Categorical Grammaticality, and Experimental Syntax", Biolinguistics, 1, pp.123~134 (2007)
- Sprouse, Jon, "Assessing the reliability of textbook data in syntax: Adger’s Core Syntax", Journal of Linguistics, 48(3), pp.609~652 (2012)
- Sprouse, Jon, "A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001–2010", Lingua, 134, pp.219~248 (2013)
- Weerakoon, Gary, "The role of computer-aided assessment in health professional education: a comparison of student performance in computer-based and paper-and-pen multiple-choice tests", Medical Teacher, 23(2), pp.152~157 (2001)