Measuring the Degree of Convergence between Informal and Formal Acceptability Judgments for Korean
0
조회
0
다운로드
7
인용
0
참고문헌
키워드
초록
Song, Sanghoun, Lee Sang-Geun , Choe Jae-Woong & Oh Eunjeong. 2017. Measuring the Degree of Convergence between Informal and Formal Acceptability Judgments for Korean. Korean Journal of Linguistics, 42-4, 761-804. The present study attempts to quantify the extent to which the acceptability judgments of naïve native speakers coincide with those of linguists in the domain of Korean syntax. We excerpted 1,554 pairs from articles published in a theoretical syntax journal on Korean, Studies in Generative Grammar, from 1991 to 2014, and then randomly chose 287 pairs. Employing PsychoPy as the toolkit and a five-point Likert scale as the task, we collected 68,388 data points from 328 naïve participants. Two statistical analyses, a one-tailed T-test and a κ-test, were then run on the Z-score transformed data points. The T-test, which determines the directionality of difference, yielded a convergence rate of 88.15% between the two types of judgments. The κ-test, which estimates both the size and directionality of difference, obtained a convergence rate of 82.93%. 16 pairs that passed the T-test in a measure of directionality failed to pass the κ-test when testing both directionality and size of difference by degree of match/mismatch between the two types of judgments. Such consideration leads us to believe that the κ-test is more appropriate for detecting the size of the difference between the two types of judgments, as well as the directionality of the difference. We also classified convergence results into 4 patterns: (i) convergence for both sentences in the pair, (ii) divergence for only the bad sentences in the pair, (iii) divergence for only the good sentences in the pair, and (iv) divergence for both sentences in the pair. These patterns play a role in interpreting different convergence rates obtained in the study. We discuss the implications of the κ-test convergence rate for our research question and the use of κ-tests and patterns in experimental syntax studies. (Incheon National University, Korea University, Sangmyung University)
Abstract
Song, Sanghoun, Lee Sang-Geun , Choe Jae-Woong & Oh Eunjeong. 2017. Measuring the Degree of Convergence between Informal and Formal Acceptability Judgments for Korean. Korean Journal of Linguistics, 42-4, 761-804. The present study attempts to quantify the extent to which the acceptability judgments of naïve native speakers coincide with those of linguists in the domain of Korean syntax. We excerpted 1,554 pairs from articles published in a theoretical syntax journal on Korean, Studies in Generative Grammar, from 1991 to 2014, and then randomly chose 287 pairs. Employing PsychoPy as the toolkit and a five-point Likert scale as the task, we collected 68,388 data points from 328 naïve participants. Two statistical analyses, a one-tailed T-test and a κ-test, were then run on the Z-score transformed data points. The T-test, which determines the directionality of difference, yielded a convergence rate of 88.15% between the two types of judgments. The κ-test, which estimates both the size and directionality of difference, obtained a convergence rate of 82.93%. 16 pairs that passed the T-test in a measure of directionality failed to pass the κ-test when testing both directionality and size of difference by degree of match/mismatch between the two types of judgments. Such consideration leads us to believe that the κ-test is more appropriate for detecting the size of the difference between the two types of judgments, as well as the directionality of the difference. We also classified convergence results into 4 patterns: (i) convergence for both sentences in the pair, (ii) divergence for only the bad sentences in the pair, (iii) divergence for only the good sentences in the pair, and (iv) divergence for both sentences in the pair. These patterns play a role in interpreting different convergence rates obtained in the study. We discuss the implications of the κ-test convergence rate for our research question and the use of κ-tests and patterns in experimental syntax studies. (Incheon National University, Korea University, Sangmyung University)
참고문헌 (31)
- Adger, David, "Core Syntax: A Minimalist Approach" (2003), Oxford University Press
- Agresti, Alan, "Statistical Methods for the Social Sciences" (1997), Prentice-Hall
- Bridges, George S., "Scaling Seriousness: An Evaluation of Magnitude and Category Scaling Techniques", The Journal of Criminal Law and Criminology, 66(2), pp.215~221 (1975)
- Chang, Suk-Jin, "Information-based Korean Grammar" (1995), Hanshin Publishing
- Chang, Suk-Jin, "Information Unpackaging: A Constraint-based Grammar Approach to Topic-Focus Articulation", Japanese, 2002, pp.10, 451~464
- 최재웅, "피험자의 배경 변인이 수용성 판단에 영향을 끼치는가?", 언어와언어학, (74), pp.181~206 (2017), 언어연구소[DOI]
- Choi, Hye-Won, "Optimizing Structure in Context: Scrambling and Information Structure" (1999), CSLI Publications
- Chomsky, Noam, "Knowledge of Language: Its Nature, Origin, and Use" (1986), Praeger
- Cohen, Jacob, "A Coefficient of Agreement for Nominal Scales", Educational and Psychological Measurement, 20(1), pp.37~46 (1960)
- Colman, Andrew M., "Comparing Rating Scales of Different Lengths: Equivalence of Scores from 5-point and 7-point Scales", Psychological Reports, 80(2), pp.355~362
- Dawes, John G, "Do Data Characteristics Change According to the Number of Scale Points Used?", International Journal of Market Research, 50(1), pp.61~77 (2008)
- Flickinger, Dan, "Language from a Cognitive Perspective: Grammar, Usage, and Processing", pp.31~50 (2011), CSLI Publications
- Greenbaum, Sidney, "Contextual Influence on Acceptability Judgments", Linguistics, 187, pp.5~11 (1977)
- Gries, Stefan Th, "Statistics for Linguistics with R: A Practical Introduction" (2013), Walter De Gruyter
- Stress, Focus, and Scrambling in Japanese, "Ishihara, Shinichiro", MIT, 142~175, pp.MIT Working Papers in Linguistics 39 (2001)
- Johnson, Keith, "Quantitative Methods in Linguistics" (2008), Blackwell Pub
- Krosnick, Jon A., "Handbook of Survey Research", pp.263~314 (2010), Emerald Publishing Group Limited
- Kwon, Jae-il, "Korean Syntax" (1992), Minumsa
- Labov, William, "What Is a Linguistic Fact?" (1975), Peter De Ridder Press
- Norman, Geoff, "Likert Scales, Levels of Measurement and the"laws"of Statistics", Advances in Health Sciences Education, 15(5), pp.625~632 (2010)
- Peirce, Jonathan W, "PsychoPy – Psychophysics Software in Python", Journal of Neuroscience Methods, 162(1), pp.8~13 (2007)
- R Core Team, "R: A Language and Environment for Statistical Computing", R Foundation for Statistical Computing, ~ (2014)
- Rooth, Mats, "A Theory of Focus Interpretation", Natural Language Semantics, 1(1), pp.75~116 (1992)
- Schütze, Carson T, "Research Methods in Linguistics", pp.27~50 (2013), Cambridge University Press
- 송상헌, "In Defense of Comparisons between Formal and Informal Acceptability Judgments", 생성문법연구, 27(4), pp.893~902 (2017), 한국생성문법학회
- Spencer, Nancy J, "Differences between Linguists and Nonlinguists in Intuitions of Grammaticality-Acceptability", Journal of Psycholinguistic Research, 2(2), pp.83~98 (1973)
- Sprouse, Jon, "Assessing the Reliability of Textbook Data in Syntax : Adger’s Core Syntax", Journal of Linguistics, 48(3), pp.609~652 (2012)
- Sprouse, Jon, "A Comparison of Informal and Formal Acceptability Judgments Using a Random Sample from Linguistic Inquiry 2001–2010", Lingua, 134, pp.219~248 (2013)
- Valian, Virginia, "Language, Mind, and Brain", pp.179~188 (1982), Lawrence Erlbaum
- Viera, Anthony J., "Understanding Interobserver Agreement : the Kappa Statistic", Family Medicine, 37(5), pp.360~363 (2005)
- Zubizarreta, Maria Luisa, "Prosody, Focus, and Word Order" (1998), The MIT Press