본문 바로가기
KERD
논문KCI 등재

한국어 실험 통사론의 전개

Experimental syntax and its application to Korean

조용준
한말연구학회|2023.09|한말연구 (Korean Language Research)|제64권 제34호|pp. 1-20|DOI 10.16876/klrc.2023.64.34.1

0

조회

0

다운로드

0

인용

0

참고문헌

키워드

실험통사론대규모 언어모형experimental syntax형식적 방법과 비형식적 방법전문가와 비전문가 집단문법성과 수용성formal and informal methodexpert and naive groupsgrammaticality and accpetabilitylarge langage models

초록

In this study, we address various aspects of Korean experimental syntax and explore potential avenues for future research. Within the complex realm of experimental syntax, we critically examine prior research outcomes related to the interaction between formal and informal methodologies. Unlike previous studies exemplified by Sprouse et al. (2013), existing findings suggest a lack of convergence between these two methods, particularly in the context of the Korean language. However, contrasting these results with the generally convergent outcomes observed in experimental studies conducted in English highlights the presence of cross-linguistic differences, warranting further investigation. Moving forward, we conduct a comparative analysis of disparities between expert and non-expert groups, finding similar non-convergence consistent with previous observations. In future research endeavors, empirical investigations will be necessary to pinpoint areas of divergence and unravel the underlying factors. Furthermore, a noticeable gap exists in empirical studies concerning grammaticality and acceptability within the Korean language. Despite this gap, the active exploration of the 'grammatical illusion' phenomenon prompts us to advocate for comprehensive research efforts in these domains specific to the Korean context. In parallel, recent advancements in artificial intelligence research have facilitated the availability of extensive corpora for acceptability judgments, offering essential benchmarks for evaluating the effectiveness of large-scale language models. A similar trend extends to the Korean language, exemplified by the introduction of the 'NIKL Grammaticality Judgment Corpus (2020).' Given their relevance to the enduring discourse on language acquisition, such initiatives are poised to persist. Moreover, following the release of ChatGPT's code interpreter, the lowered barriers to statistical processing have made experimental methodology more approachable, allowing for smoother engagement in experimental research and syntactic analysis. In conclusion, we anticipate a surge in experimental research, considering the advancements in AI and the improved accessibility to statistical methods. These developments are expected to contribute to the active development of experimental methodology in syntactic research.

Abstract

In this study, we address various aspects of Korean experimental syntax and explore potential avenues for future research. Within the complex realm of experimental syntax, we critically examine prior research outcomes related to the interaction between formal and informal methodologies. Unlike previous studies exemplified by Sprouse et al. (2013), existing findings suggest a lack of convergence between these two methods, particularly in the context of the Korean language. However, contrasting these results with the generally convergent outcomes observed in experimental studies conducted in English highlights the presence of cross-linguistic differences, warranting further investigation. Moving forward, we conduct a comparative analysis of disparities between expert and non-expert groups, finding similar non-convergence consistent with previous observations. In future research endeavors, empirical investigations will be necessary to pinpoint areas of divergence and unravel the underlying factors. Furthermore, a noticeable gap exists in empirical studies concerning grammaticality and acceptability within the Korean language. Despite this gap, the active exploration of the 'grammatical illusion' phenomenon prompts us to advocate for comprehensive research efforts in these domains specific to the Korean context. In parallel, recent advancements in artificial intelligence research have facilitated the availability of extensive corpora for acceptability judgments, offering essential benchmarks for evaluating the effectiveness of large-scale language models. A similar trend extends to the Korean language, exemplified by the introduction of the 'NIKL Grammaticality Judgment Corpus (2020).' Given their relevance to the enduring discourse on language acquisition, such initiatives are poised to persist. Moreover, following the release of ChatGPT's code interpreter, the lowered barriers to statistical processing have made experimental methodology more approachable, allowing for smoother engagement in experimental research and syntactic analysis. In conclusion, we anticipate a surge in experimental research, considering the advancements in AI and the improved accessibility to statistical methods. These developments are expected to contribute to the active development of experimental methodology in syntactic research.

참고문헌 (68)

  1. 안희돈, "한국어 수용성 판단에서의 형식적 방법과 비형식적 방법의 비교 - 이산형 측정 자료의 경우", 한말연구, (51), pp.57~86 (2019), 한말연구학회[DOI]
  2. 안희돈, "문장 문법성 판단을 위한 기초 자료 구축" (2020), 국립국어원
  3. 안희돈, "한국어 수용성판단 자료의 학술지간 비교: ‘생성문법연구’와 ‘국어학’을 중심으로", 생성문법연구, 31(2), pp.343~356 (2021), 한국생성문법학회[DOI]
  4. 조용준, "한국어 실험 통사론의 모색", 박사, pp.2015
  5. 조용준, "한국어 수용성 판단에서의 형식적 방법과 비형식적 방법의 비교(2)", 언어학 연구, (51), pp.129~160 (2019), 한국중원언어학회[DOI]
  6. 조용준, "한국어 수용성판단에서의 학술지간 신뢰도 분석", 한말연구, (60), pp.61~90 (2021), 한말연구학회[DOI]
  7. Adger, D., "Core syntax: A Minimalist approach" (2003), Oxford University Press
  8. Bader, M., "Case and reanalysis", Journal of Psycholinguistic Research, 29, pp.37~52 (2000)
  9. Bard, E. G., "Magnitude estimation of linguistic acceptability", Language, 72, pp.32~68 (1996)
  10. Chaves, R.P., "What Don’t RNN Language Models Learn About Filler-Gap Dependencies?", Proceedings of the third meeting of the Society for Computation in Linguistics (SCiL) (2020)
  11. Choshen, L., "The grammarlearning trajectories of neural language models", arXiv preprint arXiv:2109.06096, ~ (2021)
  12. Chowdhury, S. A., "RNN simulations of grammaticality judgments on long-distance dependencies", Proceedings of the 27th international conference on computational linguistics, pp.133~144 (2018)
  13. Clifton, C., Jr., "Feature manipulation in sentence comprehension", Rivista di Linguistica, 11, pp.11~39 (1999)
  14. Cohen, J., "Statistical power analysis for the behavioral sciences" (1988), Erlbaum
  15. Cowart, W., "Experimental syntax: Applying objective methods to sentence judgments" (1997), Sage Publications, Inc
  16. Culbertson, J., "Are linguists better subjects?", British Journal for the Philosophy of Science, 60, pp.721~736 (2009)
  17. Dąbrowska, E., "Naïve vs. expert intuitions: an empirical study of acceptability judgments", Linguistic Review, 27, pp.1~23 (2010)
  18. Evans, G., "Pronouns", Linguistic Inquiry, 11, pp.337~362 (1980)
  19. Frazier, L., "Natural language processing: Psychological, computational, and theoretical perspectives", pp.129~189 (1985), Cambridge University Press
  20. Fults, S., "The source of syntactic illusions", Poster presented at the 17th Annual CUNY Conference on Human Sentence Processing (2004)
  21. Gibson, E., "Memory limitations and structural forgetting: The perception of complex ungrammatical sentences as grammatical", Language and Cognitive Processes, 14, pp.225~248 (1999)
  22. Goodall, G., "The Cambridge handbook of experimental syntax", pp.1~3 (2021), Cambridge University Press
  23. Gordon, P., "Intuitive knowledge of linguistic co-reference", Cognition, 62, pp.325~370 (1997)
  24. Gulordava, K., "Colorless green recurrent networks dream hierarchically", Proceedings of the Society for Computation in Linguistics, 2(1), pp.363~364 (2019)
  25. Hill, A. A., "Grammaticality", Word, 17, pp.1~10 (1961)
  26. Hoji, H., "Language faculty science" (2015), Cambridge University Press
  27. Hu, J, "A Systematic Assessment of Syntactic Generalization in Neural Language Models", Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.1725~1744 (2020), Association for Computational Linguistics
  28. Juzek, T, "Acceptability judgement tasks and grammatical theory", Doctor, pp.2016
  29. Keller, F., "Gradience in grammar: Experimental and computational aspects of degrees of grammaticality", Doctor, pp.2000
  30. Labov, W., "The Scope of American Linguistics", pp.77~113 (1975), The Peter de Ridder Press
  31. Steven Langsford, "Quantifying sentence acceptability measures: Reliability, bias, and variability", Glossa: a journal of general linguistics, 3(1), pp.1~34 (2018), Open Library of the Humanities[DOI]
  32. Lau, J. H., "Grammaticality, acceptability, and probability: A probabilistic view of linguistic knowledge", Cognitive Science, 41(5), pp.1202~1241 (2017)
  33. Lewis, S., "Aligning grammatical theories and language processing models", Journal of Psycholinguistic Research, 44(1), pp.27~46 (2015)
  34. Linzen, T., "Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies", Transactions of the Association for Computational Linguistics, 4, pp.521~535 (2016)
  35. Linzen, T., "What can linguistics and deep learning contribute to each other?", Response to Pater. Language, 95(1), pp.e99~e108 (2019)
  36. Linzen, T., "Syntactic Structure from Deep Learning", Annual Reviews of Linguistics, 7, pp.195~212 (2021)
  37. Tal Linzen, "The reliability of acceptability judgments across languages", Glossa: a journal of general linguistics, 3(1), pp.1~25 (2018), Open Library of the Humanities[DOI]
  38. Mahowald, K., "SNAP judgments: A small N acceptability paradigm (SNAP) for linguistic acceptability judgments", Language, 92(3), pp.619~635 (2016)
  39. Christopher D. Manning, "Emergent linguistic structure in artificial neural networks trained by self-supervision", Proceedings of the National Academy of Sciences, 117(48), pp.30046~30054 (2020), Proceedings of the National Academy of Sciences[DOI]
  40. Marantz, A., "Generative linguistics within the cognitive neuroscience of language", The Linguistic Review, 22, pp.429~445 (2005)
  41. Mikhailov, V., "RuCoLA: Russian Corpus of Linguistic Acceptability", Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp.5207~5227 (2022), Association for Computational Linguistics
  42. Isabel Papadimitriou, "Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT", Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp.2522~2532 (2021), Association for Computational Linguistics
  43. Pater. J., "Generative linguistics and neural networks at 60: Foundation, friction, and fusion", Language, 95(1), pp.e41~e74 (2019)
  44. Schütze, C. T., "The empirical base of linguistics: Grammatical judgments and linguistic methodology" (1996), The University of Chicago Press
  45. Schütze, C. T., "Linguistic evidence and grammatical theory", Wiley Interdisciplinary Reviews: Cognitive Science, 2(2), pp.206~221 (2010)
  46. Schütze, C. T., "Research Methods in Linguistics" (2013), Cambridge University Press
  47. Catherine Snow, "11. On the Secondary Nature of Syntactic Intuitions", Acceptability in Language, pp.163~178 (1977), DE GRUYTER[DOI]
  48. "Proceedings of the Twenty-Eighth Annual Meeting of the Association for Natural Language Processing", 2022, 1872~1877
  49. 송상헌, "5 or 7: Is the Choice So Important in Acceptability Judgment Testing?", 언어, 41(3), pp.449~480 (2016), 한국언어학회[DOI]
  50. 송상헌, "Different Ways to Test Acceptability Judgments", 생성문법연구, 25(2), pp.447~471 (2015), 한국생성문법학회[DOI]
  51. 송상헌, "Measuring the Degree of Convergence between Informal and Formal Acceptability Judgments for Korean", 언어, 42(4), pp.761~804 (2017), 한국언어학회[DOI]
  52. Spencer, N. J., "Differences between linguists and nonlinguists in intuitions of grammaticality-acceptability", Journal of Psycholinguistic Research, 2, pp.83~98 (1973)
  53. Sprouse, J, "The Oxford handbook of experimental syntax", pp.9~11 (2023), Oxford University Press
  54. Sprouse, J., "A program for experimental syntax", Doctor, pp.2007
  55. Sprouse, J., "Experimental Syntax", Mini-course taught at the Linguistic Society of Korea Linguistic School (2018), Seoul National University
  56. Sprouse, J., "Assessing the reliability of textbook data in syntax: Adger's Core Syntax", Journal of Linguistics, 48, pp.609~652 (2012)
  57. Sprouse, J., "Design sensitivity and statistical power in acceptability judgment experiments", Glossa, 2(1), pp.14~ (2017)
  58. Sprouse, J., "A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001-2010", Lingua, 134, pp.219~248 (2013)
  59. Stevens, S. S., "On the psychophysical law", Psychological Review, 64, pp.153~181 (1957)
  60. Trotta, D., "Monolingual and cross-lingual acceptability judgments with the Italian CoLA corpus", Findings of the Association for Computational Linguistics: EMNLP 2021, pp.2929~2940 (2021), Association for Computational Linguistics
  61. Warstadt, A., "Linguistic Analysis of Pretrained Sentence Encoders with Acceptability Judgments", arXiv preprint arXiv:1901.03438, ~ (2019)
  62. Warstadt, A., "Can neural networks acquire a structural bias from raw linguistic data?", Proceedings of the 42nd Annual Conference of the Cognitive Science Society (2020)
  63. Alex Warstadt, "BLiMP: The Benchmark of Linguistic Minimal Pairs for English", Transactions of the Association for Computational Linguistics, 8, pp.377~392 (2020), MIT Press - Journals[DOI]
  64. "Corpus of Linguistic Acceptability" (2018)[URL]
  65. Wierzbicka, A., "Lexicography and Conceptual Analysis" (1985), Karoma
  66. Wilcox, E., "What do RNN Language Models Learn about Filler–Gap Dependencies?", Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp.211~221 (2018)
  67. Beilei Xiang, "CLiMP: A Benchmark for Chinese Language Model Evaluation", Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp.2784~2790 (2021), Association for Computational Linguistics
  68. Xiang, M., "Testing the strength of the spurious licensing effect for negative polarity items", Talk at the 19th Annual CUNY Conference on Human Sentence Processing (2006)

관련 논문AI추천

한국어 수용성 판단에서의 형식적 방법과 비형식적 방법의 비교 - 이산형 측정 자료의 경우조준, 김형문, 안희돈, 조용준, 하지희한말연구학회 · 한말연구2019 · 제51호한국어 수용성판단 자료의 학술지간 비교: ‘생성문법연구’와 ‘국어학’을 중심으로안희돈, 조용준한국생성문법학회 · 생성문법연구2021 · 제31권 제2호한국어 수용성판단에서의 학술지간 신뢰도 분석안희돈, 조용준한말연구학회 · 한말연구2021 · 제60호실험통사론의 형식적 연구방법에 대한 고찰: 영어 수용성 판단 중심으로이상근한국생성문법학회 · 생성문법연구2017 · 제27권 제2호한국어 수용성 판단의 실험방법론 비교 연구전문기, 조용준언어과학회 · 언어과학연구2015 · 제72호Measuring the Degree of Convergence between Informal and Formal Acceptability Judgments for Korean송상헌, 오은정, 이상근, 최재웅한국언어학회 · 언어2017 · 제42권 제4호
원문 보기