Syntactic priming in the L2 neural language model
1
조회
0
다운로드
3
인용
0
참고문헌
키워드
초록
In recent years, the increasing abilities of neural (-network) language models (NLMs) have led to examining their representation of syntactic structures. To assess the linguistic knowledge that NLMs acquire, researchers have leveraged the traditional syntactic priming paradigm to investigate the potentials of NLMs in learning abstract structural information. In this study, we concentrated on investigating the extent to which the L2 NLM is sensitive to syntactic priming in psycholinguistic. Following Sinclair et al. (2022), we adopted a novel metric with which we controled various linguistic factors. Based on this adoption, we implemented the L2 NLM trained on the L2 corpus and explored which factors influence the strength of priming effects. In so doing, we discovered that the L2 NLM is also sensitive to various linguistic factors but displays irregular syntactic priming performances depending on experiments with different types of controlled materials.
Abstract
In recent years, the increasing abilities of neural (-network) language models (NLMs) have led to examining their representation of syntactic structures. To assess the linguistic knowledge that NLMs acquire, researchers have leveraged the traditional syntactic priming paradigm to investigate the potentials of NLMs in learning abstract structural information. In this study, we concentrated on investigating the extent to which the L2 NLM is sensitive to syntactic priming in psycholinguistic. Following Sinclair et al. (2022), we adopted a novel metric with which we controled various linguistic factors. Based on this adoption, we implemented the L2 NLM trained on the L2 corpus and explored which factors influence the strength of priming effects. In so doing, we discovered that the L2 NLM is also sensitive to various linguistic factors but displays irregular syntactic priming performances depending on experiments with different types of controlled materials.
참고문헌 (17)
- Bhattacharya, D., "Filler-gaps that neural networks fail to generalize", Proceedings of the 24th Conference on Computational Natural Language Learning, pp.486~495 (2020)
- Bock, J. K., "Syntactic persistence in language production", Cognitive Psychology, 18(3), pp.355~387 (1986)
- Brown, T., "Language models are few-shot learners", Advances in Neural Information Processing Systems, 3, pp.1877~1901 (2020)
- 최선주, "How are Korean Neural Language Models ‘surprised’ Layerwisely?", 언어과학, 28(4), pp.301~317 (2021), 한국언어과학회[DOI]
- 최선주, "An L2 Neural Language Model of Adaptation to Dative Alternation in English", 현대영미어문학, 40(1), pp.143~159 (2022), 현대영미어문학회
- 최선주, "Syntactic Priming by L2 LSTM Language Models", 언어연구, 37(4), pp.475~489 (2022), 한국현대언어학회[DOI]
- 최선주, "An L2 Neural Language Model of Adaptation", 영어학, 22, pp.547~562 (2022), 한국영어학회[DOI]
- Gulordava, K., "Colorless green recurrent networks dream hierarchically", arXiv preprint arXiv:1803.11138, ~ (2018)
- 김유희, "The Ability of L2 LSTM Language Models to Learn the Filler-Gap Dependency", 한국컴퓨터정보학회논문지, 25(11), pp.27~40 (2020), 한국컴퓨터정보학회[DOI]
- Linzen, T., "Assessing the ability of LSTMs to learn syntax-sensitive dependencies", Transactions of the Association for Computational Linguistics, 4, pp.521~535 (2016)
- Prasad, G., "Using priming to uncover the organization of syntactic representations in neural language models", arXiv preprint arXiv:1909.10579, ~ (2019)
- Sanh, V., "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter", arXiv preprint arXiv:1910.01108, ~ (2019)
- Sinclair, A., "Structural persistence in language models : Priming as a window into abstract language representations", Transactions of the Association for Computational Linguistics, 10, pp.1031~1050 (2022)
- Tenney, I., "BERT rediscovers the classical NLP pipeline", arXiv preprint arXiv:1905.05950, ~ (2019)
- Van Schijndel, M., "A neural model of adaptation in reading", arXiv preprint arXiv:1808.09930, ~ (2018)
- Warstadt, A., "BLiMP : The benchmark of linguistic minimal pairs for English", Transactions of the Association for Computational Linguistics, 8, pp.377~392 (2020)
- Zhang, Y., "Dialogpt : Large-Scale generative pre-training for conversational response generation", arXiv preprint arXiv:1911.00536, ~ (2019)