How the L2 LSTM language model predicts island effects
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In this paper, we investigate how well L2 language models learn about particular types of controlled contexts that indicate representation of syntactic dependencies (e.g., the filler-gap dependency and possessive pronoun gender expectation). Building the three language models, we assess LM’s knowledge by comparing the interaction in island configurations to that in non-island minimal paris. In so doing, we provide new evidence that L2 models cannot significantly maintain their expectations through island constructions while L1 model is able to thread expectations within islands. Furthermore, L2 models fail to correctly learn about empty syntactic positions while L1 model is able to generalize about them. Additionally, we control the amount of training data for L2 models to examine the correlation of the LM’s performances. However, we cannot investigate the correlation between the size of sentences that L2ers learn and their sentence processing patterns. Previous study is inconclusive about whether L2ers learn these linguistic constraints. Here, using new methodology, we show evidence about the characteristics of L2ers’ behavior.
Abstract
In this paper, we investigate how well L2 language models learn about particular types of controlled contexts that indicate representation of syntactic dependencies (e.g., the filler-gap dependency and possessive pronoun gender expectation). Building the three language models, we assess LM’s knowledge by comparing the interaction in island configurations to that in non-island minimal paris. In so doing, we provide new evidence that L2 models cannot significantly maintain their expectations through island constructions while L1 model is able to thread expectations within islands. Furthermore, L2 models fail to correctly learn about empty syntactic positions while L1 model is able to generalize about them. Additionally, we control the amount of training data for L2 models to examine the correlation of the LM’s performances. However, we cannot investigate the correlation between the size of sentences that L2ers learn and their sentence processing patterns. Previous study is inconclusive about whether L2ers learn these linguistic constraints. Here, using new methodology, we show evidence about the characteristics of L2ers’ behavior.
참고문헌 (11)
- Baayen, R. Harald., "Mixed-effects modeling with crossed random effects for subjects and items", Journal of memory and language, 59(4), pp.390~412 (2018)[DOI]
- Bernardy, Jean-Philippe., "Using deep neural networks to learn syntactic agreement", Linguistic Issues in Language Technology (2017)
- Gulordava, Kristina., "Colorless green recurrent networks dream hierarchically", arXiv: 1803.11138, ~ (2018)
- Hale, John., "A probabilistic Earley parser as a psycholinguistic model", Second meeting of the north american chapter of the association for computational linguistics (2001)
- Hochreiter, Sepp., "Long short-term memory", Neural computation, 9(8), pp.1735~1780 (1997)[DOI]
- Jozefowicz, Rafal., "Exploring the limits of language modeling", arXiv: 1602.02410, ~ (2016)
- Levy, Roger, "Expectation-based syntactic comprehension", Cognition, 106(3), pp.1126~1177 (2008)[DOI]
- Linzen, Tal., "Assessing the ability of LSTMs to learn syntax-sensitive dependencies", Transactions of the Association for Computational Linguistics, 4, pp.521~535 (2016)[DOI]
- Smith, Nathaniel J., "The effect of word predictability on reading time is logarithmic", Cognition, 128(3), pp.302~319 (2013)[DOI]
- Wilcox, Ethan., "What syntactic structures block dependencies in RNN language models?", arXiv :1905.10431, ~ (2019)
- Wilcox, Ethan., "What do RNN Language Models Learn about Filler-Gap Dependencies?", arXiv:1809.00042, ~ (2018)