본문 바로가기
KERD
논문KCI 등재

부정극어 허가 처리 평가: 영어 교과서를 학습한 L2 신경망 언어 모델을 활용하여

An assessment of processing negative polarity items by an L2 neural language model trained on English textbooks

2

조회

0

다운로드

2

인용

0

참고문헌

키워드

초록

The GRNN (Gulordava et al. 2018) neural language model (NLM) is viewed as a language learner in that it is trained with sentences and, like humans, ‘predicts’ the next-word given a sequence of words. Recent studies employing NLMs have reported their human-like performances in ‘understanding’ various linguistic phenomena. Building on previous studies, this paper aims to assess the level of linguistic knowledge that an NLM can acquire from a collection of English textbooks published in Korea for last two decades. We applied a psycholinguistic experimental method to compare the L2-GRNN to the L1-GRNN, focusing on the learning/processing of negative polarity item (NPI)-licensing in English. The L1-GRNN LM that was trained with the dataset from Wikipedia was reported to attain the linguistic knowledge that native speakers of English have. The L2-GRNN LM was trained with learning materials for Korean English learners. The result of analyzing the data extracted from the NLMs in NPI processing showed that the overall performance of the L2-GRNN NLM was far behind that of the L1-GRNN LM. In conclusion, this study demonstrates that the L2-GRNN has attained a far lower level of grammatical knowledge in NPI processing than its L1 counterpart. This result implies that the L2 dataset representing English textbooks published in Korea is not enough for the L2-GRNN NLM to acquire substantial grammatical knowledge that the L1-GRNN NLM as well as native speakers has.

Abstract

The GRNN (Gulordava et al. 2018) neural language model (NLM) is viewed as a language learner in that it is trained with sentences and, like humans, ‘predicts’ the next-word given a sequence of words. Recent studies employing NLMs have reported their human-like performances in ‘understanding’ various linguistic phenomena. Building on previous studies, this paper aims to assess the level of linguistic knowledge that an NLM can acquire from a collection of English textbooks published in Korea for last two decades. We applied a psycholinguistic experimental method to compare the L2-GRNN to the L1-GRNN, focusing on the learning/processing of negative polarity item (NPI)-licensing in English. The L1-GRNN LM that was trained with the dataset from Wikipedia was reported to attain the linguistic knowledge that native speakers of English have. The L2-GRNN LM was trained with learning materials for Korean English learners. The result of analyzing the data extracted from the NLMs in NPI processing showed that the overall performance of the L2-GRNN NLM was far behind that of the L1-GRNN LM. In conclusion, this study demonstrates that the L2-GRNN has attained a far lower level of grammatical knowledge in NPI processing than its L1 counterpart. This result implies that the L2 dataset representing English textbooks published in Korea is not enough for the L2-GRNN NLM to acquire substantial grammatical knowledge that the L1-GRNN NLM as well as native speakers has.

참고문헌 (23)

  1. Adam, G., "Predictive power of word surprisal for reading times is a linear function of language model quality", Proceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics, pp.10~18 (2018)
  2. Boyce, V., "Maze made easy: Better and easier measurement of incremental processing difficulty", Journal of Memory and Language, 11, pp.104082~ (2020)
  3. Boyce, V., "A-maze of natural stories: Texts are comprehensible using the maze task", Proceedings of the Architectures and Mechanisms for Languages Processing Conference (2020)
  4. Futrell, R., "RNNs as psycholinguistic subjects: Syntactic state and grammatical dependency", arXiv preprint arXiv:, 1809.01329 (2018)
  5. Futrell, R., "Nerual language models as psycholinguistic subjects: Representation of syntatic state", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techonologies, 1, pp.32~42 (2019)
  6. Gulordava, K., "Colorless green recurrent networks dream hierarchically", Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2018)
  7. Hale, J., "A probabilistic Earley parser as a psycholinguistic model", Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies, pp.1~8 (2001)
  8. Hu, J., "A systematic assessment of syntactic generalization in neural language models", arXiv preprint arXiv: 2005.03692 (2020)
  9. Levy, R., "Expectation-based syntactic comprehension,", Cognition, 106(3), pp.1126~1177 (2008)
  10. Linzen, T., "Assessing the ability of lstms to learn syntax sensitive dependencies", Transactions of the Association for Computational Linguistics, 4, pp.521~535 (2016)
  11. Linzen, T., "Uncertainty and expectation in sentence processing: Evidence from subcategorization distributions", Cognitive Science, 40(6), pp.1382~1411 (2016)
  12. Linzen, T., "Syntactic Structure from Deep Learning", Annual Review of Linguistics, 7(1), pp.195~212 (2021)
  13. Marvin, R., "Targeted Syntactic Evaluation of Language Models", Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp.1192~1202 (2018)
  14. van Schijndel, M., "Modeling garden path effects without explicit hierarchical syntax", CogSci (2018)
  15. Vaswani, A., "Attention is all you need", Advances in neural information processing systems, 30 (2017)
  16. Wang, A., "SuperGLUE: A Stickier Benchmark for General Purpose Language Understanding Systems", Advances in Neural Information Processing Systems, 32 (2019)
  17. Warstadt, A., "Transactions of the Association for Computational Linguistics", pp.625~641 (2019)
  18. Warstadt, A., "Transactions of the Association for Computational Linguistics", pp.377~392 (2020)
  19. Wilcox, E., "What do RNN language models learn about filler-gap dependencies?", Proceedings of the 2018 EMNLP Workshop Black-boxNLP: Analyzing and Interpreting Neural Network for NLP (2018)
  20. Wilcox, E., "Structure supervision Improves Learning of Non-local Grammatical Dependencies", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.3302~3312 (2019)
  21. Wilcox, E., "What syntactic structures block dependencies in RNN language models?", Proceedings of the 41st Annual Meeting of the Cognitive Science Society (2019)
  22. Wilcox, E., "On the predictive power of neural language models for human real-time comprehension behavior", arXiv preprint arXiv:, 2006.01912 (2020)
  23. Wilcox, E., "A targeted Assessment of Incremental Processing in Neural Language Models and Humans", arXiv Computation and Language, pp.2106.03232~ (2021)

관련 논문AI추천

이 간행물의 다른 논문 (4건)