Does BERT Learn Syntactic and Semantic Preferences in Picture Noun Phrase Interpretation?
0
조회
0
다운로드
1
인용
0
참고문헌
키워드
초록
This research investigates the Bidirectional Encoder Representations from Transformers (BERT) model’s ability to understand semantic and syntactic preferences of low-frequency expressions through reference resolution in picture noun phrases (PNPs). To this end, we report on three experiments that evaluate BERT’s understanding of reference resolution differences between personal pronouns and reflexives in possessor-less and possessed PNPs. Our experiments show that BERT exhibits human-like referential preferences with reflexives but not with personal pronouns. The findings for reflexive resolution suggest that BERT’s deep learning training does not solely rely on frequency information but serves as a mechanism for acquiring more systematic linguistic soft constraints Moreover, the different resolution patterns from the pronouns could be attributed to the reflexives’ more explicit referential dependency and their relatively low frequency.
Abstract
This research investigates the Bidirectional Encoder Representations from Transformers (BERT) model’s ability to understand semantic and syntactic preferences of low-frequency expressions through reference resolution in picture noun phrases (PNPs). To this end, we report on three experiments that evaluate BERT’s understanding of reference resolution differences between personal pronouns and reflexives in possessor-less and possessed PNPs. Our experiments show that BERT exhibits human-like referential preferences with reflexives but not with personal pronouns. The findings for reflexive resolution suggest that BERT’s deep learning training does not solely rely on frequency information but serves as a mechanism for acquiring more systematic linguistic soft constraints Moreover, the different resolution patterns from the pronouns could be attributed to the reflexives’ more explicit referential dependency and their relatively low frequency.
참고문헌 (33)
- Asudeh, A, "Anaphora and Argument Structure: Topics in the Syntax and Semantics of Reflexives and Reciprocals", Doctor, pp.1998
- Bauer, L., "Contrast in Language and Linguistics", Journal of English Linguistics, 36(1), pp.93~98 (2008)
- Chomsky, N., "Lectures on Government and Binding" (1981), Foris
- Da Costa, J. K., "Assessing the Ability of Transformer-based Neural Models to Represent Structurally Unbounded Dependencies", Society for Computation in Linguistics, 3(1), pp.189~198 (2020)
- Davis, F. L, "On the Limitations of Data: Mismatches between Neural Models of Language and Humans", Doctor, pp.2022
- Davies, M., "The 385+ Million Word Corpus of Contemporary American English(1990-2008+) : Design, Architecture, and Linguistic Insights", International Journal of Corpus Linguistics, 14(2), pp.159~190 (2009)
- Devlin, J., "Bert: Pre-training of Deep Bidirectional Transformers for Language Understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.4171~4186 (2018)
- Futrell, R., "Do RNNs Learn Human-like Abstract Word Order Preferences?", Proceedings of the Society for Computation in Linguistics (SCiL) 2019, pp.50~59 (2019)
- "Assessing BERT's Syntactic Abilities" (2018)[URL]
- Hale, J, "A Probabilistic Early Parser as a Psycholinguistic Model", presented at Second meeting of the north american chapter of the association for computational linguistics (2001)
- Hale, J., "Information-theoretical Complexity Metrics", Language and Linguistics Compass, 10(9), pp.397~412 (2016)[DOI]
- Levy, R., "Expectation-based Syntactic Comprehension", Cognition, 106(3), pp.1126~1177 (2008)
- Kaiser, E., "Do Perspective-sensitive Anaphors and Subjective Adjectives Exhibit Perspectival Uniformity? An Experimental Investigation", Glossa : a Journal of General Linguistics, 7(1), pp.1~52 (2022)[DOI]
- Kaiser, E., "Structural and Semantic Constraints on the Resolution of Pronouns and Reflexives", Cognition, 112(1), pp.55~80 (2009)
- Keller, F., "Proceedings of the 23rd Annual Conference of the Cognitive Science Society", pp.483~488 (2001), L. Erlbaum
- Keller, F, "Gradience in Grammar: Experimental and Computational Aspects of Degrees of Grammaticality", Doctor, pp.2000
- Kuno, S., "Functional Syntax: Anaphora, Discourse, and Empathy" (1987), University of Chicago Press
- Lau, J. H., "Grammaticality, Acceptability, and Probability : A Probabilistic View of Linguistic Knowledge", Cognitive Science, 41(5), pp.1202~1241 (2017)
- Petersen, E., "Lexical Semantics with Large Language Models:A Case Study of English “break”", Findings of the Association for Computational Linguistics: EACL 2023, pp.490~511 (2023), Association for Computational Linguistics
- Pollard, C., "Anaphors in English and the Scope of Binding Theory", Linguistic Inquiry, 23, pp.261~303 (1992)
- "R: A Language and Environment for Statistical Computing" (2021)[URL]
- Reinhart, T., "Reflexivity", Linguistic Inquiry, 24, pp.657~720 (1993)
- Runner, J., "Assignment of Reference to Reflexives and Pronouns in Picture Noun Phrases: Evidence from Eye Movements", Cognition, 89(1), pp.B1~B13 (2003)[DOI]
- Shin, U., "BERT Learns More than Word Frequency Information : A Case Study of Do-Be Constructions", Korean Journal of Linguistics, 47(3), pp.467~489 (2022)
- Shin, U., "Is-command Machine-learnable", The Linguistic Association of Korea Journal, 29(1), pp.183~204 (2021)
- Sorace, A., "Gradience in Linguistic Data", Lingua, 115(11), pp.1497~1524 (2005)
- "Short Distance Pronouns in Representational Noun Phrases and a Grammar of Sentience" (2003)[URL]
- Wasow, T., "Postverbal Behavior" (2002), CSLI Publications
- Weissweiler, L., "The Better your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative", Proceedings of the 2022Conference on Empirical Methods in Natural Language Processing, pp.10859~10882 (2022)
- Yu, C., "Word Frequency Does Not Predict Grammatical Knowledge in Language Models", Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.4040~4054 (2020)
- Zhao, Y, "How to Probe Linguistic Knowledge and Bias", Doctor, pp.2022