On the Relationship between Syntactic and Semantic Encoding in Metric Space Language Models
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
The relationship between form and meaning is central to the theory of language. Traditionally, syntax and semantics are viewed as two different levels of representation. Based on insights from the intersection of dynamical systems theory and the theory of computation, and guided by linguistic data, I argue that there is only one space, a syntacticsemantic one. I model it here as a stable, countably infinite attractor of an iterated map dynamical system. One advantage of this approach is that it supports a unified treatment of grammatical and ungrammatical processing.
Abstract
The relationship between form and meaning is central to the theory of language. Traditionally, syntax and semantics are viewed as two different levels of representation. Based on insights from the intersection of dynamical systems theory and the theory of computation, and guided by linguistic data, I argue that there is only one space, a syntacticsemantic one. I model it here as a stable, countably infinite attractor of an iterated map dynamical system. One advantage of this approach is that it supports a unified treatment of grammatical and ungrammatical processing.
참고문헌 (52)
- Asher, N., "Integrating", Computational Linguistics, 42(4), pp.703~725 (2016)
- Bowman, Samuel, "Modeling natural language semantics in learned representations", PhD, pp.2016
- Cho, Pyeong Whan, "Incremental parsing in a continuous dynamical system: Sentence processing in gradient symbolic computation", Linguistic Vanguard, 3, pp.76~96 (2017)
- Chomsky, Noam, "Syntactic Structures" (1957), Mouton and Co
- Christiansen, Morten H., "Toward a connectionist model of recursion in human linguistic performance", Cognitive Science, 23, pp.157~205 (1999)
- Christianson, Kiel, "Thematic roles assigned along the garden path linger", Cognitive Psychology, 42, pp.368~407 (2001)
- Coecke, B., "Mathematical foundations for a compositional distributional model of meaning", Linguistic Analysis, 36, pp.345~384 (2010)
- Corballis, M. C., "Recursion, language, and starlings", Cognitive Science, 31, pp.697~704 (2007)
- Devlin, Jacob, "Pretraining of deep bidirectional transformers for language understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.4171~4186 (2019), Association for Computational Linguistics
- Elman, Jerey L., "Distributed representations, simple recurrent networks, and grammatical structure", Machine Learning, 7, pp.195~225 (1991)
- Ferreira, Fernanda, "Reanalysis in Sentence Processing", pp.73~100 (1998), Kluwer Academic Publishers
- Fodor, Janet Dean, "Reanalysis in Sentence Processing", pp.101~141 (1998), Kluwer Academic Publishers
- Gibson, Edward, "Linguistic complexity : Locality of syntactic dependencies", Cognition, 68(1), pp.1~76 (1998)
- Golubitsky, Martin, "The Symmetry Perspective" (2002), Springer
- Gordon, M. J. C, "The Denotational Description of Programming Languages: An Introduction" (2012), Springer
- Heim, Irene, "Meaning, Use and Interpretation of Language", pp.164~189 (1983), De Gruyter
- Jackendo, Ray, "Foundations of Language" (2002), Oxford
- Kamp, Hans, "From Discourse to Logic: Introduction to Model-theoretic Semantics of Natural Language, Formal Logic, and Discourse Representation Theory" (1993), Kluwer Academic Publishers
- Keller, Frank, "Gradience in grammar: Experimental and computational aspects of degrees of grammaticality", Doctoral, pp.2000
- Kempson, R., "Dynamic Syntax" (2001), Blackwell
- King, Jonathan, "Individual dierences in syntactic processing:the role of working memory", Journal of Memory and Language, 30, pp.580~602 (1991)
- Kozen, Dexter C, "Automata and Complexity Theory" (1997), Springer
- Levy, Omer, "Dependency-based word embeddings", Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp.302~308 (2014), The Association for Computational Linguistics
- Levy, Simon, "Infinite raam: A principled connectionist basis for grammatical competence", Proceedings of the 22nd Annual Meeting of the Cognitive Science Society, pp.298~303 (2000), Lawrence Erlbaum Associates
- Mikolov, Tomas, "Distributed representations of words and phrases and their compositionality", Advances in Neural Information Processing Systems 26: 27th NIPS (2013)
- Montague, Richard, "Formal Philosophy: Selected Papers of Richard Montague", pp.108~221 (1974), Yale University Press
- Moore, Cris, "Dynamical recognizers: Real-time language recognition by analog computers", Theoretical Computer Science, 201, pp.99~136 (1998)
- Plate, Tony A., "Holographic reduced representations", IEEE Transactions on Neural Networks, 6(3), pp.623~641 (1995)
- Rumelhart, David, "Backpropagation: Theory, Architectures, and Applications" (1995), Lawrence Erlbaum Associates
- Savitch, Walter J., "The Formal Complexity of Natural Language" (1987), Kluwer
- Siegelmann, H. T., "Analog computation via neural networks", Theoretical Computer Science, 131, pp.331~ (1994)
- Siegelmann, Hava T., "Neural Networks and Analog Computation: Beyond the Turing Limit" (1999), Birkhäuser
- Smolensky, Paul, "The Harmonic Mind: from neural computation to optimality-theoretic grammar" (2006), MIT Press
- Socher, Richard, "Recursive deep learning for natural language processing and computer vision", PhD, pp.2014
- Sorace, Antonella, "Gradience in linguistic data", Lingua, 115(11), pp.1497~1524 (2005)
- Spivey, Michael, "The Continuity of Mind" (2007), Oxford University Press
- Sprouse, Jonathan, "Experimental syntax and the variation of island eects in english and italian", Natural Language and Linguistic Theory, 34, pp.307~344 (2016)
- Steedman, Mark, "The Syntactic Process" (2000), The MIT Press
- Tabor, Whitney, "Fractal encoding of context-free grammars in connectionist networks", Expert Systems: The International Journal of Knowledge Engineering and Neural Networks, 17(1), pp.41~56 (2000)
- Tabor, Whitney, "The value of symbolic computation", Ecological Psychology, 14(1, pp.2) (2002), 21~52
- Tabor, Whitney, "Learning exponential state growth languages by hill climbing", IEEE Transactions on Neural Networks, 14(2), pp.444~446 (2003)
- Tabor, Whitney, "A dynamical systems perspective on the relationship between symbolic and non-symbolic computation", Cognitive Neurodynamics, 3(4), pp.415~427 (2009)
- Tabor, Whitney, "Recursion and recursion-like structure in ensembles of neural elements", Unifying Themes in Complex Systems. Proceedings of the VIII International Conference on Complex Systems, pp.1494~1508 (2011), New England Complex Systems Institute
- Tabor, Whitney, "Fractal grammars which recover from perturbation", Proceedings of the NIPS Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches (CoCo 2015) (2015), Creative Commons
- Tarski, Alfred, "The concept of truth in the languages of the deductive sciences (in polish)", Prace Towarzystwa Naukowego Warszawskiego, Wydzial III Nauk Matematyczno-Fizycznyc, 34 (1983)
- Tarski, Alfred, "Logic, Semantics, Metamathematics: Papers from 1923 to 1938", pp.152~278 (1983), Hackett Publishing Company
- Villata, Sandra, "Intervention eects in sentence processing", Ph.D, Department of Linguistics, pp.2017
- Villata, Sandra, "Intervention eects and relativized minimality: New experimental evidence from graded judgments", Lingua, 179, pp.76~96 (2016)
- Villata, Sandra, "Modeling ungrammaticality: A self-organizing model of islands", Proceedings of the 41st Annual Conference of the Cognitive Science Society, pp.1179~1184 (2019), Cognitive Science Society
- Villata, Sandra, "Gap-filling in syntactic islands: Evidence for island penetrability from the maze task", the 33nd annual CUNY Conference on Human Sentence Processing (2020)
- Wijnholds, Gijs, "Representation learning for type-driven composition", Proceedings of the 24th Conference on Computational Natural Language Learning, pp.313~324 (2020), Association for Computational Linguistic
