English Island Constraints Revisited: Experimental vs. Deep Learning Approach
1
조회
0
다운로드
7
인용
0
참고문헌
키워드
초록
This paper examined English island constraints with a deep learning model and compared the analysis results with those of Lee and Park (2018). This paper adopted the BERTLARGE model which was pretrained with the CoLA dataset. The dataset which was used in the current study were composed of 480 sentences. Among them, 80 sentences came from the target sentences in Lee and Park (2018), and 400 sentences from the CoLA dataset. These 480 sentences were used as an input to the BERTLARGE model, and the acceptability scores were calculated. Unlike previous deep learning models for syntactic acceptability, however, this paper calculated the probability where the given sentence could be acceptable, and the probability was converted into the acceptability scores. After the acceptability scores were obtained for all the target sentences, they were normalized into the z-scores. Statistical analyses were applied to them to compare the results of the BERTLARGE model and the experimental results in Lee and Park (2018). As a result, it was found that the results of the BERTLARGE model showed patterns similar to those in Sprouse et al. (2012) and Lee and Park (2018). Through the analysis, the followings were observed: (i) deep learning models could represent human beings’ syntactic acceptability to a considerable extends, (ii) the BERTLARGE model clearly identified the English island constraints, and (iii) both main factors (Island and Location) and their interaction (Island:Location) influenced the acceptability scores of island sentences.
Abstract
This paper examined English island constraints with a deep learning model and compared the analysis results with those of Lee and Park (2018). This paper adopted the BERTLARGE model which was pretrained with the CoLA dataset. The dataset which was used in the current study were composed of 480 sentences. Among them, 80 sentences came from the target sentences in Lee and Park (2018), and 400 sentences from the CoLA dataset. These 480 sentences were used as an input to the BERTLARGE model, and the acceptability scores were calculated. Unlike previous deep learning models for syntactic acceptability, however, this paper calculated the probability where the given sentence could be acceptable, and the probability was converted into the acceptability scores. After the acceptability scores were obtained for all the target sentences, they were normalized into the z-scores. Statistical analyses were applied to them to compare the results of the BERTLARGE model and the experimental results in Lee and Park (2018). As a result, it was found that the results of the BERTLARGE model showed patterns similar to those in Sprouse et al. (2012) and Lee and Park (2018). Through the analysis, the followings were observed: (i) deep learning models could represent human beings’ syntactic acceptability to a considerable extends, (ii) the BERTLARGE model clearly identified the English island constraints, and (iii) both main factors (Island and Location) and their interaction (Island:Location) influenced the acceptability scores of island sentences.
참고문헌 (36)
- Annamoradnejad, I., "ColBERT : Using BERT Sentence Embedding for Humor Detection", arXiv preprint arXiv:2004.12765, ~ (2020)
- Bard, E., "Magnitude Estimation of Linguistic Acceptability", Language, 72, pp.32~68 (1996)
- Barr, D., "Random Effects Structure for Confirmatory Hypothesis Testing", Journal of Memory and Language, 68, pp.255~278 (2013)
- Charniak, E., "BLLIP 1987-89 WSJ Corpus Release 1 LDC2000T43" (2000), Linguistic Data Consortium
- Chomsky, N., "A Festschrift for Morris Halle", pp.232~286 (1973), Holt, Rinehart and Winston
- Chomsky, N., "Barriers" (1986), MIT Press
- Cowart, W., "Experimental Syntax : Applying Objective Methods to Sentence Judgments" (1997), Sage Publications
- Devlin, J., "BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.4171~4186 (2019)
- Goldberg, Y., "Assessing BERT’s syntactic abilities", arXiv preprint arXiv:1901.05287, ~ (2019)
- Goodfellow, I., "Deep Learning" (2016), MIT Press
- Gulordava, K., "Colorless Green Recurrent Networks Dream Hierarchically", arXiv preprint arXiv:1803.11138, ~ (2018)
- Hu, J., "A Systematic Assessment of Syntactic Generalization in Neural Language Models", arXiv preprint arXiv:2005.03692, ~ (2020)
- Hu, J., "A Closer Look at the Performance of Neural Language Models on Reflexive Anaphor Licensing", Proceedings of the Society for Computation in Linguistics, pp.323~333 (2020)
- Keller, F., "Gradient in Grammar : Experimental and Computational Aspects of Degrees of Grammaticality", Ph. D, pp.2000
- 이용훈, "English Island Constraints by Natives and Korean Non-natives", 언어연구, 34(3), pp.439~455 (2018), 한국현대언어학회[DOI]
- Levy, R., "Expectation-based Syntactic Comprehension", Cognition, 106(3), pp.1126~1177 (2008)
- Marvin, R., "Targeted Syntactic Evaluation of Language Models", arXiv preprint arXiv:1808.09031, ~ (2018)
- Kwonsik Park, "Deep learning can contrast the minimal pairs of syntactic data", 언어연구, 38(2), pp.395~424 (2021), 언어정보연구소[DOI]
- 박연경, "English Island Sentences by Korean EFL Learners", 영어학연구, 24(1), pp.153~172 (2018), 한국영어학학회[DOI]
- R Core Team, "R : A Language and Environment for Statistical Computing", R Foundation for Statistical Computing, ~ (2021)
- Rizzi, L., "Relativized Minimality" (1990), MIT Press
- Ross, J., "Constraints on Variables in Syntax", Ph. D., pp.1967
- Szabolcsi, A., "The Blackwell Companion to Syntax", pp.479~531 (2007), Blackwell
- Schütze, C., "The Empirical Base of Linguistics : Grammaticality Judgments and Linguistic Methodology" (1996), University of Chicago Press
- Sprouse, J., "A Test of the Relation between Working Memory Capacity and Syntactic Island Effects", Language, 88, pp.82~123 (2012)
- Sprouse, J., "Experimental Syntax and Island Effects" (2013), Cambridge University Press
- Sprouse, J., "Proceedings of the 27th West Coast Conference on Formal Linguistics", pp.307~403 (2008), University of California
- Vaswani, A., "Attention Is All You Need", arXiv preprint arXiv:1706.03762, ~ (2017)
- Wang, A., "GLUE : A Multi-task Benchmark and Analysis Platform for Natural Language Understanding", arXiv preprint arXiv: 1804.07461, ~ (2019)
- Wang, A., "SuperGLUE: A Stickier Benchmark for Generalpurpose Language Understanding Systems", arXiv preprint arXiv:1905.00537, ~ (2020)
- Warstadt, A., "Neural Network Acceptability Judgments", arXiv preprint arXiv:1805.12471, ~ (2019)
- Warstadt, A., "Investigating BERT’s Knowledge of Language : Five Analysis Methods with NPIs", arXiv preprint arXiv:1909.02597, ~ (2019)
- Wilcox, E., "What Syntactic Structures Block Dependencies in RNN Language Models?", arXiv preprint arXiv:1905.10431, ~ (2019)
- Wilcox, E., "Hierarchical Representation in Neural Language Models: Suppression and Recovery of Expectations", arXiv preprint arXiv:1906.04068, ~ (2019)
- Wilcox, E., "What Do RNN Language Models Learn about Filler-Gap Dependencies?", arXiv preprint arXiv:1809. 00042, ~ (2018)
- Zhu, Y., "Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books", Proceedings of the IEEE International Conference on Computer Vision, pp.19~27 (2015)