Data-driven models and computational tools for neurolinguistics: a language technology perspective
0
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
In this paper, our focus is the connection and influence of language technologies on the research in neurolinguistics. We present a review of brain imaging-based neurolinguistic studies with a focus on the natural language representations, such as word embeddings and pre-trained language models. Mutual enrichment of neurolinguistics and language technologies leads to development of brain-aware natural language representations. The importance of this research area is emphasized by medical applications
Abstract
In this paper, our focus is the connection and influence of language technologies on the research in neurolinguistics. We present a review of brain imaging-based neurolinguistic studies with a focus on the natural language representations, such as word embeddings and pre-trained language models. Mutual enrichment of neurolinguistics and language technologies leads to development of brain-aware natural language representations. The importance of this research area is emphasized by medical applications
참고문헌 (79)
- Abnar, S., "Experiential, distributional and dependency-based word embeddings have complementary roles in decoding brain activity", Proceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics(CMCL 2018), pp.57~66 (2018)
- Abnar, S., "Blackbox Meets Blackbox : Representational Similarity & Stability Analysis of Neural Language Models and Brains", Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp.191~203 (2019)
- Bartunov, Sergey, "Breaking sticks and ambiguities with adaptive skip-gram", Artificial Intelligence and Statistics, pp.130~138 (2016)
- Bartusiak, Roman, "WordNet2Vec: Corpora agnostic word vectorization method", Neurocomputing, 326, pp.141~150 (2019)
- Bojanowski, Piotr, "Enriching word vectors with subword information", Transactions of the Association for Computational Linguistics, 5, pp.135~146 (2017)
- Buckner, R. L, "Event-related fMRI and the hemodynamic response", Human Brain Mapping (1998)[DOI]
- Cer, Daniel, "Universal sentence encoder for English", Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp.169~174 (2018)
- Chung, Yu-An, "Speech2Vec: A Sequence-to-Sequence Framework for Learning Word Embeddings from Speech", Proc. Interspeech, pp.811~815 (2018)
- Collobert, R., "Natural language processing (almost) from scratch", The Journal of Machine Learning Research, 12, pp.2493~2537 (2011)
- Ciaran Cooney, "Neurolinguistics Research Advancing Development of a Direct-Speech Brain-Computer Interface", iScience, 8, pp.103~125 (2018), Elsevier BV[DOI]
- Dai, Zihang, "Transformer-xl: Attentive language models beyond a fixed-length context", arXiv preprint arXiv:1901.02860, ~ (2019)
- Deriu, Jan, "Leveraging large amounts of weakly supervised data for multi-language sentiment classification", Proceedings of the 26th international conference on world wide web, pp.1045~1052 (2017)
- Devlin, Jacob, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.4171~4186 (2019)
- Eger, Steffen, "Pitfalls in the Evaluation of Sentence Embeddings", Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pp.55~60 (2019)
- Ettinger, A., "Modeling N400 amplitude using vector space models of word representation", CogSci., 2016, 2016, pp.1445~1450 (2016)
- Faruqui, Manaal, "Retrofitting Word Vectors to Semantic Lexicons", Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.1606~1615 (2015)
- Gerasimos Fergadiotis, "Algorithmic Classification of Five Characteristic Types of Paraphasias", American Journal of Speech-Language Pathology, 25(4S) (2016), American Speech Language Hearing Association[DOI]
- Fiebach, C. J., "Processing concrete words : fMRI evidence against a specific right-hemisphere involvement", Neuropsychologia, 42(1), pp.62~70 (2004)
- Frank Stefan L, "Reading time data for evaluating broad-coverage models of english sentence processing", Behavior Research Methods, 45(4), pp.1182~1190 (2013)
- Kathleen C. Fraser, "Multilingual word embeddings for the assessment of narrative speech in mild cognitive impairment", Computer Speech & Language, 53, pp.121~139 (2019), Elsevier BV[DOI]
- Friederici, A. D., "Segregating semantic and syntactic aspects of processing in the human brain : an fMRI investigation of different word types", Cerebral cortex, 10(7), pp.698~705 (2000)
- Fyshe, Alona, "Interpretable semantic vectors from a joint model of brain-and text-based meaning", Proceedings of the conference. Association for Computational Linguistics. Meeting, 2014, pp.489~ (2014), NIH Public Access
- Alona Fyshe, "The lexical semantics of adjective–noun phrases in the human brain", Human Brain Mapping, 40(15), pp.4457~4469 (2019), Wiley[DOI]
- Goldberg, Yoav, "Assessing BERT's Syntactic Abilities", arXiv preprint arXiv:1901.05287, ~ (2019)
- Jain, S., "Incorporating context into language encoding models for fmri", bioRxiv, 327601~ (2018)
- Jauhar, Sujay Kumar, "Ontologically grounded multi-sense representation learning for semantic vector space models", Proceedings of the 2015 conference of the north American chapter of the Association for Computational Linguistics: human language technologies, pp.683~693 (2015)
- Harris, Zellig S, "Distributional structure", Word, 10(2-3), pp.146~162 (1954)
- Michael Hanke, "A high-resolution 7-Tesla fMRI dataset from complex natural stimulation with an audio movie", Scientific Data, 1(1), pp.1~18 (2014), Springer Science and Business Media LLC[DOI]
- "CogniVal: A Framework for Cognitive Word Embedding Evaluation" (2019)[URL]
- R. J. Huster, "Methods for Simultaneous EEG-fMRI: An Introductory Review", Journal of Neuroscience, 32(18), pp.6053~6060 (2012), Society for Neuroscience[DOI]
- Alexander G. Huth, "Natural speech reveals the semantic maps that tile human cerebral cortex", Nature, 532(7600), pp.453~458 (2016), Springer Science and Business Media LLC[DOI]
- "The language network is recruited but not required for non-verbal semantic processing" (2019)[URL]
- Kiros, Ryan, "Skip-thought vectors", Advances in neural information processing systems, pp.3294~3302 (2015)
- Gina R. Kuperberg, "Neural mechanisms of language comprehension: Challenges to syntax", Brain Research, 1146, pp.23~49 (2007), Elsevier BV[DOI]
- Landauer, Thomas K., "A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge", Psychological review, 104(2), pp.211~ (1997)
- Le, Quoc, "Distributed representations of sentences and documents", International conference on machine learning, pp.1188~1196 (2014)
- Lan, Zhenzhong, "ALBERT: A Lite BERT for Self-supervised Learning of Language Representations" (2019), International Conference on Learning Representations
- Linz, N., "Using Neural Word Embeddings in the Analysis of the Clinical Semantic Verbal Fluency Task", IWCS (2017) 12th International Conference on Computational Semantics (2017)
- Liu, Nelson F., "Linguistic Knowledge and Transferability of Contextual Representations", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.1073~1094 (2019)
- Liu, Yinhan, "Roberta: A robustly optimized bert pretraining approach", arXiv preprint arXiv:1907.11692, ~ (2019)
- Mao, Junhua, "Training and evaluating multimodal word embeddings with large-scale web annotated images", Advances in neural information processing systems, pp.442~450 (2016)
- "Classification of Semantic Paraphasias: Optimization of a Word Embedding Model" (2019)[URL]
- Mikolov, Tomas, "Distributed representations of words and phrases and their compositionality", Advances in neural information processing systems, pp.3111~3119 (2013)
- Mitchell, T. M., "Predicting Human Brain Activity Associated with the Meanings of Nouns", Science, 320, pp.1191~1195 (2008)
- Kevin Murphy, "Resting-state fMRI confounds and cleanup", NeuroImage, 80, pp.349~359 (2013), Elsevier BV[DOI]
- Murphy, B., "Selecting corpus-semantic models for neurolin- guistic decoding", Proceedings of the First Joint Conference on Lexical and Computational Semantics, 1,2 (2012), Association for Computational Linguistics
- Oota, S. R., "fMRI Semantic Category Decoding Using Linguistic Encoding of Word Embeddings", International Conference on Neural Information Processing, pp.3~15 (2018), Springer
- Serguei V.S. Pakhomov, "Characterizing cognitive performance in a large longitudinal study of aging with computerized semantic indices of verbal fluency", Neuropsychologia, 89, pp.42~56 (2016), Elsevier BV[DOI]
- Pelevina, Maria, "Making Sense of Word Embeddings", Proceedings of the 1st Workshop on Representation Learning for NLP, pp.174~183 (2016)
- Pennington, Jeffrey, "Glove : Global vectors for word representation", Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp.1532~1543 (2014)
- Pereira, F., "Toward a universal decoder of linguistic meaning from brain activation", Nature communications, 9(1), pp.963~ (2018)
- Peters, Matthew E., "Deep contextualized word representations", arXiv preprint arXiv:1802.05365, ~ (2018)
- Peters, Matthew E., "Knowledge Enhanced Contextual Word Representations", Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp.43~54 (2019)
- Pires, Telmo, "How Multilingual is Multilingual BERT?", Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp.4996~5001 (2019)
- Raffel, Colin, "Exploring the limits of transfer learning with a unified text-to-text transformer", arXiv preprint arXiv:1910.10683, ~ (2019)
- PHILIPPE H. ROBERT, "Clustering and switching strategies in verbal fluency tasks: Comparison between schizophrenics and healthy adults", Journal of the International Neuropsychological Society, 4(6), pp.539~546 (1998), Cambridge University Press (CUP)[DOI]
- Rodrigues, J. A., "Predicting brain activation with WordNet embeddings", Proceedings of the Eight Workshop on Cognitive Aspects of Computational Language Learning and Processing, pp.1~5 (2018)
- Ardi Roelofs, "Phonological cueing of word finding in aphasia: insights from simulations of immediate and treatment effects", Aphasiology, pp.1~17 (2019), Informa UK Limited[DOI]
- Ardi Roelofs, "The WEAVER model of word-form encoding in speech production", Cognition, 64(3), pp.249~284 (1997), Elsevier BV[DOI]
- Rustandi Indrayana, "Integrating Multiple-Study Multiple-Subject fMRI Datasets Using Canonical Correlation Analysis", MICCAI 2009 Workshop: Statistical modeling and detection issues in intra- and inter-subject functional MRI data analysis (2009)
- Salle, Alexandre, "Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations", Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016, pp.419~424 (2016)
- Riitta Salmelin, "Clinical neurophysiology of language: The MEG approach", Clinical Neurophysiology, 118(2), pp.237~254 (2007), Elsevier BV[DOI]
- Sejnowski, T. J., "Putting big data to good use in neuroscience", Nature neuroscience, 17(11), pp.1440~ (2014)
- Søgaard, A., "Evaluating word embeddings with fMRI and eye-tracking", Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP, pp.116~121 (2016)
- Sharaev, M., "Pattern Recognition Pipeline for Neuroimaging Data", 8th IAPR TC3 Workshop on Artificial Neural Networks in Pattern Recognition, 2018, pp.306~319 (2018)
- Sharaev, M., "MRI-Based Diagnostics of Depression Concomitant with Epilepsy: In Search of the Potential Biomarkers", IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp.555~564 (2018), IEEE
- Schwartz, Dan, "Understanding language-elicited EEG data by predicting it from a fine-tuned language model", Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, pp.43~57 (2019)
- Schwartz, Dan, "Inducing brain-relevant bias in natural language processing models", Advances in Neural Information Processing Systems (2019)
- Toneva M., "Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain)", Advances in Neural Information Processing Systems (2019)
- Angela K. Troyer, "Clustering and switching as two components of verbal fluency: Evidence from younger and older healthy adults.", Neuropsychology, 11(1), pp.138~146 (1997), American Psychological Association (APA)[DOI]
- Angela K Troyer, "Clustering and switching on verbal fluency: the effects of focal frontal- and temporal-lobe lesions", Neuropsychologia, 36(6), pp.499~504 (1998), Elsevier BV[DOI]
- David C. Van Essen, "The WU-Minn Human Connectome Project: An overview", NeuroImage, 80, pp.62~79 (2013), Elsevier BV[DOI]
- Wise, R., "Language activation studies with positron emission tomography", Ciba Found Symp, 163, pp.218~234 (1991)
- Leila Wehbe, "Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses", PLoS ONE, 9(11), pp.e112575~ (2014), Public Library of Science (PLoS)[DOI]
- Wehbe, L., "Aligning context-based statistical models of language with brain activity during reading", EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference (2014)
- Yang, Zhilin, "XLNet: Generalized Autoregressive Pretraining for Language Understanding", arXiv preprint arXiv:1906.08237 (2019), ~ (2019)
- Vu, M. A. T., "A shared vision for machine learning in neuroscience", Journal of Neuroscience, 38(7), pp.1601~1607 (2018)
- Wang, Alex, "GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding", EMNLP 2018, 2018, pp.353~ (2018)
- Wang, Alex, "Superglue: A stickier benchmark for general-purpose language understanding systems", Advances in Neural Information Processing Systems, pp.3261~3275 (2019)
