Evaluating Unsupervised Parsing: Do Language Models Truly Capture Syntax?
1
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
This study critically examines the claim that unsupervised parsing algorithms can infer the syntactic structure of natural language without explicit supervision. Specifically, it investigates the Deep Inside-Outside Recursive Autoencoder (DIORA) model, which uses bidirectional LSTMs to purportedly capture sentence structure based solely on raw word order and semantic relationships (Drozdov et al. 2019). Given the inherently hierarchical and recursive nature of linguistic structure(Chomsky 1956; Montague 1970)—a feature often opaque even to native speakers—this research hypothesizes that models like DIORA may prioritize semantic chunking driven by word meanings and frequent word sequences rather than accurately representing syntactic structures. To test this, synthetic sentences of syntactically valid but semantically nonsensical sentences, akin to Chomsky’s “Colorless green ideas sleep furiously,” were developed. By comparing DIORA’s parsing outputs to tree structures represented utilizing manually constructed Chomskyan-style Context-Free Grammar (CFG), this paper elucidates the linguistic representations learned by unsupervised parsing models. The findings provide critical insights into whether such models genuinely capture syntactic structures or primarily rely on semantic patterns, contributing to the broader understanding of unsupervised language parsing.
Abstract
This study critically examines the claim that unsupervised parsing algorithms can infer the syntactic structure of natural language without explicit supervision. Specifically, it investigates the Deep Inside-Outside Recursive Autoencoder (DIORA) model, which uses bidirectional LSTMs to purportedly capture sentence structure based solely on raw word order and semantic relationships (Drozdov et al. 2019). Given the inherently hierarchical and recursive nature of linguistic structure(Chomsky 1956; Montague 1970)—a feature often opaque even to native speakers—this research hypothesizes that models like DIORA may prioritize semantic chunking driven by word meanings and frequent word sequences rather than accurately representing syntactic structures. To test this, synthetic sentences of syntactically valid but semantically nonsensical sentences, akin to Chomsky’s “Colorless green ideas sleep furiously,” were developed. By comparing DIORA’s parsing outputs to tree structures represented utilizing manually constructed Chomskyan-style Context-Free Grammar (CFG), this paper elucidates the linguistic representations learned by unsupervised parsing models. The findings provide critical insights into whether such models genuinely capture syntactic structures or primarily rely on semantic patterns, contributing to the broader understanding of unsupervised language parsing.