Presented at the the 6th Workshop on Cognitive Modeling and Computational Linguistics: Modeling fmri time courses with linguistic structure at various grain sizes. (with John T. Hale, David Lutz, Wen-Ming Luh)
Abstract
Neuroimaging while participants listen to audiobooks provides a rich data source for theories of incremental parsing. We compare nested regression models of these data. These mixed-effects models incorporate lin- guistic predictors at various grain sizes ranging from part-of-speech bigrams, through sur- prisal on context-free treebank grammars, to incremental node counts in trees that are derived by Minimalist Grammars. The fine-grained structures make an independent contribution over and above coarser predic- tors. However, this result only obtains with time courses from anterior temporal lobe (aTL). In analogous time courses from infe- rior frontal gyrus, only n-grams improve upon a non-syntactic baseline. These results sup- port the idea that aTL does combinatoric pro- cessing during naturalistic story comprehen- sion, processing that bears a systematic rela- tionship to linguistic structure.
The figure above (click to enlarge) illustrates the main idea of linking word-by-word parser states (panel c) with estimates of the hemodynamic response measured with fMRI (panels d-e) collected during naturalistic stimulation.