Monday, June 16, 2008

Noun Meaning: An Interesting But Peculiar Paper in Science

In the May 30 issue of Science there is a paper by Mitchell et al. with the provocative title, 'Predicting Human Brain Activity Associated with the Meanings of Nouns.' That is quite a promise. I think this paper shows a really innovative way to look at fMRI data; I also think that it raises more questions than it answers with respect to the meaning of nouns.

The paper presents a computational model that is trained using a trillion-word text corpus and observed fMRI data, and that is then used to predict the pattern of fMRI responses for new words. The model is based on the following: "First, it assumes the semantic features that distinguish the meanings of arbitrary concrete nouns are reflected in the statistics of their use within a very large text corpus. ... Second, it assumes that the brain activity observed when thinking about any concrete noun can be derived as a weighted linear sum of contributions from each of its semantic features."

I think this computational modeling idea is really quite cool, but I am also puzzled by the presuppositions. For example, I just don't understand what 'meaning' means when it seems to simply follow from co-occurrence statistics. The intuition appears to be that you can find 'intermediate semantic features,' but when you read the paper, those intermediate semantic features themselves must be found in the corpus. Here they are simply given to the model, but on a co-occurrence statistics view of the world, doesn't the learner have to actually get this information? Anyway, I find this fascinating and would like to learn more, but it does make specific assumptions about how meaning comes about that require more explanation, at least for someone like me. Furthermore, the second assumption, that a weighted linear sum of the contribution of semantic features constitutes the meaning, is no less puzzling.

The type of material that seems to be captured pretty well in this approach are concrete nouns as well as verbs with very obvious sensory-motor features. And indeed, the authors argue that their model "lends credence to the conjecture that neural representations of concrete nouns are in part grounded in sensory-motor features." Okay, so here they're telling us something quite specific, in line perhaps with the way Alex Martin would think about the problem. But right after that they say that "it appears that the basis set of features that underlie neural representations of concrete nouns involves much more than sensory-motor cortical regions." Well now it sounds like they want to have their cake and eat it too. And who doesn't?!

The experiment strikes me as a very creative way to interrogate fMRI data, but I think the central question remains really hard to tackle. "What is the basis set of semantic features and corresponding components of neural activations encoding meanings of concrete nouns?"

Let's say this theory is exactly right. That is, corpus statistics + sensory-motor features are a sufficient description for concrete nouns and action verbs (a conclusion that I think is pretty dodgy). Does that mean that to extend this to other aspects of meaning we need an entirely different theory? For example, how are we going to deal with a) abstraction, b) closed-class words, and c) any kind of compositionality? I do think it's inspiring that hard-core computational methods are allowing us to look at these complex data sets with new 'glasses.' But I still think that the cognitive science part, and in this case the semantics, need to be very carefully looked at in the context of interpreting such provocative data sets.

I hope either Greg or any reader or commenter can enlighten or teach me more about this intriguing new paper.

1 comment:

Anonymous said...

I just heard about this (as well as about your blog) recently. They have done a related study in which an fMRI machine, along with a machine learning algorithm, was able to "guess" the words based on the 12 most stably activated individual voxels in the brain. It was quite cool, except a lot of the voxels were sort of all over the brain and didn't end up telling much about language itself.