Friday, July 24, 2009

Maps and streams in auditory cortex -- continued, take 3

Here is another statement by Rauschecker and Scott:

3. "The postero-dorsal stream interfaces with premotor areas and pivots around inferior parietal cortex, where a quick sketch of sensory event information is compared with a predictive efference copy" p. 722, figure caption.

The problem I have with this statement is not the predictive efference copy idea (we, particularly David, along with others have proposed the same previously), but with the idea that the "inferior parietal cortex" is the pivot. I'm using their figure 5 to judge the location they have in mind for this region and based on evidence localizing sensory-motor processes in speech, it is much too dorsal. In fact, I would argue that the critical region is in the vicinity of the planum temporale/parietal operculum. This is where area Spt lives, which is an area that seems to support sensory-motor functions of the sort referred to by R&S. The inferior parietal cortex area depicted by R&S more closely corresponds to a possible target of Rauchecker's spatial "where" processing stream. There is no evidence that I am aware of that links speech-related sensory-motor processes to this location.

Friday, July 17, 2009

Maps and streams in auditory cortex -- continued...

As promised, here is some continued discussion of some of the claims made by Rauschecker & Scott in their recent Nature Neuroscience paper. Click here for the original entry.

A second claim R&S make is this:

2. "This expanded scheme [their model] closes the loop between speech perception and production and proposes a common computational structure for space processing and speech control in the postero-dorsal auditory stream" p. 722, figure caption.

Just to set the record straight, this perception-production loop has been closed since at least 1874 when Wernicke proposed his own dual stream model in which one stream links auditory speech representations with (widely distributed!) conceptual systems, a "ventral stream", and another links auditory speech representations with motor speech representations, a "dorsal stream". Of course David and I have elaborated extensively on Wernicke's proposal with our version of a dual stream model. So there's nothing really new here in R&S's proposal.

What is different than our (HP) version of a dual stream model is the second part of the statement above tying both auditory space and sensory-motor functions to a common computation network. We have suggested that spatial and sensory-motor functions are subserved by distinct systems within the planum temporale region. The only published data directly relevant to this question comes from an examination of the coordinates of activations associated with spatial hearing tasks versus sensory-motor tasks. Data from some of my own published studies suggests that distinct areas are involved. Here are coordinates for activations associated with spatial hearing (-54, -26, 11) (Smith et al., 2007) compared with those for sensory-motor processes for speech (-51, -46, 16)(Hickok et al., 2003). Note that the foci are 2cm apart in the y direction. More directly, I can tell you that we have now completed a within subject fMRI study that shows distinct activation foci for space vs. sensory-motor processes in the planum temporal. This work has been presented at conferences but has not yet appeared in print.

This, together with anatomical facts regarding the regions involved (see subsequent posts) indicate that there is not a common computational mechanism for spatial and sensory-motor functions of the "dorsal stream".

Hickok, G., Buchsbaum, B., Humphries, C., & Muftuler, T. (2003). Auditory-motor interaction revealed by fMRI: Speech, music, and working memory in area Spt. Journal of Cognitive Neuroscience, 15, 673-682.

Smith, K. R., Saberi, K., & Hickok, G. (2007). An event-related fMRI study of auditory motion perception: no evidence for a specialized cortical system. Brain Res, 1150, 94-99.

More from Pulvermuller on functional anatomic models of language

In case Talking Brains readers missed it, Friedemann has posted a comment on one of my previous posts. Check it out here, and scroll down to see Friedemann's comment:

In that comment he has a link to a more detailed response which you can find here.

Now that we have an active dialog going, I think we can make some progress on convergence of ideas. Thanks much Friedemann! I'll post a response soon.


Thursday, July 16, 2009

Somatotopic Semantics -- Comments from David Kemmerer

David Kemmerer (Purdue), who has been doing a lot of work on the issue of semantic representation of action, sent me a very thoughtful note on the topic in response to the recent discussion here about "somatotopic semantics". He has graciously allowed me to post his substantive points here. He's working on a chapter that will detail his analysis of the literature, including a new, up-to-date figure generated by Javier Gonzalez Castillo that summarizes the relation between motor cortex and action word processing (contact David K. directly if you'd like a preview). This will no doubt be required reading for those interested in this issue.


From David Kemmerer:

Hi Greg -

Interesting discussions on the blog lately. Very briefly, some of my current thoughts of the controversial issues are as follows.

1. I believe localizers are important, and if you look at my chapter, you'll see that several recent studies have used them and have found overlap with corresponding action verbs/sentences. But of course not all studies have found such overlap -- e.g. Postle et al. (2008). So the question is: Why the inconsistencies?

2. I've been reading Graziano's book about the motor cortex, and it's making me realize how shaky many common assumptions are. For example, somatotopy is overrated; other constraints also help shape the organization of the motor cortex. Also, the boundary between primary and premotor cortex is anything but clear; it may not even exist in any meaningful functional sense.

3. You raised an important issue when you pointed out, following Pinker (1989), that many verbs, like "pour" and "fill," are compatible with a huge range of real-world scenarios. But this doesn't imply that the meanings of all action verbs are so abstract or schematic that they couldn't possibly depend in part on motor representations. Some verbs are pretty specific - e.g., as I point out in my chapter, the class of Running verbs in English has around 125 members, many of which are distinguished from each other in rather fine-grained ways. And even for verbs with more general referential scope, monosemy is probably very difficult if not impossible to maintain (see Charles Ruhl's old book "Monosemy" for some interesting insights), and there are of course prototype- and exemplar-based approaches to characterizing their meanings. I could go on and on about this, drawing upon the substantial linguistics literature that's largely neglected by neuroscientists. But for now I'll just say that I think it's too dogmatic and simplistic to assert, especially at this early stage in the game, that it's in principle implausible that verb meanings could be partially implemented in the motor system. The challenge will be to tease apart, on the one hand, the language-specific semantic representations that reside in long-term memory (what Barsalou would call simulators), and on the other, the contextually influenced instantiations of those representations that occur in our brains during on-the-fly language processing (what Barsalou would call simulations). Basically, this is the type/token distinction.

4. I have a strong suspicion that task effects are going to end up playing a big role in all of these debates. After all, there doesn't seem to be a gold standard for determining what does and does not constitute language comprehension. It's a mercurial rather than a monolithic phenomenon. The contrast between automatic and controlled processing is probably relevant, but people seem to have philosophical differences about this. For example, I know from personal communications with Barsalou that he thinks the automatic/controlled distinction is sort of a red herring. As a result, he's not really bothered by questions about whether language-induced "imagery" falls inside or outside the realm of comprehension. Certainly other people would disagree. I'm not yet ready take a firm stand on the issue.

That's it for now. I've been thinking a lot about this stuff lately, but I'm afraid I don't have time to elaborate all of my ideas in detail here. I do hope to contribute now and then to the literature, though. And I enjoy following your blog.


Wednesday, July 15, 2009

Discussion with Friedemann Pulvermuller

In response to a recent blog entry and subsequent discussion of a paper by Friedemann Pulvermuller, Friedemann himself has joined the discussion to clarify a few issues. His commentary is important enough to move it out of the "basement" so I have copied it here and provide my own response below. I hope we can continue this discussion in the comments...

From Pulvermuller:


You miss the point of our HBM paper. Our study for the first time shows using an obective data driven method that the semantic circuits, the semantic cell assemblies if you wish, are distributed - and even more distributed than many of us dared to think before.

The Hickok Poeppel model does not account for such distributed and category specific semantic cirucits. Maybe you should change it?

BASIC: If the voxel of *maximal activation* is slightly besides your prediction - what does this mean in the face of really huge activation clouds? You must show that the could DOES NOT REACH INTO the premotor and motor cortex. Look at David Kemmerer's recent plot: Indeed some find somatotopic semantic activation outside the predicted fields - but most studies show good corresponence between motor areas and action/body-semantics.

BTW, as previous contributors have emphasied, we have used motor localiser and the finger movement area. Hand word activations do indeed overlap. The mapping to standardized motor maps is indeed irrelevant here.


Hickok response:

Hi Friedemann,

Great to hear from you! I hope you will continue posting here as we will all benefit from clarification of theoretical positions, details of the data, etc.

First of all, contrary to what you claim, the Hickok-Poeppel model is not a model of conceptual semantics. Our theoretical coverage stops at the interface between acoustic-phonological systems and conceptual systems.

Second, despite the fact that our model doesn’t deal with conceptual semantics directly, we have consistently and repeatedly stated our assumption (without defense and deferring to people like you who do research in this area) that conceptual systems are widely distributed.

Here are some quotes from our papers (emphases added):

“This pathway appears to be important for interfacing sound-based representations of speech with widely distributed conceptual representations” (Hickok & Poeppel, 2000, p. 131)

“The multimodal cortical fields in the vicinity of the left temporal-parietal-occipital junction are a good candidate for networks important for interfacing sound-based representations in auditory cortex with widely distributed conceptual–semantic representations” (Hickok & Poeppel, 2000, p. 134)

“These pITL structures serve as an interface between sound-based representations of speech in STG and widely distributed conceptual representations” (Hickok & Poeppel, 2004, p. 72)

“…semantic information is represented in a highly distributed fashion throughout the cortex, and middle posterior temporal regions are involved in the mapping between phonological representations in the STS and widely distributed semantic representations” (Hickok & Poeppel, 2007, p. 398)

I don’t know how to state it any more clearly.

It puzzles me why you believe the Hickok-Poeppel model assumes a non-distributed conceptual system. In the same way, it puzzles me why you believe the classical neurologists such as Wernicke & Lichtheim assume non-distributed conceptual system (see previous blog entries). These models certainly don’t need to be changed to account for your data. Maybe you could explain where your misconceptions are coming from?

Now, quite aside from the Hickok-Poeppel model, let’s talk about how conceptual information is represented. As I’ve stated in print, I believe in widely distributed systems that include both sensory- and motor-related regions. My guess, however, is that these are higher-level systems and not, as you propose for action concepts, low level motor representations (e.g., in M1). My belief is based primarily on the many-to-many mappings between low-level action coding and action concepts, and I have not yet seen any experimental evidence convincing me that this believe is incorrect including your TMS, imaging, or lesion-based studies.

Greig’s comment on your recent paper is relevant in that your general claim is not just that conceptual representations are distributed (an old idea that is widely accepted) but that it involves low-levels of the motor system in a somatotopic fashion. To make such a claim for a given result one either has to show that the action concept processing activations are in motor cortex (and imprecise method as you point out) or show direct overlap between motor actions and conceptual processing. Your recent paper seems to have done neither (I admit I haven’t yet read it, so correct me if I’m wrong).

The Irvine Phonotactic Online Dictionary (iPhod)

The Irvine Phonotactic Online Dictionary or iPhod was developed in the Hickok Lab by my (now former) grad student Kenny Vaden. iPhod provides word frequency, phonotactic probability, neighborhood density, etc. values for a large number of English words, as well as measurements for nonwords. The dictionary is publicly available for research use either by downloading it or simply using the online search that Kenny has recently set up. Check it out at:

Kenny has also set up an iPhod blog to provide a forum for questions and future development of the database.

Here is Kenny's more detailed description of what iPhod does:

The Irvine Phonotactic Online Dictionary (iPhOD) is a resource that was developed at UC Irvine in 2003 for research on phonological processing of words and pseudowords. The database can be used for word and pseudoword selection, in order to control or manipulate sublexical or lexical phonological aspects of stimuli. The IPhOD contains 33,432 words and 815,066 pseudowords with Kucera-Francis word frequencies (1967), CMU Pronouncing Dictionary transcriptions (Weide, 1994), and several values that we derived: phonological neighborhood density, positional probabilities, and second- and third-order phoneme-sequence probabilities. The database is publicly available online to search or download, so other researchers may use it in their studies. If a word or pseudoword is not included in the database, some IPhOD values can be calculated online using input phonological transcriptions. On the website, we describe the motivation for the database, the computations used, and examples of their use in experiments concerned with phonological processes in speech. There is also a blog so users can give us feedback, ask questions, and make suggestions for other interesting phonological measures.

Tuesday, July 14, 2009

Maps and streams in auditory cortex: the Rauschecker & Scott perspective

A recent Nature Neuroscience review paper by Josef Rauschecker and Sophie Scott is worth a look. On my reading, the paper is an attempt to reconcile or integrate Rauschecker's Dual Stream model, a 'what'/'where' model, with the version of a Dual Stream model that we (Hickok & Poeppel, 2000, 2004, 2007) and others (e.g., Richard Wise, Sophie Scott, et al. 2001) have been promoting, namely a 'what'/sensory-motor model. There's lots of good information and ideas in the paper (many of them previously elaborated) and it is good to see that Rauschecker is now agreeing with us that the dorsal stream supports sensory-motor integration. However, the effort to reconcile/integrate the 'where' and sensory-motor functions of the dorsal stream fall short in my view as do some of the more general claims by these authors.

Here are some of the claims made by R&S that I believe are not supported by existing evidence:

1. "Speech perception and production are left-lateralized in the human brain" p. 720

2. "This expanded scheme [their model] closes the loop between speech perception and production and proposes a common computational structure for space processing and speech control in the postero-dorsal auditory stream" p. 722, figure caption.

3. "The postero-dorsal stream interfaces with premotor areas and pivots around inferior parietal cortex, where a quick sketch of sensory event information is compared with a predictive efference copy" p. 722, figure caption.

4. "The postero-medial planum temporale area ... is an auditory area important in the act of articulation" p. 721. [emphasis added to indicate where the statement is incorrect]

So let's take these in turn, one post at a time.

1. "Speech perception and production are left-lateralized in the human brain"

I won't argue with the claim that speech production is left-lateralized, but I will argue with the speech perception claim. Evidence from chronic lesion, acute stroke, Wada, and split brain studies have shown that damage to the left hemisphere produces only relatively mild effects on the perception of speech sounds (see Hickok et al. 2008 & Rogalsky et al. 2008 for recent data and discussion). Here, for example, is the brain lesion in a recent case we (C. Rogalsky, T. Love, S. Andersen, H. Damasio, G. Hickok) had the opportunity to study (watch for our forthcoming paper which describes several cases).

This patient performed at 100% accuracy in a four alternative forced choice (4AFC) word-to-picture matching comprehension test where all foils where phonological and at least one differed from the target by only one feature. Performance on a discrimination (same-different judgment) task involving pairs of words that differed by only one feature (e.g., goat-boat) was > 95% correct and non-word discrimination performed at 88% correct. This level of performance is not expected in a patient with no left superior temporal lobe given R&S's claim for left-lateralization.


Rauschecker, J., & Scott, S. (2009). Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing Nature Neuroscience, 12 (6), 718-724 DOI: 10.1038/nn.2331

Hickok, G., Okada, K., Barr, W., Pa, J., Rogalsky, C., Donnelly, K., Barde, L., & Grant, A. (2008). Bilateral capacity for speech sound processing in auditory comprehension: evidence from Wada procedures. Brain Lang, 107(3), 179-184.

Hickok, G., & Poeppel, D. (2000). Towards a functional neuroanatomy of speech perception. Trends in Cognitive Sciences, 4, 131-138.

Hickok, G., & Poeppel, D. (2004). Dorsal and ventral streams: A framework for understanding aspects of the functional anatomy of language. Cognition, 92, 67-99.

Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nat Rev Neurosci, 8(5), 393-402.

Rogalsky, C., Pitz, E., Hillis, A. E., & Hickok, G. (2008). Auditory word comprehension impairment in acute stroke: relative contribution of phonemic versus semantic factors. Brain Lang, 107(2), 167-169.

Wise, R. J. S., Scott, S. K., Blank, S. C., Mummery, C. J., Murphy, K., & Warburton, E. A. (2001). Separate neural sub-systems within "Wernicke's area". Brain, 124, 83-95.

Friday, July 10, 2009

Somatotopic organization of action word meanings - Again!

Some interesting commentary on motor somatotopy (or lack thereof) from Greig de Zubicaray's website which I've copied and pasted here (with permission of course):

We're very interested in claims of a so-called "semantic somatotopy" in primary and premotor cortex (i.e., overlapping activity for executing/observing actions performed with a specific effector and retrieving word meanings related to that effector). See Tash Postle's paper in NeuroImage last year in which the peak maxima from neuroimaging studies cited as evidence for this claim were plotted against cytoarchitectonic maps of BA4 and BA6 in the language dominant left hemisphere. Friedemann Pulvermüller and colleagues have a new fMRI paper using cluster analysis in press in Human Brain Mapping in which they conclude that "The motor-premotor region was dominated by semantic somatotopy... with face, arm, and leg word activation foci at inferior lateral (-49/11/16), dorsolateral (-32/-5/52), and dorsal sites (-19/-21/61)." So, here are 10 mm spheres centred on those peak maxima plotted in relation to the cytoarchitectonic maps of BA4 (red) and BA6 (blue) from Eickhoff et al. (2006). Again, not much congruency with respect to the actual cortical motor areas, and as motor execution and/or observation conditions weren't included in the study, we have no evidence that these clusters are representative of the actual neuronal ensembles used by the participants to perform effector movements. Interestingly, though, the leg peak is in BA6 this time.


Pulvermüller, F., Kherif, F., Hauk, O., Mohr, B., & Nimmo-Smith, I. (in press). Distributed cell assemblies for general lexical and category-specific semantic processing as revealed by fMRI cluster analysis, Human Brain Mapping.

Eickhoff, S.B., Heim, S., Zilles, K., & Amunts, K., (2006). Testing anatomically specified hypotheses in functional imaging using cytoarchitectonic maps. NeuroImage, 32, 570–582.