Friday, September 23, 2011

Mirror Neuron Forum - Role of mirror neurons in speech and language processing - Part II

Marco Iacoboni's response to my argument that the motor system is not necessary for speech perception was short and sweet, so let's break it down line by line.
In a ‘‘virtual lesion’’ repetitive TMS (rTMS) study on speech perception, the TMS effects over premotor cortex were, if anything, a little stronger than the TMS effects over the auditory cortex (Meister et al., 2007). However, the effects were not reliably different, suggesting that both structures participated in the functional process, in contrast to GH’s suggestion that motor processes play a small, modulatory role in speech perception.
Meister et al. found that TMS to premotor cortex resulted in a modest decline in performance in identifying synthesized CV syllables presented in noise in the context of a three-alternative forced choice paradigm. There has been no study that I'm aware of to show that such an effect is found when natural stimuli are used. The stimuli have to be degraded, i.e., partially ambiguous. Can we conclude that premotor cortex is playing an "essential role in speech perception" as the title suggests? No, we can only conclude that it is playing a modest role in the performance of an artificial task under degraded listing conditions. And we can't even tell what aspect of the task is being disrupted. It is possible that TMS is not interfering with the perception at all but rather interfering with the sensory-motor memory of which response button corresponds to which syllable. This one piece of evidence is held up to counter the array of studies that I cited showing that damage to the motor speech system, developmental failure of the motor speech system, complete biological lack of the capacity for a motor speech system, does not prevent speech perception. Where does the weight of the evidence leave us? The motor system plays a modest modulatory role if that. Why didn't STG stimulation cause a greater decline in performance? There is abundant evidence that speech perception is bilaterally mediated in the STG (Hickok & Poeppel, 2000, 2004, 2007).
Again, I find it counterproductive to focus on dichotomous models (‘‘it’s auditory,’’ ‘‘no, it’s motor’’). These models, although didactically useful, tend to provide a limited understanding of the functional processes at play. Indeed, consistent with the model in GH’s Figure 2D, the most successful recent computational models of action and perception disclose the intimate relationship between motor control and perception (Friston, Daunizeau, Kilner, & Kiebel, 2010; Friston, Mattout, & Kilner, 2011).
I outlined four possible models, only two of which were dichotomous. I'm not denying that action and perception are intimately related. They are! But the functional relation is precisely the reverse to what the mirror neuron claim holds.
Eventually, we will have to get rid of these labels altogether, because they seem to get in the way of a better understanding of the phenomena under investigation.
Call it what you like, it doesn't change the fact that systems in the posterior frontal lobe aren't necessary for speech perception, whereas bilateral systems in the superior temporal lobe are. As much as some folks would like the cortex to one big happy interacting neural network with no differentiation, the fact is that damage to different parts of the system have different effects. We have to deal with these facts. Returning to the facts, here's a quote from Meister et al.
The present results demonstrate that the involvement of the premotor cortex in perception is not merely epiphenomenal and suggest that sensory regions are not sufficient alone for human perception. p. 1695
and a figure from Rogalsky et al. 2011 which shows comprehension, word discrimination, and syllable discrimination performance of two cases with lesions involving the human mirror system.
It seems pretty clear that Meister et al.'s claim is false. The recent follow up to Rogalsky et al. using a sample of 24 cases with Broca's area lesions confirms what was found in these two cases.

So, I've covered the response to my criticisms of mirror neuron theory by two of the most prominent and thoughtful defenders of the theory. Given the opportunity to present their strongest possible rebuttal to direct critiques in the Mirror Neuron Forum, both Gallese and Iacoboni failed to mount a viable defense of their model. This, of course, is my view. I'm sure they will disagree and again I invite them to post their own comments as guest entries on this blog. So far I have not heard a peep from either of them despite direct email invitations to participate.


Gallese, V., Gernsbacher, M., Heyes, C., Hickok, G., & Iacoboni, M. (2011). Mirror Neuron Forum Perspectives on Psychological Science, 6 (4), 369-407 DOI: 10.1177/1745691611413392

Hickok, G., & Poeppel, D. (2000). Towards a functional neuroanatomy of speech perception. Trends in Cognitive Sciences, 4, 131-138.

Hickok, G., & Poeppel, D. (2004). Dorsal and ventral streams: A framework for understanding aspects of the functional anatomy of language. Cognition, 92, 67-99.

Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nature Reviews Neuroscience, 8(5), 393-402.

Meister, I. G., Wilson, S. M., Deblieck, C., Wu, A. D., & Iacoboni, M. (2007). The essential role of premotor cortex in speech perception. Curr Biol, 17(19), 1692-1696.

Rogalsky, C., Love, T., Driscoll, D., Anderson, S. W., & Hickok, G. (2011). Are mirror neurons the basis of speech perception? Evidence from five cases with damage to the purported human mirror system. Neurocase, 17(2), 178-187


James Kilner said...

Hi Greg,

I have found the Mirror Neuron Forum piece very interesting from a number of different reasons. I guess the questions that I still think need are addressing are

1) Do mirror neurons have a functional role?

2) If so what is it?

In my opinion the problem with high order functions like "action understanding" is that I would be suprised if there was a unique pathway that underlied this level of inference (see KIlner (2011) TICs. In other words it does not suprise me that the lesion studies to date have not shown significant effects. Given that mirror neurons do not seem to be involved in these levels during motor execution it suprises me that people assume that they are in action observation

Greg Hickok said...

1) Yes, mirror neurons have a functional role.

2) Rizzolatti and flirted with and then rejected the correct idea in favor of the action understanding hypothesis.

Basically, the correct idea, in my opinion, is the reverse of the action understanding claim: mirror neurons are part of the system that maps action goals (coded in sensory systems) onto motor solutions for achieving those goals.

Read all about it here:

Hickok, G., & Hauser, M. (2010). (Mis)understanding mirror neurons. Curr Biol, 20(14), R593-594. doi: S0960-9822(10)00650-0 [pii]

James Kilner said...

So according to this functional role what deficits should patients with lesions to mirror neuron areas have during action observation?

VilemKodytek said...

1) Most of the brain is about mappings something somewhere. So mirror neurons everywhere?

2) By "understanding" Gallese & comp presumably mean f-understanding, i.e. functional, not conscious). Is there a difference between "recognition" and "f-understanding".

Greg Hickok said...

JK said, "So according to this functional role what deficits should patients with lesions to mirror neuron areas have during action observation?"

I'd say minimal to no deficits, if you select the right task. If you select a task that involves the motor system (e.g., as some forms of working memory do) then you've set up the experiment to find a role for the motor system.

Where you see the real deficits with damage to this system is in action execution and there is plenty of evidence for this.

Greg Hickok said...

Mirror neurons everywhere? As I've pointed out before, the proliferation of mirror neurons into every brain system is actually a problem for the theory because the cells lose their explanatory power. In the extreme, if every cell is a mirror neuron, then, as Pat Churchland said, the problem reduces to the "how does the brain work" problem, and we are back to square one.

As far as I can tell Gallese and company never really define what they mean by "understanding". The current term they use is "understanding from the inside", which is far from a precise scientific concept.

VilemKodytek said...

Aha! It's "Mirror Neurons - The unfalsifiable theory", March, 19, 2010. Thank you.

James Kilner said...

So would it be fair to say that if one could find a behavioural deficit when observing an action in patients with damage to mirror neuron areas this would be evidence against the functional role your proposed in HIckok and Hauser (2010)? I ask as one problem with this field is trying to find a test that disambiguates between alternative hypothesises.

VilemKodytek said...

James, I like your 2011 paper. But how does prediction work in cases we can’t simulate, such as Greg’s favourite coiling and flying?

As to your last question above, deficit refers to a benchmark, e.g. an average of normals at hand. Having a deficit doesn’t necessarily mean being outside the limits of human performance. So I guess your criterion wouldn’t be fair.

James Kilner said...

To answer the question how can we predict actions that we can not simulate I would argue that we can always simulate an action. However, the simulation we produce might be an incredibly bad prediction of the action we observe. However if we know that this is a bad prediction we can accomodate our uncertainty in our prediction by changing the prior precision of our prior prediction. One consequence of this is that we reduce prediction error and thus our perception of the action is not as precise. This idea is shown in Neal and KIlner (2010) What is simulated in the action observation network when we observe actions? EJN.

So if we take the example of flying. Humans can not fly. However if you ask humans to simulate flying all humans will flap their arms. Why? I would argue that this is because it is the best motor simulation we have of the motor commands required for flying. Clearly it is rubbish. But it can still function as a prediction.

Greg Hickok said...

James - why the obsession with motor prediction? Don't you think it is possible to make predictions in other ways? Do you think a baseball outfielder chasing down a fly ball is simulating the flight of the ball with his motor system? How would you do that? And wouldn't using motor simulation interfere with the athlete's own motor actions required to perform the task (catching the ball), which are different that what might be required for simulation? It seems to me that predictions regarding a baseball's flight are made via past sensory experience the trajectory of flying baseballs, not by trying to simulate it with the motor system. And if you predict the flight of a baseball without simulating it in your motor system, why can't you do it for the flight of a bird?

James Kilner said...

Yes I certainly believe that we make predictions in other ways. The baseball is a good example. I would argue that we predict the flight of the ball not using our motor system at all. The flight of the ball can be predicted by newtonian equations of motion that we can generalise across all objects. I believe we learn these during development. I am agnostic to the process by which they are learned and this could well include associative learning. This process of perceptual learning differs from perceptual inference. However the flight of a bird or person or animal can not be predicted by newtonian equations of motion. For me the most generalizable form of prediction of observed biological actions would be to use our own motor systems to predict the actions. Critically though we do not always have to have a correct prediction of actions just our best prediction. Just to restate that I certainly do not believe that all predictions are made by the motor system. Indeed so far I have only suggested that we use this for predictions of other humans, but I think we could use this for other animals although i have not tested it.

Greg Hickok said...

Interesting discussion. Does my dog compute a bad motor system simulation of a throwing action when he predicts the direction of the ball throw based on my movements? See the video:

Dogs can't execute throwing motions. Is this an example, then, of sensory association rather than motor simulation? I think so. He learns to predict throw direction based on past experience associating a particular movement and the resulting direction of the ball throw.

So if my dog using sensory learning to make predictions about the consequences of actions, humans must be able to do it as well. Given that there is an existing mechanism for action understanding that doesn't involved motor simulation, why do we need a second one?

Ok, maybe you will say motor simulation allows for better predictions than sensory learning based prediction. Doesn't such a mechanism predict the following counterintuitive result? A baseball batter who is really good at predicting the velocity and path of the pitcher's throw should actually be worse at hitting the ball because of interference between the pitching simulation needed for prediction and the actual swinging action needed for hitting the ball.

James Kilner said...

The simple answer to your question is yes. I think that the dog example is an example of learned associations and I think that humans do the same thing. However, I do not believe that mirror neurons are required for this process at all. I think there is a difference in between learning a sequence of events and learning the sensory consequences of an observed action. In other words mirror neurons would enable the prediction of the observed action - the throw. I do not believe dogs encode this. The relationship between this action and the ball trajectory is a sequence of events that could be encoded outside of mirror neuron areas. I think that these different predictions might be mediated by different neuronal pathways - something I mention in the TICs article.

In terms of interference then there is evidence of interference of an observed action on a simultaneously executed action Kilner et al. 2003. The key thing here is the observed and executed actions must be simultaneous. As far as I can tell this is not true of the baseball example you gave. In football (soccer) you see this often with spectators or the manager visibly trying to head the ball when watching their player heading the ball towards the goal.

Greg Hickok said...

It takes about 400 msec for a fastball to reach home plate. Maybe that's enough time to shut down the motor simulation and program the swing, but I kind of doubt it. Maybe a boxer is a better example? Simulation of your opponent's actions would seem to be fatal for a boxer, no? In this case you need to generate a non-mirror movement pretty much simultaneously with the perception of movement. Even if you could simulate just the beginning of your opponent's punch then switch to an evasive response, a boxer who could bypass the simulation and avoid interfering with his own evasive movements would have an advantage, I would think. Translate this thought to the animal world, a prey animal that has to simulate the predator's movements to predict them would seem to be at an evolutionary disadvantage. So what's the motivation for developing a simulation system to support understanding when you can do it more straightforwardly another way?

How is a sequence of body positions (the different positions of an arm during a throw) different from a sequence of events? Are you suggesting that a dog can't learn to predict the arm's trajectory in a hitting action?

VilemKodytek said...

Excellent discussion. Thank you!

There are two famous slogans: “Neurons that fire together wire together” and “Correlation does not imply causation”. I am still not able to choose one and reject the other. In fact I am beginning to believe that both are valid. Here is why:

If someone’s playing violin, I can enjoy but my brain can’t simulate the action of playing – contrary to that of a violinist in the audience. My position is equivalent to that of Greg’s dog. Now what bothers me is this: having listened to Dvorak’s From the New World many times, I may think I have a detailed knowledge of the symphony, can notice differences in particular arrangements, and yet my motor system doesn’t understand a bit the actions of the musicians. But what about a violinist, a trumpeter, and a conductor in the audience?

James Kilner said...

There are a number of interesting questions and I am happy to share my thoughts in an attempt to answer them. The first can be summarised as isn't it maladaptive to simulate and action we observe if we need to execute a different action? In the examples Greg gave, the boxer and the prey, both are examples where the observer must identify action X and then execute action Y. In these examples there is a sequence if observed action = X then execute Y. What I am interest in is how we recognize action X from lots of possible actions so that we can execute Y as rapidly as possible. In this field of recognition one widely held idea is that this process is more accurate, faster and more robust if we use generative models to predict the input, in this case the observed action X. Rather than processing the visual input then decoding then inferring it is action X we do the opposite. So the boxer might 'think' I have to watch out for the big right hook. So in his mind he generates a prediction of what this might look like. When the visual information matches this - this might be the an initial roll back of the right shoulder prior to the punch - the observer can execute action Y. We know that prior information influences the observers behaviour as in sport people often disguise/fake any action i.e make one action look like another.

The second question is why isn't a single action a sequence of events. Computationally it can be thought of as a sequence after all this is precisely what a film is. However I think that this is not the most efficient way to encode the information. If we did this we would have to encode every possible sequence for every single action! It would be more efficient/plausible to encode a few parameters that encode the entire dynamics. This is clear if we take the example of the baseball flying through the air. Knowing just a few parameters - ball weight, trajectory and force applied to the ball, and gravitational pull we can generate an accurate prediction of the flight of ball. In other words we do not have to encode the flight of the ball as a sequence of positions of the ball in time. I think the same is true for an action. Using our own motor sytem we can parameterise observed actions.

Greg Hickok said...

Again, why the obsession with the motor system? Prediction - yes. The concept of neural prediction is certainly in vogue at the moment, and might even be accurate. By why does it have to come from the motor system? If you admit that we can make sensory (or cognitive) predictions, and that we can do this even for actions, what is driving you to claim that you need to simulate a movement in order to make a prediction?

I like the idea of parameterizing perception of dynamic objects. You suggest that if we know a few things about how balls fly through the air, we can make very accurate predictions. You then suggest we parameterize action perception using our own motor system. Why can't you just learn a few things about how bodies move -- hands tend to move with arms, arms don't bend backwards, etc. -- then use these parameters just like ball weight, trajectory, and speed? I bet you can make predictions about the a flamingo's leg movements when it is walking even though your own knees don't bend backwards.

I'm not saying that in principle the motor system can't be a source of prediction in perceiving others' actions, I'm just asking why many investigators are not even considering the possibility that prediction can come from outside the motor system. Don't we want to consider all possibilities in order to have a better chance of really understanding how the system works?

VilemKodytek said...

Great stuff! I deleted my own comments and printed it out.

James Kilner said...

Yes I think it is possible that observed actions could be parameterised in different ways as you suggest. Personally, I have studied the role of the motor system in this parameterisation for two reasons. 1) the motor system is active during action observation (we don't know why) 2) I think that motor system in action observation must encode the same features in action execution and action observation. In action execution neural populations seem to parameterise the action in a different way to that which you suggest. However, alternative ideas are great for science as they allow for proper tests between different ideas. For me one of the problems with the field is that there is not really a testable hypothesis to that which Rizzolatti and co put for nearly 20 years ago. I think there is a lot of evidence that some of the evidence in favour of the Rizzolatti original idea might not be quite right but the field needs an alternative explanation with testable hypothesis. Just my opinion.

Greg Hickok said...

There are at least two alternative hypotheses. One is that it is pure sensory-motor association (Heyes). Seeing your own arm move during motor execution causes the two activities to become associated and therefor wired together. Evidence in favor of this is the Heyes and company result that if you change the association, the neural activity in the motor system changes with it. In fact, it can become anti-mirror.

Heyes, C. (2010). Where do mirror neurons come from? Neurosci Biobehav Rev, 34(4), 575-583.

The other is what I've proposed and is based on two assumptions that we know to be true: 1. the motor system is involved in motor control, and 2. sensory information is critical for motor control. The motor system (e.g., premotor regions, like monkey F5) has cells that respond to object shape because that information is important for action selection and guidance. The motor system also has cells that respond to others' actions because that information is important for action selection and guidance. Monkey's don't imitate, which lead Rizzolatti away from this interpretation of mirror neurons. But observational learning is a behavior that exists in macaque and many other species, involves a kind of mirroring of others' actions, and therefore is likely supported by mirror neurons.

The motor system activates during action observation because, as Rizzolatti and colleagues put it in the very first paper reporting mirror neurons, “the actions performed by other monkeys must be a very important factor in determining action selection." In other words, sensory information is the input that drives actions. It is therefore no surprise that seeing something, including others' actions, will activate the motor system.

I have outlined this alternative view for the behavior of macaque mirror neurons in two publications:

Hickok, G., & Hauser, M. (2010). (Mis)understanding mirror neurons. Curr Biol, 20(14), R593-594.

Gallese, V., Gernsbacher, M. A., Heyes, C., Hickok, G., & Iacoboni, M. (2011). Mirror neuron forum. Perspectives on Psychological Science, 6, 369-407.

I have also discussed this alternative, motor control interpretation in much more depth in relation to speech processing here:

Hickok, G., Houde, J., & Rong, F. (2011). Sensorimotor integration in speech processing: computational basis and neural organization. Neuron, 69(3), 407-422.

Five years ago it was fair to say that the action understanding theory was the only game in town. That is not the case anymore. It is time to re-examine mirror neuron/system function in light of alternative hypotheses.

James Kilner said...

As far as I understand it the Heyes ASL theory is challenging the ontogeny of mirror neurons and not their function. My understanding is that according to this theory mirror neurons might have a functional role but it is not a functional role that has been selected for by evolution. In other words this is a model of perceptual learning and not perceptual inference. (Press, Heyes, Kilner Learning to understand others' actions. Biology Letters 2011).

So if I understand your theory then mirror neurons discharge during observation of an action so as to allow us to appropriately select an action to execute?

Greg Hickok said...

True, I suppose you could interpret Heyes' position as being agnostic on the issue of MN function in action perception.

Yes, just like seeing an object defines the parameters of a possible movement (grasping with a particular grip configuration), seeing an action similarly defines possible action parameters. If the monkey sees another reaching toward some food, one possible action is to compete with the animal/experimenter and try to reach for that same morsel (a mirror response). I would expect that if anyone had the motivation to look, non-mirror responses could also be identified in response to actions. E.g., train the monkey to expect a drop of juice when an experimenter reaches for a food morsel and I bet you'll find F5 cells that respond both to viewing grasping and executing sucking actions, a non-mirror action-action association. No one has the motivation to explore alternative possibilities though, because the action understanding theory is *assumed* at this point. This is bad for scientific progress.

VilemKodytek said...

The experiment in Neal & Kilner 2010 is marvelous!

I believe that their subjects, engaged in decision making, activeta motor system. But is it simulating the observed action? I wonder whether it cannot just close the circuit (no action is needed) or/and compute corrections to what corresponds to the subject’s experience or desire rather than simulate the action.

Aglioti et al., Nat Neurosci 2008, 11, 1109, present MEP data in decision making of basketball players and expert non-players whether an observed action (penalty shot) is correct or not. In the critical point the MEP in the players is significantly higher for an off shot than on shot, while in untrained subjects the response is basically the same in both cases. Of course, MEP is not a measure of the activity in the PMC or even IFG. But I have another argument: Above James said: “In football (soccer) you see this often with spectators or the manager visibly trying to head the ball when watching their player heading the ball towards the goal.” But think of what they are doing when observing a rival’s player heading the ball towards the goal!

Marius Sommer said...

Hello Gentlemen!
I've read this discussino and find it very interesting. however, its been almost 2 years since the last post. What would you say is status quo in this topic today?

Marius Sommer

Greg Hickok said...

James has recently published an interesting paper that I think reflects the direction this discussion is going, at least from the direction of folks who are sympathetic to the idea that mirror neurons have some role to play in action understanding:

Kilner, J. M. (2011). More than one pathway to action understanding. Trends in Cognitive Sciences, 15, 352-357.

Also have a look Michael Arbib's many publications on the issue.

I remain dubious (but open) to the possibility that MNs can play a supportive role in action understanding via some form of predictive coding.