I led a discussion on auditory-motor interactions with the CogHear group today. There were many interesting questions written in the chat window. Here I quote and respond to them. My responses are in italics.
If you are interesting in watching the live discussion, contact Malcom Slaney: malcolm@ieee.org
Matt Leonard
9:20 AM
Is "the motor system" defined here solely based on canonical anatomy? Or does it also have a functional definition?
We look at the question both ways, but with an emphasis on the functional definition.
Liberty Hamilton
9:22 AM
Have you looked at the specific phonological contrasts that are or are not able to be performed in these cases? Which pairs are tested (only initial consonant? also vowel?)
No, we have not.
roberta bianco
9:26 AM
would those patients able to pick on prosodic cues? e.g. discrimintae question and statement : pear vs pear?
Great question and I don’t know the answer.
Sean Gilmore
9:26 AM
What kind of noise was used?
White noise. 14 dB.
Julie Elie
9:26 AM
In case of complete anarthria, people cannot even express non-word prosody sounds (e.g. anger groan)?
Emotional vocalizations (groans, laughs) can be possible as can spontaneous smiles (the so-called Duchenne smile).
Sam Norman-Haignere
9:32 AM
Isn't a 10% effect kind of large for TMS?
Yes, potentially. But note that it’s 10% at threshold level SNRs with contrived tasks only. These effects are not found in normal comprehension. In response to my task critique, this study used a comprehension paradigm and found no effect on error rate. An RT effect was observed but only for some stimuli. So the TMS effect on comprehension appears to amount to a few milliseconds for some contrasts, out of context. It would be good replicate this if this is indeed the magnitude and type of the effect of motor disruption on receptive speech
Schomers, M. R., Kirilina, E., Weigand, A., Bajbouj, M., & Pulvermuller, F. (2014). Causal Influence of Articulatory Motor Cortex on Comprehending Single Spoken Words: TMS Evidence. Cereb Cortex. doi: 10.1093/cercor/bhu274
Jack L. GALLANT
9:34 AM
I agree with the argument that the motor system isn't partocularly importnat for speech perception. When people are passibely listening to speech, a motor (articulatory) model does a really poor job of predicting activity in motor cortex.
I think Eddie Chang and colleagues found a result that is similar in spirit.
Abby Noyce
9:34 AM
Greg, so you’re arguing that the motor contribution is about the working memory demands on these kinds of tasks?
Kind of: a component of what goes into working memory. More generally, I think the motor speech system contributes to the performance of the *task* rather than the perception of the phonemes. It could be that articulatory recoding (Baddeley’s articulatory rehearsal part of the phonological loop) is the bit that matters. But it would seem that if you have a nonword span of at least two items, you should be able to do a discrimination. Our patient had such a span but still struggled with nonword discrimination. One response to this is that span is measured with phonologically dissimilar items thus over estimating the span for similar items like on our discrimination task. Or, it could be that the motor system is the means through which we can allocate conscious attention to the sublexical structure of speech, something we don’t normally do unless an experimenter asks us to or if we are learning to read. (Illiterate people tend to have trouble with discrimination, by the way https://pubmed.ncbi.nlm.nih.gov/23379298/).
Frederic Theunissen
9:35 AM
If the contribution of the motor system is about working memory, you might get the same effect with remembering images?
No, it is not generalized working memory but something more closely related to phonological STM.
Yulia Oganian
9:38 AM
was the opercular lesion patient's working memory impaired
thank you!
Yes, nonword span = 2 items.
Sean Gilmore
9:41 AM
It's not necessarily speech related, but what about the large amount of literature in music cognition showing the involvement of the motor system in auditory perception. Does this implicate the motor system in auditory perception to some degree?
An interesting topic: musicians not only are skilled in listening to music, but also in producing music via their motor system. We can expect more auditory-motor interaction in skilled musicians.
Jack L. GALLANT
9:45 AM
Again, both during speech perception and music perception. activation throughout the motor system is quite low and motor features do not predict well in the motor system.
If there are motor responses during speech perception, one possibility is that this activity is related to action words. A similar thing is seen in vision, where some have reported small motor system activation while people watch movies that involve motor manipulation, but not otherwise.
The action word-motor system association is an interesting, but different question. I addressed it at length in the embodied cognition chapter of my book, The Myth of Mirror Neurons.
Andrew Meier
9:46 AM
Why was it previously thought that arcuate fasciculus damage causes conduction aphasia?
Conduction aphasia appears to be an auditory-motor integration deficit in that word comprehension is intact, production is fluent but error prone with mostly phonological errors. If it were a sensory problem, perception would be impaired. If it were a motor speech problem, production should be non-fluent or at least apraxic (cf, apraxia of speech). The problem seems to be in the ability to use auditory-based networks to guide motor speech planning leading to errors. I talk about this issue here: Hickok, G. (2012). Computational neuroanatomy of speech production. Nature Reviews Neuroscience, 13(2), 135-145. doi: 10.1038/nrn3158
Matt Leonard
9:46 AM
^ another possibility is that "motor areas" have auditory or at least sensorimotor circuits. So I think it's really important here to more specifically define what "motor" means.
I agree, kinda. I think there are “ventral stream” auditory-conceptual networks and “dorsal stream” auditory-motor networks. In this sense, we might do away with the unmodified term “motor” completely. However, it does make sense to parse the auditory-motor network into its component parts where some part play a role as auditory-based targets for motor actions.
Christoph Daube
9:47 AM
How would we fit papers like this into this narrative? https://elifesciences.org/articles/12577
I think it fits once we recognize that motor systems are fundamentally sensorimotor. The motor system is dependent on sensation, so we find sensory responses in motor cortex. The reverse is not true, however: sensory/recognition systems are NOT dependent on motor (frontal) systems.
Liberty Hamilton
9:48 AM
^ Agree with Matt, we know there are projections from the motor cortex to the auditory cortex, I don't know about auditory to motor but we do see responses in motor cortex that are more strongly related to manner of articulation and not place (so acoustic). Would be interesting to hear from birdsong folks about this too, because I know there is work on auditory projections to motor song areas
Jean-Rémi KING
9:47 AM
@jack "motor features do not predict well in the motor system. " => motor features? Would things like rhythms count? Aren't those relatively large responses?
Liberty Jack L. GALLANT
9:48 AM
The SNR of motor cortex is really low during perception so the noise ceiling is very low and all the models fit poorly. We usually use timing models and articulatory models
Matt Leonard
9:48 AM
@Christoph @jack - that's exactly what I'm thinking of. We also see related effects in much of our other data (https://pubmed.ncbi.nlm.nih.gov/29958109/)
Jean-Rémi KING
9:50 AM
@greg hickok: did you look at other frequency bands for this planum suppression during speech?
Yes. Have a look at the paper for details. Forseth, K. J., Hickok, G., Rollo, P. S., & Tandon, N. (2020). Language prediction mechanisms in human auditory cortex. Nat Commun, 11(1), 5240. doi: 10.1038/s41467-020-19010-6
Jack L. GALLANT
9:51 AM
@christoph hopefully @liberty will offer some perspective based on the paper that you cite. They do say that motor activity during production and perception are quite different. What they see during perception could be some sort of generalied suppression, or a change in attention, or something else. Its hard to tell without a good encoding model and I don't recall if they did that.
Liberty Hamilton
9:52 AM
@jack we did use encoding models, actually found relatively good performance of spectrotemporal models in motor cortex but for a *small region* (two blobs in motor cortex, one ventral, one dorsal. Looks like it overlaps with laryngeal area)
Jonathan Fritz
9:52 AM
Greg, suppression of self-vocalization does not seem like such a good strategy if you are a singer trying to hit the right note. Are there different "modes" of activation/suppression depending upon constraints of self-monitoring? You may recall the study suggesting that the arcuate fasciculus is larger in RH in singers?
The idea is that the suppression signal is coding for a prediction of the sensory consequences of the action. So if the correct note is perceived the excitatory bottom up input will cancel the motor-to-sensory suppression. It’s when the inhibitory prediction mismatches the actual feedback that an error signal is detected (more activity for the actually perceive feedback). Details here: Hickok, G. (2012). Computational neuroanatomy of speech production. Nature Reviews Neuroscience, 13(2), 135-145. doi: 10.1038/nrn3158
Jack L. GALLANT
9:53 AM
@liberty ah thanks. That reflects a reasonable possibility that will complicate all experiments: there are some direct perception-related motor responses, but they are in a very limited region or a very limited percptual domain.
I will miss this chat feature when zoom meetings go away.
Malcolm Slaney
9:55 AM
Jack, it's like passing notes in class... :-)
Except the “teacher” gets to read them all. ;)
Jack L. GALLANT
9:57 AM
@liberty and @matt you should give us your perspectives based on the electrophysiology. No one believes fMRI or TMS data... :^)
Sam Norman-Haignere
9:58 AM
The bird song seems like a place where a lot is known about the circuit that allows auditory-motor feedback to generate song. Can any of that circuit be mapped onto the human speech production system?
Liberty Hamilton
9:58 AM
Another interesting finding in the animal literature -- if you pair motor movement with predictable acoustics, the motor cortex suppresses the response to that sound https://www.nature.com/articles/s41586-018-0520-5
Christoph Daube
9:58 AM
So would Greg argue that the activations that Cheung, Hamilton, etal see are somewhat non-causal for the perception of speech in a language one is proficient in? (as in, altering that activity w TMS would maximally result in 10% change in perception)? Would this be somewhat in contrast with papers like this: https://journals.plos.org/plosbiology/article/authors?id=10.1371/journal.pbio.2004473 ?
Yes, I would argue that these effects are non-causal for perception in the context of normal comprehension tasks. As far as I am aware, there are no experiments showing that motor disruption affects perception in comprehension tasks.
Andrea Halpern
10:00 AM
In a recent study we found that suppressing phonation impairs poor matchers more than good pitch matchers,
Interesting! Please send me a link to the paper!
Liberty Hamilton
10:02 AM
^Andrea that's really interesting!
roberta bianco
10:02 AM
would your model hold for processing of higher levels structures in speech, as sentence level, etc?
We are ventured into that realm. Short answer is yes, broadly at least. Details here: Matchin, W., & Hickok, G. (2020). The Cortical Organization of Syntax. Cereb Cortex, 30(3), 1481-1498. doi: 10.1093/cercor/bhz180
Victor Minces
10:03 AM
It is common that when you damage a system you don't see an effect of that damage unless the system is under stress. In the laboratory conditions you shown, you don't see a motor effect when the subject is performing at ceiling, but you see it under harder conditions like with background noise or non-words. I think that interpreting that you can unveil the motor effect under stress is more parsimonious than the conclussion that the motor system only kicks in when the subject is performing a task
Good point. The question then becomes, are those more demanding conditions ever part of the natural ecology of speech perception? From a speech perception perspective, natural language is pretty much all about comprehending the sounds we hear not deciding whether the pairs of successive nonwords are the same or different. (Yes, we tested whether comprehension of speech in noise depends on the motor system; it doesn’t: https://www.biorxiv.org/content/10.1101/2020.04.02.022822v1). So you may be right that stressing the system reveals a contribution of the motor system, if that contribution is never used in actual language processing, then the “contribution” is moot.
Victor Minces
10:04 AM
Plus, "normal" conditions resemble more speech in noise than the aseptic laboratory tasks
See above. Also, normal speech in noise experiences are strongly constrained by context (situational, sentential, semantic). I predict that if we pitted such contexts against the motor contribution, non-motor sources of constraint would completely obliterate whatever motor effects might exist.