So we (Okada et al.) decided to try to nail this issue down with the following fMRI experiment: we localized auditory cortex in two different ways in individual subjects, using an AM noise localizer scan and using an anatomically defined mask covering Heschl's gyrus. We then measured the activity in the ROIs (left and right hemis) while listening to auditory speech only and while listening to auditory+visual speech. We found that the addition of the visual speech signal significantly up-regulated the activity of auditory cortex compared to auditory speech alone.
It's confirmed then: visual speech information drives auditory processing at the first stages of the cortical processing hierarchy.
Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C. R., McGuire, P. K., Woodruff, P. W. R., Iversen, S. D., & David, A. S. (1997). Activation of auditory cortex during silent lipreading. Science, 276, 593-596.
(2013) An fMRI Study of Audiovisual Speech Perception Reveals Multisensory Interactions in Auditory Cortex. PLoS ONE 8(6): e68959. doi:10.1371/journal.pone.0068959
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2005). Visual speech speeds up the neural processing of auditory speech. Procedings of the National Academy of Sciences, 102, 1181-1186.
Congrats for this nice study on AV speech perception.
You mention in your paper that 'Although previous studies of silent lip-reading have found activation in lower-level auditory cortical regions ...these studies relied on group average activation maps, which can lead to mislocalizations due to averaging error'. This is no exactly true, Pekkola and colleagues (your reference 22) used individual ROI analyses on Heschl's gyrus.
Second, Calvert and colleagues (1997, 2001) mentioned that a supra-addtive response is necessary to really determined audio-visual interactions in a specific brain area (i.e., AV > A+V). A point which is not clearly mentioned in the Discussion.
Right. My error in the post. We did note in the paper that they had localized activity to early auditory areas, those reports only looked at visual speech with no auditory speech. So you have to worry whether it is some form of auditory imagery.
Supra-additivity is a controversial definition of AV interactions. Would you like to argue based on this idea that what we observed is not visual modulation of the auditory response? Ie, what's the point of your comment?
I just did not understand why you did not (also) test visual-only speech perception and did not mention that point on your paper.
Post a Comment