Tuesday, August 24, 2010

Do you like movies? MEG? Multi-sensory perception?

Doing our best to keep this rather technical and slightly nerdy paper light (e.g. the stimuli were clips from the film Dumb and Dumber), this new paper in PLoS Biology by Huan Luo, Zuxiang Liu, David Poeppel argues for active cross-modal phase modulation as a key mechanism underlying the construction of internal representations of naturalistic audiovisual stimuli. The paper builds on a technique that Huan Luo developed for an earlier paper (Luo & Poeppel, 2007, Neuron) that argues for the relevance of theta phase in auditory cortex. If you are looking for something to read during the last days of your summer vacation ...

Auditory Cortex Tracks Both Auditory and Visual Stimulus Dynamics Using Low-Frequency Neuronal Phase Modulation

'Low-key' author summary for PLoS:

When faced with ecologically relevant stimuli in natural scenes, our brains need to coordinate information from multiple sensory systems in order to create accurate internal representations of the outside world. Unfortunately, we currently have little information about the neuronal mechanisms for this cross-modal processing during online sensory perception under natural conditions. Neurophysiological and human imaging studies are increasingly exploring the response properties elicited by natural scenes. In this study, we recorded magnetoencephalography (MEG) data from participants viewing audiovisual movie clips. We developed a phase coherence analysis technique that captures—in single trials of watching a movie—how the phase of cortical responses is tightly coupled to key aspects of stimulus dynamics. Remarkably, auditory cortex not only tracks auditory stimulus dynamics but also reflects dynamic aspects of the visual signal. Similarly, visual cortex mainly follows the visual properties of a stimulus, but also shows sensitivity to the auditory aspects of a scene. The critical finding is that cross-modal phase modulation appears to lie at the basis of this integrative processing. Continuous cross-modal phase modulation may permit the internal construction of behaviorally relevant stimuli. Our work therefore contributes to the understanding of how multi-sensory information is analyzed and represented in the human brain.

1 comment:

seeingwithsound said...

Your PLoS Biology paper is fascinating reading. In the development of sensory substitution for the blind we are looking for means to boost linkage between auditory and visual brain areas, and one idea is to add theta range (and other) stimuli as a kind of virtual dynamic "scaffold" to help auditory and (recruited) visual cortex synchronize for improved cross-modal binding and information transfer: Entrainment for sensory substitution.

Now we have no idea yet if this idea could really work, or if so what parameter settings would be effective, but your paper definitely contributes to narrowing the educated guesses that we have to apply at this stage.