Friday, August 8, 2014

How useful is neural oscillation entrainment?

I've been struggling with this question for a while.  I haven't looked at it deeply.  It's more like a nagging ache that I've been meaning to examine closely at some point.  Here's the basic observation/idea in the auditory domain:

-Observation #1: Neural oscillations tend to entrain in the phase of their response to periodic stimuli.
-Observation #2: Many natural sounds, such as speech, are quasi periodic.
-The Claim: Oscillation entrainment facilitates perception by synching periods of maximal neural sensitivity to temporal windows in the stimulus stream that contain the most useful information.

I like this idea.  It is a reasonable idea.  I don't know if it is correct.  The alternative possibility, given the two observations is that neurons and neural networks oscillate just because--i.e., build a network with excitatory-inhibitory interactions, balance them so you don't get runaway excitation or constant suppression, and it tends to oscillate and will tend to reset it's phase with inputs (this is a fact, btw).  On this alternative possibility, phase-locked oscillations are more of a by product of network design, not something that was selected for to enhance perception.  In fact, it's conceivable that phase locking to a stimulus could actually interfere with neural signal processing, although not sufficiently to preclude the systems useful as a signal processor.

So what we need to show to decide whether oscillation entrainment is a design feature to enhance perception or whether it is a spandrel-like emergent property is examples of perceptual enhancement resulting from neural oscillation entrainment.

A recent paper by Henry & Obleser (Frequency modulation entrains slow neural oscillations  and optimizes human listening behavior, PNAS) claims to do just this.  They used an FM modulating stimulus and inserted gaps at various phases of the modulation.  The listener's task was to detect the gaps.  They found that listener performance was affected by the position of the gap relative to phase angle.  Cool!  But it wasn't consistent across subjects.  Bummer!  Different listeners had different phases that led to best performance.  BUT, when they looked at the EEG signal oscillation phase, then a consistent relation was observed.  Cool!  Behavioral was tied to neural, not signal rhythmicity.

This sounds like a very nice and important result, which is why it's in PNAS.  But again I have a nagging worry.  Maybe it's nothing to get twisted up about but here it is: If the point of neural phase locking is to time neural sensitivity to stimulus rhythmicity, then shouldn't we worry if there is no consistent relation between stimulus phase and perceptibility?  You only see the relation when you look at the neural oscillations themselves.  Maybe different listeners just have different neural delays. But by half a stimulus cycle?  Makes me worry.

So despite how appealing the theory, and how suggestive the growing collection of results is, I still worry that oscillations are doing all that much--at least on purpose.


Jonas said...

Hi Greg,

your question is a good one. It’s also an obvious one, and I presume the duty to “prove” anything here is on the side of people like me who stick their head out in saying (or rather being overwhelmed by the evidence) that neural oscillatory phenomena do affect (note the conceiled causality claim here) perception.

I like your matter-of-factly way of saying that neural oscillations, any oscillation, would become reset by environmental stimuli. If so, I would maybe go and argue that ANY emergent property of a system would be used by this system (vulgo, the brain) to solve its problems. Or as Jonathan Simon recently put it -- admittedly talking about another, but also emergent phenomenon: “Evolution takes advantage of whatever is there to take advantage of.”

I think it’s a dangerous path (for me, too, of course) to listen to the siren song of entrainment “doing” or “fixing" anything. I think it is quite simply a powerful feature of an oscillating system, and, yes, it might have consequences for the behaving individual. And it would indeed be useless, if I would NOT have these – perceptual, behavioural – consequences (oh, that’s a Peter Lakatos quote re-mixed; see a pattern here?). The biggest danger currently, probably as always, lies in equating psychological terms (“attention”) with physiological or dynamical-systems phenomena (“entrainment”, “phase reset”).

Let me end on saying that I hope our PNAS paper and neural entrainment in speech and hearing won’t become a next chapter in your myths-busters book series! But I am confident it won‘t.


Greg Hickok said...

It very well could be that an emergent property of the system's architecture--the tendency for neural systems to oscillation--could prove useful computationally by generating stimulus entrained predictions that maximize neural sensitivity at the right time. I think what we need to ask next is (1) how much does it help when things go exactly right, i.e., when the stimulus periodicity matches the oscillatory prediction? And (2) how often do things go exactly right? The computational utility of oscillations could be compromised by the answers to either question. Even if it helps a lot when things go right (the scenario we are all studying), maybe it goes right only 10% of time in real life.

I'm not trying to bust any myths, here. My point is simply this: given the attractiveness of the ideas of prediction and oscillation-based enhancements, we need to be extra careful as we move forward. Intuitive ideas require less empirical support to gain acceptance.

Magic Tony said...

Although not focused on speech perception (or audition at all), I've been rather thoroughly convinced by Kyle Mathewson's work on attentional entrainment and visual perception. He's doing some very sexy stuff.

David Poeppel said...

Consider reading Doelling et al. (2014) from my lab for arguments supporting a critical role for perceptual analysis.

Greg Hickok said...

Full disclosure: I didn't read every word of Doelling et al. so I might be missing something important. Please correct errors below.

It's a very cool study. Take away the "edges" of a speech signal and intelligibility drops dramatically, as does MEG oscillation entrainment. Replace the edges (which alone are also not very intelligible) and intelligibility increases (not fully though) along with MEG entrainment.

But does this clear demonstration of a correlation between oscillation entrainment and intelligibility mean that the oscillations are driving the syllable parsing? Or could it just be that you need edges and edges just so happen to drive entrainment because that's the way the network responds?

Unknown said...

Hi Greg,

You are missing one crucial bit from our paper which can be seen in Figure 5c (though you could be forgiven for missing it... it is not the clearest figure in the world...). You have described the first half of our analysis very well. As you mention, replacing with artificial edges increases intelligibility as well as oscillation entrainment.

The other half is this: the amount of oscillatory entrainment increase (from no edges to artificial edges) correlates with the amount of intelligibility increase (on the subject level). Here, the stimuli are the same across subjects and cannot account for the relationship of individual differences in intelligibility and oscillatory entrainment. Therefore, we believe that such entrainment must drive (or reflect, if you prefer) some cognitive process involved in the comprehension of speech (presumably syllabic parsing as has been theorized).

Our interpretation, then, suggests that oscillations entrain to stimulus edges, but it is the efficacy of that neural entrainment (combined with phonetic content in the stimulus to be decoded) which aids in the processing of speech (and presumably other natural complex sounds... stay tuned)


Greg Hickok said...

Hi Keith,

Ah, thanks for this. I did mention that I didn't read the whole thing, so I missed that point. :-) But my question remains, which is driving which? In the intro you point out,

"It has recently been demonstrated that theta envelope tracking of speech is enhanced by stimulus intelligibility (Peelle and Davis, 2012; Peelle et al., 2013), while earlier work showed similar neural phase- locking for sentences played backwards (no intelligibility) and forwards (Howard and Poeppel, 2010)."

Doesn't this suggest that (1) intelligibility can drive oscillations and (2) oscillations don't necessarily result in intelligibility?

The hypothesis you are pushing is elegant, believable, and consistent with a lot of data. I even believe that oscillations can modulate perceptibility (e.g., the phase angle effects documented by Jonas and others). But damn, I'm still not sure that oscillations are *the mechanism* by which we parse our perceptual world.

Jonathan Peelle said...

I agree it's sensible to be a little cautious here and that more data are needed (as always). Two quick points:

• The fact that intelligibility affects entrainment suggests that oscillations are not ONLY reflecting responses to acoustics (of course it could be we haven't controlled for the right acoustic manipulation, but it's nice to see converging evidence from multiple labs start to speak to this issue)

• It's probably useful to think about how oscillatory entrainment may relate to established behavioral phenomena related to speech rhythm. Matt and I speculated on a couple of possible applications in our 2012 Frontiers paper regarding phoneme perception in the context of sentences that vary in speech rate (Figure 6), and there are also nice effects of lexical perception shown by Laura Dilley and colleagues that also seem relevant (Dilley and Pitt 2010; Figure 3 in our review paper).

Linking oscillatory entrainment to specific aspects of speech perception will be useful in better understanding what's going on. I agree that more work needs to be done here.

I don't particularly have an issue with individual differences in the relationship of the neural phase to that of the stimulus; as long as the relationship is consistent within an individual, it seems this would be sufficient to affect behavior. I agree that the reasons for this difference are interesting and that perhaps animal electrophysiology would shed some light here.

Dilley LC, Pitt MA (2010) Altering context speech rate can cause words to appear and disappear. Psychological Science 21:1664-1670.

Peelle JE, Davis MH (2012) Neural oscillations carry speech rhythm through to comprehension. Frontiers in Psychology 3:320.