Monday, September 14, 2015

Can neuroscience inform cognitive theories?

I often get this question from cognitive scientists, usually coupled with a dubious attitude.  It seems worthwhile to build a collection of papers that represent success stories in the effort to use neuroscience to constrain cognitive theories. Send me your examples via email, Twitter @gregoryhickok, or as a comment to this post and I'll update this list.


How to test a cognitive theory with fMRI.
Chatham and Badre

Bridging computational approaches to speech production:...
Walker and Hickok

Greater neural pattern similarity across repetitions is associated with better memory.
Xue, et al.

What can functional imaging tell the experimental psychologist?
Henson.

Amnesia and the distinction between long- and short-term memory.
Baddely & Warrington

43 comments:

Max Coltheart said...

Re the Baddeley & Warrington paper: I see no constraining of cognitive theories by neuroscience here. It is true that the amnesic patients they studied had brain damage. But the authors did not care about the nature, cause or location of this damage, and in their cognitive theorizing they were not constrained in any way by any neuroscientific data.

I think it is a great idea to seek to assemble a collection of papers that represent success stories in the effort to use neuroscience to constrain cognitive theories. But this effort will not succeed unless each paper is accompanied by a little annotation that goes like this:

“In this paper, the cognitive theory under consideration was A, and the neuroscientific data which constrained that theory was B”.

Only papers for which both A and B can be unambiguously identified belong on this list.

Applying this criterion would immediately show us that the Baddeley & Warrington paper doesn’t belong there because, though it clearly has an A, it has no B.

I am not at all sure that the other 4 papers on the list pass either. Let’s take the Walker & Hickok paper, for example. What is its A, and what is its B?

Max C

Dan Mirman said...

There was a good back-and-forth about this topic between Coltheart and Cooper & Shallice prompted by their book.

Among my favorite examples is how human and non-human neuroscience produced the biased competition view of attention and (to my mind) resolved the cognitive debate about early vs late attention. Two very different examples: O'Craven et al. (1997, Neuron) and Briggs et al. (2013, Nature).

Andrew Wilson said...

(Looking at these as I get a chance)

The Henson paper...

"I argue here that functional neuroimaging data - which I restrict to the haemodynamic techniques of fMRI and PET - can inform psychological theorizing, provided one assumes a "systematic" function-structure mapping in the brain.
...
I then consider in greater detail what is meant by a "systematic" function-structure mapping and argue that, particularly for structure-to-function induction, this entails a one-to-one mapping between functional and structural units..."

So, like Kosslyn, this approach only works if there is little or no degeneracy and neural reuse. So that's no good.

Greg Hickok said...

Max: I'm just posting what people send me. I agree, we need to evaluate each and lay this out clearly. Re: Walker & Hickok, A = a cognitive theory that has a single phonological level of representation; B = a cognitive theory that has a split phonological level corresponding to auditory and motor networks.

Andrew: So I probably disagree with both you and Henson.

Dan: Thanks for the additions!

Andrew Wilson said...

Andrew: So I probably disagree with both you and Henson.
No you don't; his thesis is that neural data can constrain cognitive theories! If you disagree about the need for a roughly 1:1 structure:function mapping you'll need a good reason.

Greg Hickok said...

Not that part. To be honest, my approach to this issue is more applied. I've found that looking to neural data has helped me develop new hypotheses about cognitive processes that I can then test. The dorsal-ventral model that I developed came from neural data. The auditory-motor interface (dorsal stream) models that I've been developing were neuroscience inspired. It just works in my experience. I suppose that's my reason.

Dan Mirman said...

This question is often framed around what neuroscience can do, but I think it is also useful to think about how we formulate cognitive models. Neuroscience will have a hard time informing cognitive models that make no neural commitments. Taking Greg's example, as I understand it, the auditory-motor interface model makes a commitment to the dorsal stream. This means that neuroscience research about how the dorsal stream works will inform the auditory-motor interface model. Someone else could have developed a SLAM-type model without any neural commitments, which would make their model immune to neuroscience evidence. That's not a property of neuroscience; it's a property of the cognitive model.

mettle said...

The first that come to mind are:
- Work on phonological representations, e.g., Eulitz & Lahiri (2004), Obleser, Lahiri & Eulitz (2004), Chang, Rieger et al., (2010).
- BLumstein's and Dronker's work on aphasia, which has informed speech processing theories.

Max Coltheart said...

Greg, I defined B as: "the neuroscientific data which constrained the theory B”. Your reply re B in the Wlaker & Hickok paper was that "B = a cognitive theory that has a split phonological level corresponding to auditory and motor networks" This is not a reference to any neuroscientific data.

So let me ask in again: in the Walker & Hickok paper, what were the neuroscientific data -the facts about the brain - which constrained the cognitive theorising?

Greg Hickok said...

Max: Oops. Sorry. Read it too quickly. What started me thinking about all this was an fMRI study that I did years ago: it showed activation of auditory cortex during silent naming.

Hickok, et al. (2000). A functional magnetic resonance imaging study of the role of left posterior superior temporal gyrus in speech production: implications for the explanation of conduction aphasia. Neuroscience Letters. Volume 287, Issue 2, 23 June 2000, Pages 156–160

This got me thinking about the role of the auditory system in phonological aspects of speech production...

Hickok, G. (2000). Speech perception, conduction aphasia, and the functional neuroanatomy of language. In Y. Grodzinsky, L. Shapiro & D. Swinney (Eds.), Language and the brain (pp. 87-104). San Diego: Academic Press.

which led to the development of the Hickok & Poeppel Dorsal-Ventral model of speech processing...

Hickok, G., & Poeppel, D. (2000). Towards a functional neuroanatomy of speech perception. Trends Cogn Sci, 4, 131-138.

Hickok, G., & Poeppel, D. (2004). Dorsal and ventral streams: A framework for understanding aspects of the functional anatomy of language. Cognition, 92, 67-99.

Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nature Reviews Neuroscience, 8(5), 393-402.

and to the discovery of auditory-motor integration area Spt

Buchsbaum, B., Hickok, G., & Humphries, C. (2001). Role of Left Posterior Superior Temporal Gyrus in Phonological Processing for Speech Perception and Production. Cognitive Science, 25, 663-678.

Hickok, G., Buchsbaum, B., Humphries, C., & Muftuler, T. (2003). Auditory-motor interaction revealed by fMRI: Speech, music, and working memory in area Spt. Journal of Cognitive Neuroscience, 15, 673-682.

Hickok, G., Okada, K., & Serences, J. T. (2009). Area Spt in the human planum temporale supports sensory-motor integration for speech processing. J Neurophysiol, 101(5), 2725-2732.

which led me to work develop of a theory of what computations Spt is carrying out vis-a-vis auditory motor integration in the control of speech production

Hickok, G., Houde, J., & Rong, F. (2011). Sensorimotor integration in speech processing: computational basis and neural organization. Neuron, 69(3), 407-422.

Hickok, G. (2012). Computational neuroanatomy of speech production. Nature Reviews Neuroscience, 13(2), 135-145

which led to Walker and Hickok developing a computational implementation of (a part of) my theory, which outperformed a long-standing cognitive/computational model of speech production (Dell's). And all of this has led me to re-think the nature of phonological processes as understood in linguistics and psycholinguistics.

Hickok, G. (2014). The architecture of speech production and the role of the phoneme in speech processing. Lang Cogn Process, 29(1), 2-20.

Hickok, G. (2014). The architecture of speech production and the role of the phoneme in speech processing. Lang Cogn Process, 29(1), 2-20.

My entire career is built on looking to neuroscience and other sources as a means to constrain and shape how I think about cognitive processes. I can personally attest: it works.

Max Coltheart said...

These 5 papers are offered as examples of “success stories in the effort to use neuroscience to constrain cognitive theories”. In order to determine whether they really do count as examples, we need to determine (a) what cognitive theories a paper is concerned with and (b) what neuroscientific data are used in that paper. I’ll consider each paper in turn.

Baddeley & Warrington: this should not be on the list because it does not use neuroscientific data to constrain the cognitive theories with which it is concerned, It does not even report any neuroscientific data. Sure, the behavioural data were collected from people who had (various forms of) brain damage. But that is irrelevant because the authors ignored the nature, location etc. of the brain damage when doing their cognitive theorizing.

Henson: This should not be on the list because it is a review paper which does not report any new cognitive theorizing or neuroscientific data. It does refer to a number of publications on memory which, Henson considers, are examples in which neuroscience has been used to constrain cognitive theories. It is these original research reports that might be appropriate for the list (if indeed it is clear from scrutiny of them both a) what cognitive theories a paper is concerned with and also (b) what neuroscientific data are used in the paper.

Chatham and Badre: exactly the same points apply to this paper as to the Henson paper.

Xue et al: lots of neuroscientific data. But were these data used to constrain the cognitive theorizing? Well, to answer that, we would first need to know what the cognitive theory is that this paper proposes. Can anyone tell me what that cognitive theory is? Note that the theories mentioned in the last sentence of the abstract are neural theories, not cognitive theories. Same goes for the paper’s title. So; there is neuroscience here, but I don’t see that there is cognitive theorizing.

Walker & Hickok: No worries re cognitive theories here, because the cognitive theory (the SLAM model of speech production) is plain in Figure 3. So: what about the neuroscience? Is there anything about the design of the SLAM model that was dictated or constrained by neuroscientific data. Not in this paper. Nowhere in the paper do we find statements such as “we made this architectural decision about the SLAM model on the basis of the following neuroscientific considerations . . .”.

In sum, I don’t believe that any of these 5 papers show us “success stories in the effort to use neuroscience to constrain cognitive theories”.

What to do? Well, this is an important endeavour and it can be sharpened up by two little changes in the rules.

First, submitters should be asked to submit references only for primary sources, papers that reported the original cognitive theorizing and neuroscientific data.

Second, to assist readers and focus the minds of submitters, the submitters should be asked to include a sentence that specifies what cognitive theory the paper deals with and another sentence that states what relevant neuroscientific data the paper contains.

What about it, Greg?

David Poeppel said...

My own view is that discussing this issue "in principle" is very difficult - a more pragmatic stance strikes me as helpful. That being said, I tend to agree with Max that there are not many examples ...

One problematic issue is that very rarely are careful linking hypotheses formulated between cognitive theories and neurobiological implementation. Dave Embick and I have worried about this issue for a long time and written about it here:

http://psych.nyu.edu/clash/dp_papers/EmbickPoeppel2014.pdf
http://www.psych.nyu.edu/clash/dp_papers/Poeppel2012.pdf
http://www.psych.nyu.edu/clash/dp_papers/poeppelEmbick2005.pdf

It's not (even) enough to provide what Max asks for, i.e. cognitive theory A and neural data B. To develop a satisfying causal, mechanistic model, one needs to specify the proper granularity of the linking hypothesis and spell out how implementation, algorithm, and computation work together. A high bar.

That being said -- there are cool examples that show how neural data contributes decisively to theoretical debates. For example, I am moved by the experiments that Alec Marantz has done on the stature/role/function/"foundationalness" of morphological structure in language procressing. There, for instance, the theoretical debate concerns whether or not morphemes are necessary, fundamental building blocks for language processing (contra to many proposals in psycholinguistics), and Alec provides MEG data (yes, we can point the details, if necessary) that show - I think compellingly - that you cannot have a successful online processing story that doesn't have morphology.

http://psych.nyu.edu/morphlab/people/alec.html

Greg Hickok said...

Walker and Hickok didn't detail the neuroscience behind the architecture development because I already published three recent papers on that.

What do you advise us as a field, Max? Should those of us interested in cognitive science give up doing neuroscience? Or is your interest just in evaluation of where we stand in cog neuro integration?

By the way, I suspect if we looked closely at people doing computational neuroscience we would see a lot of examples. But this field is not usually interested in testing one cognitive boxology against another. They are developing their own models (which we could nonetheless compare to existing boxologies) and would be an example of how how neuroscience constrains theory development.

Greig de Zubicaray said...

I'll throw my hat in the ring...

Greig de Zubicaray said...

Here are two more on single vs. dual process memory models:

de Zubicaray, G. I., McMahon, K. L., Dennis, S., & Dunn, J. C. (2011). Memory strength effects in fMRI studies: a matter of confidence. J Cogn Neurosci, 23(9), 2324-2335. doi: 10.1162/jocn.2010.21601

de Zubicaray, G. I., McMahon, K. L., Hayward, L., & Dunn, J. C. (2011). Pre-experimental familiarization increases hippocampal activity for both targets and lures in recognition memory: an fMRI study. J Cogn Neurosci, 23(12), 4164-4173. doi: 10.1162/jocn_a_00092

Mike Page said...


Hi All,

Thanks to Max for directing me to this interesting conversation. My own Cortex (2006) paper on this topic was restricted to looking at whether fMRI had, or even could, contribute to cognitive theorizing. Anyone who is interested can read it here:

Published version (£): http://www.ncbi.nlm.nih.gov/pubmed/16771054

Late draft: http://uhra.herts.ac.uk/bitstream/handle/2299/2329/103180.pdf;jsessionid=211806FA5DB414C7349F1F7470228D98?sequence=1

It was really a direct response to Henson (my former PhD student!) and, to some extent, Shallice.

Given the particular weakness of fMRI, I would like to see a subcategory comprising claims based on fMRI. Like Max, though, I do think it would be helpful to say (i) exactly what cognitive theories were being contrasted, and (ii) exactly what neural data were decisive in this regard. Without these statements, we will have just a list of unevaluated (and difficult to evaluate) claims - rather like the literature itself (though I am a bit out-of-date). In my paper, I discuss dual-route versus single-route theories, so it would be interesting Greig, if you could flesh out your claim as above, so we can evaluate it in regard to the papers you mention.

My own feeling is that certain "brain" techniques are likely to be more productive than others. I guess even Max would concede that something like TMS could be used strictly experimentally to test between theories, in as much as the application of the technique can be thought of as giving a temporary "stroke" - a lab analogy for good, old fashioned cog neuropsychology. (Max, please correct me if I am wrong.) Single-cell recordings from, say, from visual areas might be able to decide, for example, whether low-level visual processes are affected by "higher-level" visual attention - though whether one could prove that the PHYSICAL effect is the same as that proposed in a COGNITIVE theory might be more tricky. (It would not be sufficient, for instance, to identify the presence of "top-down" connections, without demonstrating that they function as predicted during actual processing.)

Anyway, perhaps we can make a distinction between different types of "brain" data/manipulation. My bet is that fMRI will be the weakest contributor to theory (and possibly the most expensive :-)).

Let battle commence!

Best wishes,

Mike Page

Max Coltheart said...

Dear Greig,

Those papers are certainly two promising candidates.

But could you say for each one (a) what the neuroscientific data reported in the paper are that are constraining the cognitive theories and (b) what otherwise plausible lines of cognitive theorising are ruled out by these neuroscientific data?

Best,

Max

Max Coltheart said...

Replying to Greg Hickok's post of 16 Sept:

Depends what you mean by “field”, Greg.

Cognitive psychology can explore cognitive theories just with behavioural data: brain data not required.

Cognitive neuroscience can explore the neural bases of cognition without seeking to use brain data to test cognitive theories: cognitive theories not required.

Both are fine and successful enterprises.

It is only when they try to speak to each other that the difficulties arise. As you recently tweeted re mirror neurons, an awful lot of junk science emerges when people claim to be doing this.

My aim has been to identify unambiguous examples of success stories in the effort to use neuroscience to constrain cognitive theories, by asking people for examples. Have been doing that since 2006. No examples so far.

[Coltheart, M. (2006). What has functional neuroimaging told us about the mind (so far)? Cortex, 42, 323-331; Coltheart, M. (2006). Perhaps cognitive neuroimaging has not told us anything about the mind (so far). Cortex, 42, 422-427.]

Greg Hickok said...

My career doesn't count, Max?

Greig de Zubicaray said...

Dear Mike and Max,

Let’s start with a distinction (or choice of battle ground): some cognitive theories explicitly invoke neural mechanisms, and so are more amenable to testing with neuroscientific methods. Dual-process memory theories are one example, as they invoke dissociable neural responses to support putatively specific processes of recollection and familiarity. Embodied language theories are another.

So, in response to Max:

(a) Both of the papers employ analyses of fMRI data informed by single vs dual process signal detection model estimates, and show the neural responses are consistent with the former, but not the latter.

I think (b) neglects the fact that science often progresses by post-hoc extemporising to derive hypotheses for future studies. Experiments are always designed to rule out a limited set of a priori hypotheses. Any number of “plausible lines of cognitive theorising” can be proposed post hoc, and subjected to later testing with novel experiments. No single experiment definitively falsifies/rules out a theory.

As for embodied language:

de Zubicaray, G., Arciuli, J., & McMahon, K. (2013). Putting an "end" to the motor cortex representations of action words. J Cogn Neurosci, 25(11), 1957-1974. doi: 10.1162/jocn_a_00437

I think we all agree that experiments are informative when they test an alternative hypothesis. Too often, the null is tested. When fMRI is involved, that is an expensive waste of time and effort.

regards,

Greig

Brad Buchsbaum said...



Buchsbaum, B. R., & D'Esposito, M. (2008). The search for the phonological store: from loop to convolution. Cognitive Neuroscience, Journal of, 20(5), 762-778.

("loop to convolution" -- get it?)

The above paper proceeds by way of a triangulation method. The phonological store is "placed on the chess board". The pieces are moved about. Finally, the store has nowhere left to go. check mate.

Moritz Wurm said...

I think this one could be a prime example:

Wurm MF, Lingnau A (2015) Decoding actions at different levels of abstraction. J Neurosci 35:7727-7735.

Motor theory predicts A but not B, cognitive theory predicts B but not A.
Results reveal B but not A, in line with cognitive theory and challenging motor theory.

Brad Buchsbaum said...

In response to Max's comment:

"It is only when they try to speak to each other that the difficulties arise..."

First, lets be honest, most of cognitive neuroscience is "bootstrapped" largely on ideas from cognitive psychology. So the idea that they are separate is not quite right.


However, it's true that arise. This occurs to my mind because of a kind of "impedance mismatch" between the domains. As David Poeppel pointed out this is where "linking propositions" are needed. Such propositions allow one to move between concepts expressed at two levels (e.g. mental and brain processes).

Example: see Ratcliff's diffusion model applied to both monkey and neuroimaging data of decision making.

Max Coltheart said...

Greig de Zubicaray wrote on September 18, 2015:

I think (b) neglects the fact that science often progresses by post-hoc extemporising to derive hypotheses for future studies. Experiments are always designed to rule out a limited set of a priori hypotheses. Any number of “plausible lines of cognitive theorising” can be proposed post hoc, and subjected to later testing with novel experiments. No single experiment definitively falsifies/rules out a theory.

I agree and have said pretty much this in print:

“ One can’t hope for data to show which theory is certainly true and which is
certainly false; science isn’t like that. But one can hope for data that makes one theory seem a sounder bet than the other; even better, if there several different results that all make one of the theories seem the sounder bet, and there are no results favouring the other theory, it is rational to prefer the universally supported theory. That is all I will mean by data “distinguishing between competing psychological theories”. Cortex, (2006) 42, p 323)

Max Coltheart said...

Greig de Zubicaray wrote on September 17, 2015 nominating two papers as examples of work in which there has been constraining of cognitive theories by neuroscience. My response:

There are two theories of recognition memory being considered in Greig’s papers: dual-process (DP) theory and single-process (SP) theory. What exactly is the aim of the work reported in these two papers, though? There seem to be two possibilities:

(a) Is it to use imaging data to decide which of the theories, when considered purely as cognitive theories, should be favoured?

Or

(b) is it, as the first sentence of the “Pre-experimental familiarization” paper says, to decide how best to decide between “opposing interpretations of the differential hippocampal activity elicited by studied information”?

Or perhaps both?

Both are valuable aims. But only the first is relevant to our discussion. So I focus on that aim.

Consider first the “Pre-experimental familiarization”

What does this paper provide us with respect to Aim (a)? Well, the end of the Abstract does not seem to say anything about Aim (a); it is about Aim (b) i.e. how to interpret the hippocampal data.

If this paper were at least partly about Aim (a), it would have contained a section at the end of the Introduction which states what the different predictions DP and SP theories make re what will be seen in the imaging data, and if the paper wished to conclude that the imaging data favoured SP over DP the Discussion would have pointed to aspects of the imaging data which conflict with the previously stated predictions of DP theory and are in accordance with the previously stated predictions of SP theory. Neither of these things is in the paper. That is why the paper is in my view not about Aim (a).

What about the “A Matter of Confidence” paper? Is it about Aim (a) or Aim (b) or both?

The first paragraph states the DP and the SP theories in purely cognitive terms (and also distinguishes between two variants of DP theory). However, the last sentence of the Introduction states the aim of the study as evaluating previous attempts to dissociate familiarity and recollection signals in the hippocampus and perirhinal cortex (the first sentence of the Discussion is similar). No contrasting predictions from the DP and SP theories about the outcome of the imaging experiment are stated in the Introduction.

I can’t see anything in the Discussion or the Summary that says anything like: these results are inconsistent with predictions from the DP theory as it is defined in the first paragraph of the paper. The Discussion is largely about Aim (b).

So it seems to me that, interesting though both these papers be, neither pursues Aim (a) and so neither is an example in which cognitive theorizing has been constrained by neuroimaging data.

Max Coltheart said...

‪Greg Hickok‪ said. plaintively
My career doesn't count, Max?‬
‪September 18, 2015 at 7:18 AM‬

That illustrious career counts for many things, Greg, but does it count as providing examples of constraining of cognitive theories by neuroscience? The only way I can see of deciding that is for you to do as e.g. Greig de Zubicaray and Moritz Wurm have done: nominate specific papers that you consider to be examples of this, and specify what the cognitive theories are that the paper is about and what the neuroscientific data are that have been used to constrain these theories

Max Coltheart said...

Blogger Moritz Wurm said on September 23, 2015
I think this one could be a prime example:

Wurm MF, Lingnau A (2015) Decoding actions at different levels of abstraction. J Neurosci 35:7727-7735.

Motor theory predicts A but not B, cognitive theory predicts B but not A.
Results reveal B but not A, in line with cognitive theory and challenging motor theory.

Dear Moritz,

On which pages of the paper are these two theories described?

And are both theories cognitive theories? They have to be if this paper is to be relevant to this blog, because the topic of the blog is the constraining of cognitive theories by neuroscientific data.

On my first reading of the paper, the two theories you have in mind are not cognitive but neural. One theory says that premotor cortex is critical for action mappings and the other says instead that regions such as LOC are what's important. Those are neuroscientific theories, and it would not be surprising to find examples where neuroscientific theorising has been constrained by neuroscientific data. What we are looking for instead are examples where cognitive theorising has been constrained by neuroscientific data.

Max

Max Coltheart said...

Blogger Moritz Wurm said on September 23, 2015
I think this one could be a prime example:

Wurm MF, Lingnau A (2015) Decoding actions at different levels of abstraction. J Neurosci 35:7727-7735.

Motor theory predicts A but not B, cognitive theory predicts B but not A.
Results reveal B but not A, in line with cognitive theory and challenging motor theory.

Dear Moritz,

On which pages of the paper are these two theories described?

And are both theories cognitive theories? They have to be if this paper is to be relevant to this blog, because the topic of the blog is the constraining of cognitive theories by neuroscientific data.

On my first reading of the paper, the two theories you have in mind are not cognitive but neural. One theory says that premotor cortex is critical for action mappings and the other says instead that regions such as LOC are what's important. Those are neuroscientific theories, and it would not be surprising to find examples where neuroscientific theorising has been constrained by neuroscientific data. What we are looking for instead are examples where cognitive theorising has been constrained by neuroscientific data.

Max

Moritz Wurm said...

Dear Max,

You are right, in the paper we focus on the neural aspect of the theories. But the underlying concepts are cognitive/psychological: motor theories of action understanding can be summarized as theories that claim an internal motor simulation of an observed action in order to access the meaning of the action (simulation theory, mirror theory) or that the conceptual meaning of an action is "grounded" in, i.e., associated with, the observer's own motor repertoire (some embodied theory accounts). Cognitive theories of action understanding claim that action understanding builds on perceptual analysis without the need of motor reenactment.
In short, these two psychological theories make claims about distinct systems required for action concept access (motor system vs. perceptual system) that can be translated into concrete neuroanatomical predictions.

Note that the case might be less straightforward than I present here. For example, in motor theory action understanding of course requires a perceptual analysis as well. But the generalization to abstract action concepts is claimed to take place in the motor system (which makes only little theoretical sense to me, because these representations cannot, by definition, be motor anymore). Other accounts (e.g., strong embodied theory) assume that action recognition requires no concept access at all, simulation of the concrete observed action is fully sufficient. This account can be refuted because we do find evidence for abstraction. For a more comprehensive analysis, check out Greg's excellent book, The Myth of Mirror Neurons.

In summary, I hope I could convince you that the theories we tested in our study are in fact psychological theories that assume distinct cognitive models of action recognition and understanding. These cognitive models can be formulated as neurocognitive models, at least for human cognition, as humans have brains, which is the substrate of cognition. I am curious to hear your opinion on that!

Max Coltheart said...

Dear Moritz,

Theory A: Action understanding uses an internal motor simulation of an observed action in order to access the meaning of the action.

Theory B: Action understanding builds on perceptual analysis without the need of motor reenactment.

I agree with you that these are purely cognitive theories. And also, importantly, they are logically incompatible: at most only one of them can be true.

So: what is the prediction from Theory A that your data conflict with, and what is the prediction from Theory B that is confirmed in your data?

Max

Moritz Wurm said...

Dear Max,

you asked "what is the prediction from Theory A that your data conflict with, and what is the prediction from Theory B that is confirmed in your data?"

As I wrote, in Theory A "the generalization to abstract action concepts is claimed to take place in the motor system" --> We should find evidence for abstract action representations in or close by premotor cortex but not in occipitotemporal cortex.
Theory B predicts the opposite: The generalization to abstract action concepts should be observed in or close by occipitotemporal cortex but not in premotor cortex.
The latter is exactly what we found.

Greig de Zubicaray said...

Adopting a line of argument that is perilously close to sophistry, Max writes:

“(a) Is it to use imaging data to decide which of the theories, when considered purely as cognitive theories, should be favoured?”

In my earlier response, I distinguished cognitive theories that explicitly invoke neural mechanisms, so “when considered purely as cognitive theories” is an obvious interpolation designed to constrain the argument to suit a functionalist perspective.

Elsewhere I’ve written:

“It is worth noting that the functionalist perspective reflects only one attempt at a solution to the perennial mind-body problem in philosophy. Despite the occasional antipathy expressed among philosophers, such solutions are not dogma, and they should not be revered as such. Many scientists, including cognitive scientists, adhere to materialist philosophies that link cognitive and brain processes.” Brain and Cognition, (2006), 60, 273


Returning to the materialist perspective of the 2 fMRI papers...

For the “Pre-experimental familiarization” paper Max writes:

“If this paper were at least partly about Aim (a), it would have contained a section at the end of the Introduction which states what the different predictions DP and SP theories make re what will be seen in the imaging data, and if the paper wished to conclude that the imaging data favoured SP over DP the Discussion would have pointed to aspects of the imaging data which conflict with the previously stated predictions of DP theory and are in accordance with the previously stated predictions of SP theory. Neither of these things is in the paper.”

That is a misrepresentation. Below are those statements, faithfully reproduced.

From the Introduction –

“We focused our investigation of hippocampal activity on pre-exposed, familiar (i.e., strong memory) items. If “old” responses involving correct versus incorrect attributions of a familiar item to the study list show differentially increased activity, then this result may be interpreted as supporting the operation of a qualitatively distinct process such as recollection, as memory strength should be equivalent for both targets and lures. However, if hippocampal activity is also elevated for old responses with incorrect attributions of familiar items to the study list, then this may be considered evidence in favor of a memory strength account…”

And from the Discussion –

“The equivalent hippocampal activity across familiar targets and lures, including that observed for only high-confidence items, may therefore be considered consistent with a memory strength account.”


For the “A Matter of Confidence” paper Max writes:

“No contrasting predictions from the DP and SP theories about the outcome of the imaging experiment are stated in the Introduction... I can’t see anything in the Discussion or the Summary that says anything like: these results are inconsistent with predictions from the DP theory"

Again, that is a misrepresentation.

From the Introduction –

“…DPSD proposals based around perirhinal cortex and hippocampal neuronal responses view item repetition as …influencing familiarity to a greater extent than recollection (e.g., Eichenbaum et al., 2007, Table 1). The UVSD account instead predicts monotonic, although not necessarily linear, responses...”

And from the Discussion –

“…on the basis of statistical comparisons of the coefficients and slopes for the hippocampus and perirhinal cortex activity, our results indicate that the two MTL regions show equivalent strength responses, arguing against an anatomical division of labor in terms of different types of information processing.”

Perhaps some confusion has arisen because I used “memory strength account” synonymously with “single-process theory”, although in my defense this is relatively common in the literature.

Brad Buchsbaum said...

Moritz,

Your theory has an embedded neural prediction (i.e. the brain areas). It's a linking hypothesis from the "cognitive" to the biological domain. By making contact with the physical universe (how dare you!), you've given up the cognitive "purity" of your test. I don't think Max is going to accept this line of argument.

Brad

Moritz Wurm said...

Hehe, well let's see, Brad!

Remember, the question was "can neuroscience inform cognitive theories?"

So let's see: "pure cognitive motor theory" claims that action understanding relies on the observer's own motor repertoire. I could test this, e.g., by testing if the absence of motor experience results in an impairment of action understanding. And let's assume that this is the case (it is not, by the way). Is there now a conflict with "neurocognitive motor theory" (action understanding relies on motor representations in premotor cortex)? Does my neuroimaging study influence the interpretation of the psychological study? I would say yes: whatever happens in premotor cortex, it is not the access to the abstract action concept. Maybe we can use our own motor system to simulate the concrete observed action and to predict its outcome, and this could facilitate the access to the concept. But the concept seems to be more associated with the perceptual system, not the motor system. We know this thanks to my study. --> Neuroscience informed a cognitive theory. Psychologist therefore should read my study, and actually they do. By the way, needless to say that I should also check out the psychological studies to inform my hypothesis space.

Actually, I would frame it a bit differently. There is no "cognitive/psychological motor theory" and, detached, a "pure neurocognitive motor theory". There is only one motor theory. But we can formulate distinct psychological and neuroscientific predictions. Testing this predictions will inform the theory. Notably, the results of the psychological and neuroscientific studies should not contradict each other, otherwise there is something wrong with the formulation of the distinct predictions (do not properly target the theory) or with the experiments (do not test the predictions).

Antonia Hamilton said...

here is another example in a different domain. our fmri study http://www.antoniahamilton.com/WangRamseyHamilton_JNSci_2011.pdf used DCM to test how a social cue (eye contact) modulates visual -> motor processing. Heyes 2011 (http://www.ncbi.nlm.nih.gov/pubmed/21280938) makes the strong prediction that the direct visual->motor coupling is NEVER modulated, but only the inputs to / outputs from this coupling are modulated. Our paper tests 12 DCM models including 4 in which there is visual-motor modulation (4 anti-Heyes models) and 8 in which there isn't (pro-Heyes models). It turns out that one of the pro-Heyes models wins. But this paper gives an example where a DCM study can directly test a clearly specified cognitive model.

I've tried to argue this one with Max before and I don't think he was convinced, but I'm putting it up here as another example of how fMRI with DCM really can tell us about information processing models.

Tobias said...

Thanks for hosting this discussion! I think Max is to be applauded for asking this question repeatedly and critically reading neuroscience papers claiming to evaluate cognitive theories.

Despite being a cognitive neuropsychologist, I'll give it a shot. Many single case studies in cognitive neuropsychology formulate an alternative position in their introduction to provide the theoretical background for the null hypothesis. For example, Miceli and Caramazza (1990) observed that letter substitutions of a dysgraphic individual, LB, were constrained by C/V status. This result supports a model representing C/V status in addition to letter identity and letter position. A single case study was sufficient to rule out an alternative model not assuming additional representations in the orthographic system.

Suppose that a patient with a large left hemisphere stroke with extensive perisylvian lesions was able to carry out auditory lexical decisions and exhibited functional activation in right hemisphere homologues of 'language areas' (not going into the details here). Wouldn't that lead us to the conclusion that a mental input lexicon was located in both hemispheres compatible with Hickok and Poeppel (2007)? Couldn't we rule out a cognitive model assuming a single input lexicon? The logogen model, while not making any claims about neural implementation of the different modules, would be an example of such "single lexicon only" models.

Within a cognitive setting, we would perhaps never even consider introduction of another lexicon – why assume two input lexicons? Within a purely cognitive setting, this would not be a theoretically parsimonious model.

Based on anatomical (and, perhaps, functional imaging) data we would reject "single lexicon only"-type of models.

Cheers

Tobias Bormann, Dept. of Neurology, Freiburg

Max Coltheart said...

I think the discussion with Moritz has got to this pint:

In Wurm & Lingnau (2015), observers passively watched videos of actions performed while fMRI data were recorded (with occasional catch trials: Ss had to respond if something additional happened after the action). We’ll assume that this task engages the action understanding system.

The actions were such that they could be encoded at 3 levels: at a purely concrete level (open a wine bottle), at an intermediate level of abstraction [open a bottle – any kind of bottle], and at an abstract level [open a receptacle – any kind of receptacle]. MPVA was used to measure the degree to which different brain regions encoded information at each of these three levels.

There are two cognitive theories of how action understanding works. These theories, and their predictions about what should happen in this experiment, are:

Theory A: Action understanding uses an internal motor simulation of an observed action in order to access the meaning of the action. Theory A predicts:

(i) that intermediate and abstract levels of representation will be encoded in ventral premotor cortex (mPV).

(ii) that intermediate and abstract levels of representation will not be encoded in lateral occipitotemporal cortex (LOTC).

Theory B: Action understanding builds on perceptual analysis without the need of motor re-enactment. Theory B predicts:

(i) that intermediate and abstract levels of representation will not be encoded in mPV.

(ii) that intermediate and abstract levels of representation will be encoded in LOTC.

To complete this exercise we just need to make explicit what the bridging assumptions are that allow predictions about brain activity to be derived from the two purely cognitive theories. Two assumptions that seem essential to me are:

1. The derivation of predictions from Theory A assumes that whenever internal motor simulations of actions is carried out, even those as abstract as simulating the opening of some receptacle no matter what that receptacle is, mPV is obligatorily engaged, whereas LOTC is never engaged;

2. The derivation of predictions from Theory B assumes that perceptual analysis is being used for action understanding, even understanding actions as abstract as simulating the opening of some receptacle no matter what that receptacle is, LOTC is obligatorily engaged, whereas mPV is never engaged;

Which of these assumptions is supported by empirical data and what are those data?

Max

Moritz Wurm said...

Dear Max,

if I get your point correctly, you pick up Brad's point. You ask for the nature of evidence that establish the links between cognition and neuroanatomy. The answer is clear (neuroscientific evidence).

Does this show that I was wrong (neuroscience cannot inform cognitive theories), because testing the cognitive theories neuroscientifically builds on assumptions that were derived from neuroscientific experiments?

If you say yes, could it be interesting to ask "Can psychological experiments inform cognitive theories"? Would we have the same trouble because testing the cognitive theories psychologically builds on assumptions that were derived from psychological experiments?

Max Coltheart said...

Dear Moritz,

Clearly the bridging assumptions that enable predictions to be made about neuroscientific data from cognitive theories have to be derived from neuroscientific research. Nothing wrong with that. Sorry if I didn't make my views clear. What I meant was: what are some actual neuroscientific studies that support the view that whenever internal motor simulations of actions is carried out, even those as abstract as simulating the opening of some receptacle no matter what that receptacle is, mPV is obligatorily engaged, whereas LOTC is never engaged.

Even if there are no such studies, as long as these assumptions are made explicit we can then see how the neuroscientific predictions were derived from the cognitive theory. If there is neuroscientific evidence to support these assumptions, that greatly strengthens the conclusions of the study, of course.

Max

Max Coltheart said...

Re Antonia's post of September 29: Yes, she and I had an amiable discussion about this paper four years ago but did not reach any common ground. However, I do think it is a good paper to discuss in this forum.

The paper's Abstract states the conclusions thus: "Results supported a model in which eye contact controls mimicry by modulating the connection strength from mPFC to STS". Out like that, the conclusions of the paper are about the brain, not cognitive theory.

However, if in this sentence "mPFC" were replaced by some cognitive term describing what the cognitive process is that is involved in mimicry and which is implemented in mPFC, and if the same can be done with the term "STS", then the sentence would be expressing a cognitive theory.

Can this be done?

Max

Moritz Wurm said...

Thank you for your clarification, Max. Here is a quote from Rizzolatti et al, Physiol Rev, 2014: "In fact, crucial for action understanding are those areas that contain neurons that encode the [action] goal with a high degree of generality. With the problem posed in these terms, the evidence shows that generalization is an aspect that characterizes the parieto-frontal mirror network rather than STS [LOTC] neurons." The reverse stance can be derived from e.g. Hickok, 2014; Watson & Chatterjee, Neurology, 2011 (high generality in LOTC) and Cook & Bird, Cortex, 2013; Kilner, TICS, 2011 (no high generality in PMv).

One aspect should be clarified here. Rizzolatti et al.'s claim refers to generality across effectors (e.g. picking up a raisin with the hand or the mouth). However, there is the possibility of confound by the concrete object involved in the action. Generality across objects has never been considered so far, but I think that this kind of generality is crucial for action understanding (the understanding of "eating" should be independent of whether I observe someone eating a raisin or a banana). This issue is discussed in more detail in Wurm et al., Cereb Cortex, 2015.

Max Coltheart said...

Some useful discussions with Moritz (thanks, Moritz) have led to what's below. In my opinion in any studies aiming to use neuroscientific data to constrain cognitive theorising, spelling out in this way the specific bridging assumptions that are needed to generate the neuroscientific predictions from the cognitive theories should always be done. And it is also essential to specify, for each theory, what results the study could yield that would be in conflict with the theory (not just consider results that would be consistent the theory).

------------------------------------------------------------

1. THE COGNITIVE THEORIES:
There are two cognitive theories of how action understanding works.

1 (a): Theory A: Action understanding uses an internal motor simulation of an observed action in order to access the meaning of the action.

1 (b): Theory B: Action understanding builds on perceptual analysis without the need of motor re-enactment

2. THE BRIDGING ASSUMPTIONS

2(a): whenever internal motor simulations of actions is carried out, even those as abstract as simulating the opening of some receptacle no matter what that receptacle is, PMv is obligatorily engaged as indicated by encoding of intermediate and abstract levels of representation in MVPA analysis, whereas LOTC will not show such encoding. Evidence that motor simulation does engage PMv but not LOTC is provided by e.g. Jeannerod (Neuroimage, 2001)

2(b): whenever any kind of perceptual analysis is being used that might be relevant for action understanding, even understanding actions as abstract as simulating the opening of some receptacle no matter what that receptacle is, LOTC is obligatorily engaged as indicated by encoding of intermediate and abstract levels of representation in MVPA analysis, whereas PMv will not show such encoding. Evidence that perceptual analysis of action-relevant information does engage LOTC but not PMv is provided by e.g. Beauchamp et al, Neuron, 2002.

3. THE PREDICTIONS:

From Theory A: if 1(a) is true and 2(a) is true, it follows from Theory A that in any task involving action understanding we will see
(i) engagement of PMv and
(ii) no engagement of LOTC.

From Theory B: if 1(b) is true and 2(b) is true, it follows from Theory B that in any task involving action understanding we will see
(i) engagement of LOTC and
(ii) no engagement of PMv.

4. THE RESULTS.
Wurm & Lingnau (2015) confirmed both of the predictions from Theory A and disconfirmed both of the predictions from Theory B.

Unknown said...

There is only behavior which can be measured on different "levels". No mind, no brain as the cause of the mind (as there is no mind). We just model behavioral data from differents sources. Brain behavior is evidently more interesting than liver behavior with regard to observable human behavior.

In cognitive psychology (i.e. psychology in terms of cognition theory) we agree on the acceptance of latent (unobservable) cognitive structures and processes for theoretically modelling behavioral data (no cognition exists, it's epistemic). Behaviorists call this mentalism or the unnecessary doubling of the world.

My opinion: If data from the neuronal "level" can be modelled together with data from the organism-environment-interaction level in a meaningful way - nice!

Whatever the brain contributes to psychology - no doubt that, reversely, the so called "cognitive" concepts are capable to model neuronal behavior, e.g. fMRI data. In fact, brain scientists learn far more about their topic by applying cognitive psychology/concepts/methods than vice versa psychologists learn about human behavior (and experience) by applying fMRI and others. Cognitive neuroscience aims at the psychological characterization of brain structures and processes - not at a neuronal characterization of human behaviors/experience.