Comments on Talking Brains: What can we learn from the processing of syntactic...

I agree, you didn't say that and I didn't ...

2012-02-07T16:34:07.201-08:00

I agree, you didn't say that and I didn't intend to imply you did. I should have written "If we assume that..." I am sorry.

Yes, it is possible that the subjects rehearsed silently - though the sentences were rather short and (at least the active ones) very transparent.

It's possible there's an automatic/unconsc...

2012-02-07T15:19:46.071-08:00

It's possible there's an automatic/unconscious vs. controlled/conscious distinction here.

I've never said Broca's area is not involved in syntax. What I actually claim is "there is no compelling evidence
that there are sentence-specific processing regions [for comprehension] within Brocaʼs area" (Rogalsky & Hickok) -- a rather different claim. With that in mind, let's consider Menenti et al. The region that they find activated for syntax is posterior Broca's (~BA 44). This is the same region that activates very robustly during simple articulatory rehearsal and in fact, "syntactic effects" go away if in the BOLD signal if you control for articulatory rehearsal. In Menenti et al., subjects were listening to sentences and deciding weather the meaning of the sentence matched a picture. Question: is it possible that subjects sub vocally rehearsed at least a portion of the stimuli to help in task performance? If the answer is 'yes' (and that is the correct answer), then we have an articulatory confound that could explain the findings. I am very willing to be proven wrong, but I have not yet seen ANY evidence that convincingly shows that posterior Broca's activation is nothing more than articulatory rehearsal.

Greg, would it be reasonable to assume that, in se...

2012-02-07T15:02:54.546-08:00

Greg, would it be reasonable to assume that, in sentence comprehension, the syntactic processing of complex grammatical sentences is (typically) automatic while that of ungrammatical sentences is (in part) conscious? If so, then re-analysis can include production, which could explain why Broca’s area is activated.

However, if Broca’s area is not involved in syntax during comprehension of grammatical sentences, how would you explain syntactic repetition suppression in the left BA44/BA6 in Menenti et al. (2011)?

Vilem

Menenti, L., S. M. E. Gierhan, K. Segaert and P. Hagoort (2011): Shared Language. Overlap and Segregation of the Neuronal Infrastructure for Speaking and Listening Revealed by Functional MRI. Psychological Science 22, 1173-1182. DOI: 10.1177/0956797611418347

There's plenty of evidence that parts of Broca...

2012-02-06T16:51:38.310-08:00

There's plenty of evidence that parts of Broca's area (and beyond!) activate more to complex than simpler sentences defined in various ways (center embedding vs. right branching, object vs. subject extraction, long vs. short movement). All of these, however, confound difficulty with complexity and Rogalsky & Hickok have shown that a good chunk of that activation can be potentially be accounted for by articulatory rehearsal, which subjects are more likely to use on more difficult stimuli.

Yes, if different types of violations cause different brain responses then that is evidence for different systems being involved. But again, without a good theory of the response to violations, it is hard to draw strong conclusions. A start? Yes. Something to use as the foundation for a model of syntactic processing in the brain? No.

Greg, thanks for your reply. If I remember correct...

2012-02-06T07:22:08.867-08:00

Greg, thanks for your reply. If I remember correctly, an early study by Stromswold et al. (1996, B&L 52, 452) from David Caplan's lab showed increased activation in left pars opercularis when subjects had to process center-embedded versus right-branching sentences. This would argue for a role of that region when processing syntactically complex sentences. That study did not involve violations, if I remember correctly.

I agree that we know too little to perfectly understand the meaning of increased activation for different types of violations. But suppose you found an increase of activation in region A in response to syntactic violations in a sentence and an increase in region B in response to a semantic violation. In this case, I would conclude that, say, region B was involved in any sort of processing of meaning. Activation could reflect access to stored (semantic) knowledge, detection of a violation, update of the mental model or all sorts of combinations. So, using violations could be a start, right?

I assume your critique also involves ERP data, then?

Cheers,

Tobias, Freiburg

Dominik, thanks for your response. “Syntax as a m...

2012-01-30T11:32:22.998-08:00

Dominik, thanks for your response.

“Syntax as a mishmash”: Yes, syntax, regardless whether English or Czech, has tiers. There is some literature indicating that morphosyntax and word order may be supported by different neural circuits. The same may be true for the violation of both.

What neuroscience needs from linguistics is what most linguists can agree on. CG is an interesting enterprise but a bit too holistic. This may be the reason it reduces the combinatorial nature of syntax to mere integration of ready-to-use pieces. Of course, the boundary between syntactic and conceptual structure/processes is fuzzy. On the other hand, syntax, unlike speech sounds categorization (see Greg’s blog of Oct 6, 2011, [http://www.talkingbrains.org/2011/10/monkeys-and-their-auditory-cortex.html]) , still appears to be uniquely human, in a sense the core of language. There is the poverty of stimulus.

And you need not know the meaning of all words, either - remember Hornicek’s (?) excuse: “I’m sorry I’m late. The shtift fell into the treab.” :-)

Vilem Kodytek

@vkodytek Ahoj! I think there are a number of prob...

2012-01-28T00:26:28.855-08:00

@vkodytek Ahoj! I think there are a number of problems with: "any syntax theory has to be combinatorial". First, do we need a theory of syntax? Syntax as we know it now is kind of a strange mixture of word order and dependency operating at a number of levels. It's a mishmash of problems encountered during attempts to mathematicize and computationalize the study of language. The order words come in is important and so is the relationship between them. But just because of that it may not constitute a separate level of description.

Second, there are different ways to take combinatorial. In a formal sense, it requires certain properties (e.g. be describable by a tree theory). But any such view completely ignores large swathes of language. In an informal sense, it is obvious that you have to "combine" smaller bits to make bigger bits. But this can be accounted for by a theory of integration (I like blending theory, but am open to others) that is more cognitively plausible than complex serial operations with large lexicon lookups. Construction grammar shows that you can account for much more about language in this way. Of course, this does not always lend itself to computational treatment of language but that should always be an engineering problem, not one of theoretical linguistics.

@Greg Re: "an implicit assumption that these signals are relevant". I am increasingly worried that similar assumptions of relevance underlie much of modern psycholinguistic and neurolinguistic research. The consistency of results is certainly very suggestive but their close alignment with a theory of language that does not describe language very well is worrying. (Also, there's the "language as a fixed object fallacy" that Clark warned about.)

Absolutely right Tobias. But still we don't k...

2012-01-27T13:02:14.502-08:00

Absolutely right Tobias. But still we don't know the relation between the violation responses and the computations that underlie sentence processing normally. In fact, as far as I'm aware, there isn't even a theory, just an implicit assumption that these signals are relevant.

But different violations (semantic, syntactic) yie...

2012-01-27T12:53:20.194-08:00

But different violations (semantic, syntactic) yield increase of activation at different sites, right? Therefore, it can't be a general "error detection mechanism" or "working memory" or anything else "general purpose XY mechanism".

Asks a non-practioner.

Tobias, Freiburg

Hi Dominik, It’s nice to meet a compatriot here! ...

2012-01-27T03:38:01.372-08:00

Hi Dominik,
It’s nice to meet a compatriot here!

As far as I know, any syntax theory has to be combinatorial. So, with respect to the violation paradigm, what differences can be found between particular theories? I think none.
Vilem

To be honest, I don't know enough about the vi...

2012-01-26T15:46:09.849-08:00

To be honest, I don't know enough about the violations results to formulate an explanatory theory. But from this discussion, it appears to me that we don't have the whole story yet.

I'd like to see some results from morphologically rich languages with relatively loose syntax. For instance, in the 1940s, one Czech linguist observed that Czechs are much more likely to perceive case form violations in the speech of foreigners than aspectual violations. Or rather they perceive case ending violations as errors but try to find felicitous interpretations for aspectual violations. I can confirm that this seems to be the case from informal observations but I'd be interested to know what happens under the processing violations research paradigm.

I'd like to see some explanation of violations of idioms such as "the proof is in the pudding" or novel expressions like "to sneeze the napkin off the table".

I think any processing theory will also need to account for the fact that most of the time people don't notice violations - syntactic, semantic or idiomatic.

But most importantly, I'm a bit worried about our ability to match brain phenomena with linguistic phenomena at this stage. I took a bash at explaining what I mean in more detail here: http://metaphorhacker.net/2011/03/the-brain-is-a-bad-metaphor-for-language

Thanks for your comment Dominik. The theoretical ...

2012-01-26T14:51:59.689-08:00

Thanks for your comment Dominik. The theoretical work one places in the syntactic vs. the lexical system is certainly a matter of debate. So what's your theory of the violation results?

I'd recommend going even further back. I'd...

2012-01-25T13:32:18.718-08:00

I'd recommend going even further back. I'd take issue with "I think we all agree that listening to structured sentences involves syntactic computation." It is far from a foregone conclusion that that is the case. At least, since the 60s there have been many influential theories in linguistics that proposed that the difference between the lexicon and syntax is one of continuum and not a clear divide. There is so much evidence for that view in language that it always amazes me how little attention this hypothesis gets. It's partly because even its proponents keep writing dictionaries and grammar books - simply because the lexicon/grammar divide is such a useful heuristic. But if experiments are based on the assumption that syntax and lexicon are completely separate, they are likely to miss important things.

If you think of language processing as being more similar to face recognition than serial computation, you can come up with interesting models that account for the violation results just as well (or at least I hope so).

You make some very good points. I would like to b...

2012-01-25T12:36:28.469-08:00

You make some very good points. I would like to believe that violation studies provide useful information. But here's the thing that worries me: if the source of the violation signal (say Broca's area), doesn't activate during processing of a stimulus that does not contain a violation, isn't that a problem?

Greg, I’m excited to see that this discussion has ...

2012-01-25T07:42:17.302-08:00

Greg, I’m excited to see that this discussion has made it to your blog’s front page. The violation paradigm should, as any widely used paradigm that may just have gotten a little too popular, indeed be scrutinized.

Considering robust findings in EEG studies, I think the fact that there’s a clear-cut difference between semantic violation ERPs (N400) and syntactic ones (P600) means that there is at least something inherently semantic or syntactic about them. So they must, to some extent, reflect syntactic and semantic processing. Moreover, a posterior P600 is thought to be a marker of grammatical “revision and repair”, and it doesn’t get more syntactic than that.

Carrying this over to an fMRI approach, perhaps contrasting syntactic violation conditions with other kinds of violations - semantic ones in language, or timbre-related ones in music - could reveal an overlap in function representing a general violation response and a difference in activation that represents specific syntactic or semantic functioning. Even overlapping activation patterns could be categorized with multivariate pattern analysis. It would, however, be very easy to get lost in a maze of blurry contrasts, and there is a limit to the number of meaningful operations one can perform on scans.

What I’m getting at is that there must be some truth in syntactic ERPs, but it is difficult to extract this truth. Perhaps a more efficient approach would be one that does not center around violation, but rather around a non-surprising difference in processing difficulty. It has been established, I would say, that mental operations that are more difficult require more processing resources. Contrasting easy to difficult (garden-path?) sentences could therefore reveal differential activation representing syntactic processing, just as it reveals differences in self-paced reading times and other behavioral measures.

Going back to what sparked this debate (the hypothesized sharing of syntactic processing resources by mental representations of language and music), it may then be possible to carry out such an investigation of easy vs. difficult in music-syntactic processing as well. And that could lead to a better understanding of shared resources in music and language.