Friday, October 9, 2009

Is Broca's area the site of the core computational faculty of human language?

There has been a lot of interesting claims made recently by Angela Friederici and colleagues about an association between Broca’s area, the pars opercularis in particular, and what they call "the core computational faculty of human language": hierarchical processing. Two previous studies used artificial grammar stimuli/tasks (Bahlmann, Schubotz, & Friederici, 2008; Friederici, Bahlmann, Heim, Schubotz, & Anwander, 2006). Syllable sequences were presented according to one of two rules (presented to different groups of subjects). One rule involved only adjacent (linear) dependencies (e.g., [AB][AB]) and one involved hierarchical dependencies (e.g., [A[AB]B]). Once learned, subjects were presented with “grammatical” and “ungrammatical” strings during fMRI scanning. Violations of either grammar type showed activation in the frontal operculum, whereas only violations for the hierarchical grammar showed activity in the pars opercularis region. This latter finding is held up as evidence for the claim that the pars opercularis supports hierarchical structure building. I’m not sure what to conclude from studies involving artificial grammars so I’d like to focus on a more recent study that aimed at the same issue using natural language stimuli.

Makuuchi, et al. (2009) used a 2x2 design with STRUCTURE (hierarchical vs. linear) and DISTANCE (long vs. short) as the factors. Hierarchical stimuli involved written German sentences that were center-embedded (e.g., Maria who loved Hans who was good looking kissed Johann) whereas the linear stimuli were non-center embedded (e.g., Achim saw the tall man yesterday late at night). Already I have a problem: the notion that non-center embedded sentences are linear is puzzling (see below). The number of words between the main subject (noun) of the sentence and the main verb served as the distance manipulation. Restricting their analysis only to the left inferior frontal gyrus, Makuuchi et al. report a main effect of STRUCTURE in the pars opercularis, a main effect of DISTANCE in the inferior frontal sulcus, and no significant interactions. This finding was interpreted as evidence for distinct localizations supporting hierarchical structure building on the one hand (in the pars opercularis) and non-syntactic verbal working memory related processes on the other (inferior frontal sulcus) which was operationalized by the distance manipulation.

(LPO=left pars opercularis, LIFS=left inferior frontal sulcus, A=Hierarchical & long-distance, B=hierarchical & short-distance, C=linear & long-distance, D=linear & short-distance.)

I find this study conceptually problematic and experimentally confounded. Conceptually the notion “hierarchical” is defined very idiosyncratically to refer to center-embedding. This contradicts mainstream linguistic analyses of even simple sentences which are assumed to be quite hierarchical. For example, the English translation of a “linear” sentence in Makuuchi et al. would have, minimally, a structure something like, [Achim [saw [the tall man] [yesterday late at night]]], where, for example, the noun phrase the tall man is embedded in a verb phrase which itself is in the main sentence clause. Most theories would also assume further structure within the noun phrase and so on. Thus, in order to maintain the view that the pars opercularis supports hierarchical structure building, one must assume that (i) center-embedded structures involve more hierarchical structure building than non-center embedded structures, and (ii) that hierarchical structure building is the only difference between center-embedded and non-center embedded structures (otherwise the contrast is confounded). Makuuchi et al. make no independent argument for assumption (i), and assumption (ii) is false in that center-embedded sentences are well known to be more difficult to process than non-center embedded sentence (Gibson, 1998), thus there is a difficulty confound. One might argue that center-embedded structures are more difficult because they are hierarchical in a special way. But the confound persists in other ways. If one simply counts the number of subject-verb dependencies (the number of nouns that serve as the subject of a verb), the “hierarchical” sentences have more than the “linear” sentences.

It is perhaps revealing that the activation response amplitude in the pars opercularis to the various conditions qualitatively follows the pattern of the number of subject-verb dependencies in the different conditions: “hierarchical long” (HL) = 3 dependencies, “hierarchical short” (HS) = 2 dependencies, “linear long” (LL) = 1 dependency, “linear short” (LS) = 1 dependency, which corresponds with the pattern of response to these conditions in pars opercularis which is HL > HS > LL ≈ LS (see left bar graph above).

I also have a problem with the working memory claim, namely that inferior frontal sulcus = non-syntactic working memory. This claim is based on the assumptions that (i) the distance manipulation taps non-syntactic working memory, (ii) that the hierarchical manipulation does not tap non-syntactic working memory, and (iii) that inferior frontal sulcus showed only a distance effect and no interaction. You might convince me that (i) holds to some extent but both (ii) and (iii) are dubious. Center-embedded sentences are notoriously difficult to process and even if such structures invoke some special hierarchical process, isn’t it also likely that subjects will use whatever additional resources they have at their disposal, like non-syntactic working memory? Regarding (iii) a quick look at the amplitude graphs for the inferior frontal sulcus activation (right graph above) indicates that most of the “distance” effect is driven by the most difficult sentence type, the long-distance center-embedded condition. Thus, the lack of an interaction probably reflects insufficient power.

In short, I think all of the effects Makuuchi et al. see are driven by general processing load. Increased load likely leads to an increase in the use of less exciting processes such as articulatory rehearsal, which drives up activation levels in posterior sectors of Broca’s area. In fact, a recent study that directly examined the relation between articulatory rehearsal and sentence comprehension found exactly this: rehearsing a set of nonsense syllables produced just as much activation the pars opercularis as this comprehending sentences with long-distance dependencies (Rogalsky et al., 2008).

References

BAHLMANN, J., SCHUBOTZ, R., & FRIEDERICI, A. (2008). Hierarchical artificial grammar processing engages Broca's area NeuroImage, 42 (2), 525-534 DOI: 10.1016/j.neuroimage.2008.04.249

Friederici, A. (2006). The brain differentiates human and non-human grammars: Functional localization and structural connectivity Proceedings of the National Academy of Sciences, 103 (7), 2458-2463 DOI: 10.1073/pnas.0509389103

Gibson E (1998). Linguistic complexity: locality of syntactic dependencies. Cognition, 68 (1), 1-76 PMID: 9775516

Makuuchi, M., Bahlmann, J., Anwander, A., & Friederici, A. (2009). Segregating the core computational faculty of human language from working memory Proceedings of the National Academy of Sciences, 106 (20), 8362-8367 DOI: 10.1073/pnas.0810928106

Rogalsky C, Matchin W, & Hickok G (2008). Broca's Area, Sentence Comprehension, and Working Memory: An fMRI Study. Frontiers in human neuroscience, 2 PMID: 18958214

5 comments:

tom said...

Hi Greg,

I read this paper not too long ago, and was also worried about the difficulty effect. It's interesting to compare this paper to a previously published paper that used exactly the same data. In this version, a whole brain analysis is used, and the IFG doesn't quite reach significance. Instead, the significant effects are seen in posterior temporal regions and left dorsal premotor cortex. The pattern of responses are the same in these regions as are reported for the IFG in the PNAS paper, i.e following the pattern of subject-verb dependencies. This suggests to me that there's nothing special going on in IFG, i.e. whatever might be happening there in this study is also happening (to a more marked extent) elsewhere in the brain.

Friederici, Makuuchi, Bahlmann. (2009) The role of the posterior superior temporal cortex in sentence comprehension. Neuroreport 2009;20(6):563-8

Greg Hickok said...

Wow is that for real? I wasn't aware of this other study, never mind it being exactly the same data. That's a good trick. Publish the group data in one journal and if you don't get the effect you like in the group data, do an ROI analysis on your favorite region and publish it in PNAS. Yikes!

tom said...

Hi Greg,

I wouldn't call it a trick, although it would have made more sense to me to include all the analyses in the same paper. My point was that, to my mind, the findings of the Neuroreport paper weaken the logic behind the arguments in the PNAS paper.

Back to the difficulty effects. I find the behavioural results really puzzling, but perhaps this is because my knowledge of the German language is embarrassingly limited. Hopefully someone with more knowledge can explain it. The authors report no significant difference in RTs or accuracy between the centre and non-centre embedded clauses. I would have guessed (at least for English) that centre-embedded (CE) clauses are harder than non-centre (NC) embedded clauses, and take more time to process. In this study, the mean accuracy rate and RTs for short CE and short NC follow this pattern (in a t-test for accuracy rates from this data, I estimate the probability of short CE < short NC as ~ .07). This makes sense to me. However, the accuracy rates for long CE and long NC clauses are identical, and the mean RT for long CE was slightly quicker than for long NC. Most confusing of all, within the CE class, long CE seem to be easier than short CE and mean RTs were 100ms faster for long CE than short CE. Shouldn't this be the other way around? Aren't longer sentences generally harder, or is it just that I'm seeing this from an Anglophone viewpoint?

Greg Hickok said...

By "trick" I mean efficient publication strategy. ;-) As long as they cited the NeuroReport paper in the PNAS paper it is legit. If they failed to cite it (I don't have it in front of me) then it is much closer to a trick in the negative sense.

I don't think the odd performance data is a German language thing. The responses were "off-line": after processing a fraction of the sentences, a comprehension question was presented. It is the response to these after-the-fact comprehension questions that is reported in RTs, etc. The *processing* effects that you correctly expect are already long gone by then because you've already processed the sentence. All you have at that point is a conceptual-semantic representation of what happened. It is no surprise then that performance doesn't reflect processing difficulty. All this behavioral data do is make sure the subjects are paying attention and performing the task. They say nothing about processing load.

Greg Hickok said...

I just looked at the Friederici et al. NeuroReport paper and saw that the Broca's area effect only barely missed significance (p=.053) -- that counts.