What can we learn from the processing of syntactic violations?

There is an interesting discussion brewing in the comments of one of my previous posts that I think is worthy of the front page.  It was sparked by a question from Jeroen van Baar:

Greg, I think one of your last comments here captures the issue: "We are dealing with the fact that Broca's area doesn't get involved with sentence processing unless the stims contain violations or things get really difficult (working memory? cognitive control?)." (March 14, 2011).
If we want to learn about the processing of syntax, we need to make sure the brain is engaged in syntax analysis, and I guess there are two obvious cases in which this happens; 1. when syntax is violated and 2. when syntax analysis is needed in order to extract meaning from a sentence. So if a sentence is not attended and meaning does not need to be extracted, the auditory stimulus will flow into your brain but higher-order processing will not take place, which leaves an absence of syntax-related activation. Likewise, if a sentence has no meaning at all, as in your jabberwocky stimuli, syntax plays no role in understanding the sentence anymore, so your brain will just not bother. What I'm trying to say, I guess, is that syntax serves meaning (in good Chomskian tradition), and meaningful stimuli may just engage that syntax-processing unit we are thought to have. 
As for music: same story? What do you think?

My response was this:

Jeroen,You seem to be suggesting that syntactic analysis may not be utilized for simple, grammatical sentences. I think there is a paradox hidden in there though. You suggest that one condition in which the brain is engaged in syntactic analysis is when syntax is violated. But presumably there has to be some mechanism in the brain telling you when a violation has occurred. This implies that syntactic analysis is being carried out even when no violation occurs. So even though, "The dog chased the cat" is simple and contains no violation, the fact that we readily detect a difference between that sentence and, "The dog chase the cat" means that syntactic analysis is being carried out, no?

Jeroen countered:

Greg, You're right. A syntactic monitoring system must be active at all times. But don't you think that its activity will spike when a violation is detected? In neurolinguistics this seems to be a common paradigm: in both EEG and fMRI studies, conditions of no violation are often subtracted from conditions of violation to leave the activity that is specifically concerned with the analysis (or repair) of a syntax error. I think that with such an approach, we may be more successful in identifying a musical syntactic processing system, if there is one. 

Your findings, of course, suggest that if any overlap is found in activity elicited by musical and linguistic syntax violations, this overlap is not a sign of a shared syntactic integration system, but rather of a general violation-activated system of some kind. This is contradicted by a couple of studies that found interactions specifically between syntax violating conditions in music and language, and not between other violation conditions, such as timbre-related surprises and semantic oddballs (c.f. Koelsch et al. (2005), Interaction between Syntax Processing in Language and Music: An ERP Study, J. Cogn. Neurosc. 17(10): 1565-77; and Slevc et al. (2009), Making psycholinguistics musical: Self-paced reading time evidence for shared processing of linguistic and musical syntax, Psychonomic Bulletin & Review 16(2): 374-81). 

All in all, do you agree that a study comparing violation vs. control within-mode (language or music) and within-subjects with fMRI could provide useful insights? 

Here's the issue I'd like to bring up for discussion here: "don't you think that [syntactic] activity will spike when a violation is detected?"

This is the underlying assumption of all the studies that use the violation paradigm, but it is an empirical question and I'm not convinced we have a clear answer on this.  Does syntactic theory predict a spike in syntactic computation when a violation occurs?  Not really.  In fact, you might argue that the syntactic mechanism shuts down and something else takes over!


The problem I have with violation studies is that the occurrence of a violation is confounded with other processes, some of which may not be syntactic at all.  E.g., conflict resolution, various forms of working memory, the probability of your subject talking to him/herself may increase ("Was that a violation?  Should I push the button?), etc.  In short, I don't know what is being measured in the response to a violation so I don't how to interpret a neurophysiological response triggered by such a violation.  If you want to map the neural system involved in violation processing, then fine, study violations.  But this that really what we're after here?  Or are we trying to understanding the circuits involved in syntactic computations as they are normally carried out in grammatical sentences?  I'm interested in the latter and so I'm not interested at all in the response to violations.  I think it is misleading.

This is a theoretical concern but it is backed up by empirical results.  I think we all agree that listening to structured sentences involves syntactic computation.  If we look at that activation pattern associated with listening to sentences contrasted with rest or with listening to scrambled sentences or spectrally rotated versions of those sentences, we do not typically find activation differences in Broca's area.  However, the most robust site of activation in violation studies is Broca's area.  Given these two facts, how can we say that the violation response reflects syntactic computation?

INTERNATIONAL SYMPOSIUM ON IMITATION AND CONVERGENCE IN SPEECH (ISICS 2012) Aix-en-Provence, France, 3-5 September 2012

INTERNATIONAL SYMPOSIUM ON IMITATION AND CONVERGENCE IN SPEECH (ISICS 2012) Aix-en-Provence, France, 3-5 September 2012

 

First call for communications

mail: isics2012@lpl-aix.fr

website: spim.risc.cnrs.fr/ISICS.htm

 

OVERVIEW

In the course of a conversational interaction, the behavior of each talker often tends to become more similar to that of the conversational partner. Such convergence effects have been shown to manifest themselves under many different forms, which include posture, body movements, facial expressions, and speech. Imitative speech behavior is a phenomenon that may be actively exploited by talkers to facilitate their conversational exchange. It occurs, by definition, within a social interaction, but has consequences for language that extend much beyond the temporal limits of that interaction. It has been suggested that imitation plays an important role in speech development and may also form one of the key mechanisms that underlie the emergence and evolution of human languages. The behavioral tendency shown by humans to imitate others may be connected at the brain level with the presence of mirror neurons, whose discovery has raised important issues about the role that these neurons may fulfill in many different domains, from sensorimotor integration to the understanding of others' behaviour.

 

The focus of this international symposium will be the fast-growing body of research on convergence phenomena between speakers in speech. The symposium will also aim to assess current research on the brain and cognitive underpinnings of imitative behavior. Our main goal will be to bring together researchers with a large variety of scientific backgrounds (linguistics, speech sciences, psycholinguistics, experimental sociolinguistics, neurosciences, cognitive sciences) with a view to improving our understanding of the role of imitation in the production, comprehension and acquisition of spoken language.

 

The symposium is organized by the laboratoire Parole et Langage, CNRS and Aix-Marseille Université, Aix-en-Provence, France (www.lpl.univ-aix.fr). It will be chaired by Noël Nguyen (LPL) and Marc Sato (GIPSA-Lab, Grenoble), and will be held in the Maison Méditerranéenne des Sciences Humaines.

 

INVITED SPEAKERS

Luciano Fadiga, University of Ferrara, Italy

Maëva Garnier, GIPSA-Lab, Grenoble, France

Simon Garrod, University of Glasgow, United Kingdom

Beatrice Szczepek Reed, University of York, United Kingdom

 

CALL FOR PAPERS

Papers are invited on the topics covered by the symposium. Abstracts not exceeding 2 pages must be submitted electronically and in pdf format by 15 April 2012. They will be selected by the Scientific Committee on the basis of their scientific merit and relevance to the symposium. Notifications of acceptance/rejection will be sent to the authors by 31 May 2012.

 

IMPORTANT DATES

- 15 April 2012: Abstract submission deadline

- 31 May 2012: Notification of acceptance / rejection

- 30 June 2012: Early registration deadline

 

SCIENTIFIC COMMITTEE

. Patti Adank, University of Manchester, UK

. Martine Adda-Decker, laboratoire de Phonétique et Phonologie, Paris, France

. Gérard Bailly, GIPSA-Lab, Grenoble, France

. Roxane Bertrand, laboratoire Parole et Langage, Aix-en-Provence, France

. Ann Bradlow, Northwestern University, Evanston, USA

. Jennifer Cole, Department of Linguistics, Urbana-Champaign, USA

. Mariapaola D’Imperio, laboratoire Parole et Langage, Aix-en-Provence, France

. Laura Dilley, Department of Psychology and Linguistics, Michigan State University, USA

. Sophie Dufour, laboratoire Parole et Langage, Aix-en-Provence, France

. Carol Fowler, Haskins Laboratories, New Haven, USA

. Jonathan Harrington, University of Munich, Germany

. Jennifer Hay, University of Canterbury, Christchurch, New Zealand

. Julia Hirschberg, Columbia University, New York, USA

. Holger Mitterer, Max Plank Institute for Psycholinguistics, Nijmegen, The Netherlands

. Lorenza Mondada, laboratoire ICAR, Lyon, France

. Kuniko Nielsen, Oakland University, Rochester, USA

. Noël Nguyen, laboratoire Parole et Langage, Aix-en-Provence, France

. Martin Pickering, University of Edinburgh, UK

. Marc Sato, GIPSA-Lab, Grenoble, France

. Jean-Luc Schwartz, GIPSA-Lab, Grenoble, France

. Véronique Traverso, laboratoire ICAR, Lyon, France

. Sophie Wauquier, Université Paris 8, Saint-Denis, France

Computational neuroanatomy of speech production

For your Happy New Year reading enjoyment, let me point you to my just (online) published synthesis of computational, psycholinguistic, and neuroanatomic research on speech production: Hickok, G. (2012). Computational Neuroanatomy of Speech Production, Nature Reviews Neuroscience.  The aim was to shatter barriers between the motor control folks, the psycholinguists, and the neuroscience oriented researchers studying speech production.  This integration has some interesting consequences (in my view).  Here are a few:

1. Speech motor control is hierarchically organized (no big surprise) with an auditory-(pre)motor circuit representing a relatively higher level and a somatosensory-motor circuit a relatively lower level.

2. The auditory grounded circuit primarily deals in units the size of syllables whereas what we normally think of as segmental units (~phonemes) are processed primarily in the lower-level somatosensory based circuit.  Yes, I'm arguing that "phonological representation" is distributed over auditory and somatosensory cortex.

3. Phonological encoding, in the sense of typical two-stage models of speech production is achieved in the context of a state feedback control circuit (from the motor control tradition).

4. Efference copy signals as they are currently conceptualized in the motor control literature do not exist (let's see what kind of push back I get on this one!).  That is, the motor controller does not issue a copy of the command that it has executed.  Rather, motor to sensory feedback is part of the motor planning process from the start.  In other words, in my view, the "efference copy" is an iterative feedback loop that enables sensory systems to be a part of the programming of the movement rather than just evaluating the outcome of movement commands. This conceptualization integrates the notion of motor planning, efference copies, forward prediction, and error correction into one mechanism.  In addition, this computational architecture solves the problem of how both internal and external feedback monitoring can be achieved by the same network even though the timing of the two feedback sources differ.  I present a simply simulation to demonstrate the feasibility of these assumptions.

5. Forward predictions are instantiated computationally as inhibitory inputs to sensory systems.

6. Conduction aphasia and apraxia of speech involve disruption to two different components of the same hierarchical level of state feedback control, the relatively higher level auditory-pre-motor circuit.

7. Sensory representations are central to the motor planning process and explain the tight interaction between sensory and motor speech systems.  It is a sensory theory of speech production in a sense as opposed to a motor-oriented theory of speech perception.

I would love to get your thoughts on this paper.  There's lots in here to discuss/argue about and it will be fun to debate some of the data and/or theoretical claims.

Doctoral position in General Linguistics at the Johannes Gutenberg University Mainz, Germany

The Emmy Noether Research Group ''Neurolinguistic Foundations of Information Structure'' (funded by the German Research Foundation, DFG) headed by Petra Schumacher is seeking to fill the position of a doctoral researcher. Starting date: February 1, 2012 or soon thereafter. 
The doctoral position is part of a neurolinguistic project that investigates language comprehension at the interface of syntax, semantics and information structure (http://www.linguistik.uni-mainz.de/schumacher/research-group/). The doctoral researcher will work within our group on a project that investigates referential processing in German. The project will either extend previous research with unimpaired comprehenders or investigate referential processing in aphasia patients. The primary experimental method employed will be event-related brain potentials. 

The ideal candidate should have a degree in linguistics or a related discipline. S/he should have a keen interest in experimental work, as well as in syntactic and pragmatic theory. Experience with event-related brain potentials and/or language impairment research will be a plus. Knowledge of German would be beneficial. The successful candidate will work towards a PhD in Linguistics from the University of Mainz on a topic related to the research group.

The position has an initial term for two years, with the possibility of extension contingent on future funding. The salary and social benefits are determined by the German pay scale for state employees (EG 13 TV-L 50%).

Interested candidates are invited to send their application materials electronically to petra.schumacher@uni-mainz.de. Pdf files are preferred. Applications can be written in German or English and should include CV, contact information of at least two referees, and a brief statement of research interests.

Applications received by January 06, 2012 will receive full consideration, but the search continues until the position is filled.

For more information contact Petra B. Schumacher by email at petra.schumacher@uni-mainz.de

open-rank faculty position in Cognitive Neuroscience at Florida International University in Miami, FL

Position Title: Cognitive Neuroscience (Open-rank). We seek an outstanding researcher with expertise in fMRI, EEG, or TMS methodologies. Applicants with interest in developmental or adult cognitive neuroscience are encouraged to apply. Area of focus is open, but we especially welcome applicants with a focus on memory, language, executive function, or multisensory processing. Address inquiries to Dr. Bennett Schwartz, Chair, Neuroscience Search, schwartb@fiu.edu.

THE DEPARTMENT OF PSYCHOLOGY AT FLORIDA INTERNATIONAL UNIVERSITY: seeks applicants for five faculty positions, to begin fall 2012, including open-rank positions in Behavior Analysis, Cognitive Neuroscience, Developmental Science, and Quantitative Methodology and an instructor level position in Research Methods. Successful candidates will join a growing department with nationally prominent programs and faculty. The Department offers Ph.D. programs in Child-Adolescent Clinical Science, Developmental Science, Industrial Organizational Psychology and Legal Psychology and M.S. programs in Behavior Analysis and Counseling Psychology. A Ph.D. in Psychology or related area is required for all positions. For open-rank positions, preference will be given to applicants with strong research credentials and demonstrated potential to obtain external funding. Interested candidates should send application materials to fiupsych@fiu.edu with the position title in the subject line. Include curriculum vitae, recent publications or reprints, contact information for three potential references, and a letter describing research interests. Review of applications will begin November 1 and continue until the positions are filled. For more information about the Department, visit our website: http://casgroup.fiu.edu/psychology/. FIU is a member of the State University System of Florida, serving a diverse population of over 40,000 students, and is an Equal Opportunity/Equal Access/Affirmative Action Employer.

Why autism has nothing to do with 'broken mirrors'

I've argued that the mirror neuron theory of action understanding is backwards.  Mirror neurons do not fire because they are critically involved in action understanding (the typical claim), they fire because perceiving an action is critically involved in selecting actions for movement (Hickok & Hauser, 2010).  This point has broad implications for any theory that builds on the mirror neuron theory of action understanding.  I've had plenty to say about the broader implications in the domain of speech (e.g., Hickok, 2009; Hickok, et al. 2011; Lotto, et al. 2009); now it's time to take stab at the claims made beyond speech.

One of the most prominent and potentially important extrapolations of the mirror neuron theory of action understanding concern autism spectrum disorder (ASD).  The "broken mirrors" hypothesis of ASD, exemplified by a Scientific American article by V.S. Ramachandran, is built on the following logic.

1. The mirror system allows us to understand the action of others.
2. The mirror system, by extrapolation, allow us to understand the emotions, intentions, and perspectives of others.
3. ASD involves a lack of sensitivity to the emotions, intentions, and perspectives of others (in particular a lack of empathy).
4. Therefore ASD results from functional disruption to the mirror system, that is, from "broken mirrors".

Since I've argued that assumption #1 is false, the logic simply falls apart.  However, let me attack another of these assumptions, namely #3, that ASD involves a lack of sensitivity to emotion, etc.  I'm going to argue instead that ASD involves, in fact, hyper-sensitivity to emotional states, both their own and others.  This hypothesis is not new.  In fact, Henry Markram, Tania Rinaldi, and Kamila Markram proposed exactly this in their 2007 paper titled, The Intense World Syndrome -- An Alternative Hypothesis for Autism. I would just like to underline their perspective here.

An analogy is useful for seeing why hyper- rather than hypo-sensitivity makes more sense.  Imagine a person who is hypo-sensitive to sound.  Is such a person more or less likely to walk into very loud environments?  More likely!  If you are less sensitive to sound you might actually prefer loud environments because that's how you get your acoustic sensation to a normal level. Conversely, a person who is hyper-sensitive to sound is going to avoid loud environments because it hurts.

Now consider the same scenario translated to the social/emotional domain.  For starters let's agree that a large part of one's emotional stimulation comes from social situations.  Not all emotional stimulation comes from the social domain, to be sure: I can get pretty emotional when I can't figure out how to fix my leaky faucet.  But we get a lot more emotional and more often when a family member is sick or injured, or when a colleague rolls his or her eyes when we try to make a point, and so on. Now, imagine a person who is hyper-sensitive emotionally. Is such a person more or less likely to engage in social situations?  Less!  For someone who is hyper-sensitive to emotion, engaging in a normal social situation would be like walking into an excessively loud environment: it's uncomfortable and causes an avoidance response. ASD individuals may avoid social interaction not because they lack empathy but rather because social interaction is simply too stressful.

Some data:  A characteristic of autism is decreased gaze fixation, particularly on faces.  Correspondingly, the fusiform face area (FFA) appears to be less active in autistic individuals during face processing tasks.  This might be interpreted as social indifference, a lack of interest in faces.  However, another interpretation is that looking at faces, which are a major source of emotional information, is stressful for individuals who are hyper-sensitive to emotion.  An imaging study of face perception published by Dalton et al. in 2005 supports this view.  This study found that the autistic group did indeed spend less time fixating the eyes and that there was less activity overall in the FFA for the autistic than control group.  However, this is not all that surprising because if you spend less time looking at a stimulus, your brain activation in regions sensitive to that stimulus will naturally be lower.  And in fact, when Dalton et al. looked the correlation between FFA activity and gaze duration they found a strong and positive correlation in autistic subjects.  In other words, the FFA of autistic folks is working fine, it's just that they spend less time looking at the stimulus.  More importantly, they report that amygdala activity was strongly correlated with gaze duration in the autistic group but not in the control group.  Interpretation: looking at faces is more emotionally arousing in autistic than control individuals.

Hyper-sensitivity to emotion is perfectly consistent with the well-known sensory sensitivity noted in ASD.  In other words, there are other reasons to believe that hyper-sensitivity is a key feature of the syndrome across the board.

The mirror neuron folks got it perfectly backwards again.  You don't have to take my word for it though.  Here are some links to essays written by individuals with ASD or by a parent.  The links were provided by Morton Gernsbacher, an author on the study referred to above.  I found them particularly illuminating.

http://www.autismandempathy.com/?p=483
http://www.autismandempathy.com/?p=478
http://www.autismandempathy.com/?p=471

References

Dalton, K., Nacewicz, B., Johnstone, T., Schaefer, H., Gernsbacher, M., Goldsmith, H., Alexander, A., & Davidson, R. (2005). Gaze fixation and the neural circuitry of face processing in autism Nature Neuroscience DOI: 10.1038/nn1421

Hickok, G., Eight problems for the mirror neuron theory of action understanding in monkeys and humans. J Cogn Neurosci, 2009. 21(7): p. 1229-43.

Hickok, G. and M. Hauser, (Mis)understanding mirror neurons. Curr Biol, 2010. 20(14): p. R593-4.

Hickok, G., J. Houde, and F. Rong, Sensorimotor integration in speech processing: computational basis and neural organization. Neuron, 2011. 69(3): p. 407-22.

Lotto, A.J., G.S. Hickok, and L.L. Holt, Reflections on Mirror Neurons and Speech Perception. Trends Cogn Sci, 2009. 13: p. 110-114.

Lab Research Assistant position at UCSF Speech Neuroscience Lab

UCSF is a world-class research institution with a wide array of scanner facilities that includes MRI (both 3T and 7T systems) as well as a 275-channel whole-head MEG/EEG scanner. There is also an active program of research using intracranial ECoG recordings from epilepsy patients. Here at the Speech Neuroscience Lab, we make use of these technologies to investigate the neural basis of speech motor control. The research focus of the lab is investigating the neural basis of feedback processing in speech production, but other ongoing projects in the lab include studies of sequential speech production, as wells as studies of speech motor disorders like spasmodic dysphonia and stuttering.

We are looking for a research assistant to join our research group for a 1-2 year stint, with possibility of extension. Our ideal candidate is a recently graduated undergraduate engineering student who is interested in running human imaging and psychophysics experiments and also has reasonable programming skills in MATLAB.

Start date for the position would be as soon as possible

Those interested in applying should contact Prof. John F. Houde (houde@phy.ucsf.edu)