Friday, June 19, 2009

Dispatches from the front of irrational reviewing

Here is a comment taken from a review that a friend just received. Given the reviews I have gotten over the years -- some on papers that Greg and I have written jointly -- I must admit that it takes the edge off a bit to see my friends get hassled this way ... :-) Of course, we all experience this kind of irritation. And it goes without saying that the bad reviews one receives about one's own work are always the most dumbass and irrational.

This stuff, though, is good, a real gem I just saw in a grant review: "they set their goals too high attempting to reveal the "holy grail" of cognitive neuroscience, the brain mechanisms of a particular psychological function."

HUH?? All I can think of is John McEnroe's line, yelled at the tennis umpire: "You can't be serious!! You can't be serious!!!!" My friend, a productive and creative scientist who has done new and cool stuff in cognitive neuroscience, has plenty of street cred. So where a reviewer gets off with this kind of comment is hard to imagine. My mirror neurons and my empathic system are unable to recreate this level of bs.

What was the reviewer hoping for? What is the credible alternative?? Are we looking, instead, for, say, 'just some vague hints concerning some general function'??? What would that even mean? If cognitive neuroscience reviewers think that ambitious and theoretically well-motivated research violates their religious predilections because of the holy-grail-iness of the research aim, they should recuse themselves.

Aren't specific mechanistic hypotheses about specific psychological functions (and their parts) exactly what we should be looking for? I thought we were in the business of identifying the 'parts list' of the mind/brain and figuring out how the parts relate and underlie perception, action, memory etc etc etc ...

And just to round things out ... Here a comment from a referee on a manuscript submission (different genre, different friend) -- equally unusual commentary, though: "The authors' disdain for language as it is used also becomes apparent when they construct their stimuli." Hmmm, really? Wow. Disdain for language, eh? In the stimuli. Right. This reviewer clearly has to stick with decaf ...

Yours from the grail-trail,


yisroel said...

Calls to mind a certain scene from 'Monty Python's Holy Grail'. 'Tiny brained wipers of other people's bottoms!'

Anonymous said...

While the tone of the second referee's comment is certainly unpleasant, and the quoted comment is pretty useless unless it was fleshed out, it's not necessarily absurd. It clearly refers to "disdain for language _as it is used_" - not "disdain for language" per se. I've read enough papers with linguistic stimuli that are constructed along theoretically motivated lines but are so patently unnatural - so unlikely ever to occur in "language as it is used" - that (a) they should probably have led the authors to question the underlying theory and (b) the ecological validity of any results obtained with them should be called into question.

The first reviewer's comments, though, seem pretty idiotic.