Poor review

Firstly, let’s namecheck the wittiest punner in stats for that title, Stephen Senn.

This recent post on Andrew Gelman’s blog is essential reading. I suspect my readers are all over there too, but I’ll mention it here because of this point in wrapping up:

Peer review can serve some useful purposes. But to the extent the reviewers are actually peers of the authors, they can easily have the same blind spots. I think outside review can serve a useful purpose as well.

I’ve seen this a lot in my life as a medical (read ‘health and social care more broadly, with a dash of education’) statistician. There are distinct tribes of healthcare professionals and they do things, including research designs, analytical methods and communicating findings, in their own sweet way. There’s generally no reason, it’s just custom and ritual. If you don’t fit that mould to some extent, you get rejected. Often, I find myself consciously peppering the paper / slides with some shibboleths that will ease my journey to REFland. (Of which, sample size calculations for anything that isn’t a randomised controlled trial is the most common, although I am no stranger to the Totally Unnecessary Reporting Diagram (TURD). I draw the line at Cohen’s D though; D stands for d’oh.)

‘Outside review’ reminds me of the idea of ‘strong inference’, or having your worst enemy analyse your data too and see if they can destroy your conclusions. You don’t have to go that far though, you could just make sure that reviewers extend beyond the specialism and profession of the authors to break that parochialism and question the unquestionable.

People Against Goodness And Normalcy

Essentially, if they can’t understand it, then it’s not written well. I don’t accept any argument that the subject is just too complex for outsiders – because the authors’ interests were once upon a time confined to Lego or the Smurfs, so it must be possible – nor do I claim to have got it right myself – it’s a constant challenge to pitch Bayesian latent variable models just so for a subject-expert audience.

I don’t know about you, but I enjoy reading academic papers outside my own field (OK, no critical theory please, but I’ll consider pretty much anything else). Maybe I should start an occasional series of randomly selected academic papers here, or maybe I just don’t have time for that.



  1. I sympathize with the feelings. I don’t remember where, but I once read a great article on the improvised “internet” expert whereby anyone can be an expert by virtue of Googling things. The “real” experts end up having to justify and explain fundamental concepts that the Google expert does not know. Frustrating.

    What I would welcome is to have a peer review system capable of matching experts in one field with experts in another. So for instance, if you do survival analysis in cancer research, a possible peer could be a an actuarial person or even a reliability engineering researcher. Imagine the cross collaboration, knowledge transfer among fields that are otherwise so distant from each other.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s