The Ultimate Rebuttal Letter
Floated in my email inbox recently. Bears blogging.
Dear Editor,
I would like to thank the editorial board and the referees for their comments and contributions to our manuscript. We have carefully considered the comments when rewriting the manuscript, and believe it to be much improved now…
…Oh, screw this. Let’s cut the bull. Mmkay?
Referee #1 did not even bother to read the paper. He basically glanced at the references, realized he was not cited enough to his taste, got pissed off, and attached a Pubmed dump of his papers in the last 10 years. All three of them. There is a reason none of these papers went beyond a single digit number of citations: they suck! Also, I fail to see how a paper discussing semantic distances as applied to an “endoplasmatic reticulum membrane elasticity ontology” has anything to do with my paper. Or with anything of interest, for that matter.
Referee #2 requested reanalysis of our data, using Boyle-Scott statistics. Applying Boyle-Scott statistics to our work would be like draping a hornet’s nest with clingwrap while wearing a bathing suit: a long and painful process which is utterly pointless. B-S statistics are exactly what they are, and if you think I will be bothered to do that, with my grad student finally graduating and taking off, you’re as delusional as Dr. Boyle was when he was researching REM sleep in cannabis-treated amphibians just before he went completely schizo and had to be locked up.
Referee #3 Actually read the manuscript carefully. Which is both commendable and rare. Unfortunately, judging by the comments presented, it was not my manuscript.
Finally, I would request that you as an editor grow a brain. Did you even read their comments before passing them on to me? Shipping out papers to referees, then getting them back, pasting them together and slapping on some boilerplate text from your journal’s editor’s site is not editorial work. In fact, a middle school student that volunteers in my lab wrote up a script yesterday that does just that. We are thinking of installing it in your esteemed journal’s author’s website and waiting to see if this editorial version of the Turing test would pass. We are very optimistic about the results, and we plan to write a paper about them.
Sincerely,
Prof. I. M. Irritated
You point out correctly that peer-reviewing probably does not work. I have gotten three kinds of reviews you refer to in some form or the other ?
Could the reviewers need to be rewarded or penalized someway based on the quality of reviews ? Can/Should the editor play a significant role in this ?
-HR
I think that peer reviewing actually does work. To paraphrase Churchill, it’s the worst form of science communication, except for all the other forms that have been tried from time to time.
There is some implicit reward/penalty system for review quality. Reputation among journal editors is one. Being promoted to editorial board of a journal, networking points, service points counting towards academic promotion, etc. Your idea of rewarding good reviewers more explicitly is interesting, but I am not sure how to do that.
Here is one thought …
Reward reviewers with some “money”, discounts or total reviewer score (an academic measure that is accepted by department head) as a factor of change made to the original submission.
All change cannot be good so the editors can rate the reviewer feedback and decide. The authors could vouch for the reviews but they would probably do it only when their paper or submission is accepted ;)!
We shouldn’t lose sight of the point that peer review doesn’t stop at publication. Papers continue to be criticised or praised (and occasionally even cited ;)) after they are published. I think we should take a longer-term view of the process. Journal peer review is just a filtering process that attempts to ensure some minimum standard prior to publication, though its success in that is obviously debatable. Publication in peer-reviewed journals only implies that some peer review has been done, not that it has necessarily been done well, or that the conclusions or work are correct. I’m sure we’ve all seen a number of godawful papers in very high-ranking journals, and great papers in lower-ranked journals, and could bore each other stiff with the anecdotes.
I’m OK with incorrect or unsound work being rejected, but in general I’d prefer that borderline work was published for community appraisal, rather than consigned to the bin. IMO it’s the community, in citation, repetition and use of the work, that is the real peer review process.
@widdowquin: Yeah that’s why papers should be open access and let you post comments on them like the PLoS journals. =)