Extraordinary claims attract extraordinary blogging
Since its publication, the paper about bacteria using arsenic instead of phosphorous has been criticized from several different angles. First for the media pre-publication stoking, which lead many journalists to speculate about microbes from Titan while the paper was still embargoed (titanic microbes?), when ultimately it was revealed that we are dealing with earthlings, although with a rather unusual biochemistry. This let-down was only enabled by a rather unfortunate build-up which should not have occurred in the first place. Playing the media game to draw attention to science is good and welcome, and scientists should probably verse themselves a lot more in the skill of properly communicating their findings to the general public. However, to build expectations so high, that once the actual findings are revealed,the pre-embargo expectation (Life on Titan!) has led to an undeserved disappointment ( Arsenate-based life on Earth… yawn…), where actually, arsenate-based life is really cool!
But now the science in the article itself is coming under fire. Several blog posts by notable microbiologists and biochemists have questioned the claims made in the paper. To sum those up: yes, the microbes contain arsenate, the can grow on arsenic-rich media but there is no convincing evidence that arsenic gets incorporated into DNA, much less other molecules that use phosphate. Because this research is so much in the spotlight, the comments on it are in the spotlight too. I believe we will see some very interesting correspondence on the website and in the upcoming issues of Science.
Which brings me to the point of this post: is the peer-review publication culture undergoing a reform? The arsenate bacteria article itself went through the peer-review mill, which means that at least three scientists which are credited as experts in the field have looked at it and given it a clean bill of health. But once it got published, hundreds of microbiologists and biochemists had a look, and many were less than convinced of some of its claims. So which is better for the process of peer-review: three anonymous referees before publication, or 100 after? Or maybe we should use both?
A personal example: I recently published a paper in PLoS Computational Biology, which went through two pre-publication review cycles making it much better. However, even after those revisions an error (minor, fortunately) slipped through. A reader emailed me about it, and I immediately went to PLoS-CB‘s site and addressed that error as an inline comment in the paper. This mechanism provided by PLoS is laudable: I wish it were used more, and that other journals could provide it.
So, post-publication peer-review seems to be a good thing: it quickly identifies issues with the science, and helps to fix them. So why is it not done more? Well, for one, there is the lack of anonymity. Post-publication commentators do not have the luxury of the official peer-reviewers of hiding their identity. Another is lack of credit: while some credit is given for pre-publication review, which is recognized as service rendered to the community, none is given yet for post-publication review. But why not? It is scientists like Rosie Redfield, Larry Moran, Jim Hu and others who did a great public service by taking the time to carefully read and then publicly critique the paper. And in case there are still doubters of the value of science blogging, please read this piece by Larry Moran and for blogging as a career enhancer in science, “10 benefits for my career of blogging/ tweeting etc.) #fb” by Jonathan Eisen.
Where am I going with this? I’m not sure. But it seems like the fallout from the arsenate bacteria paper brings to light a new kind of science culture, in which post-publication critiques in expert science blogs are given. Perhaps all this energy could be harnessed to provide a better publication environment for research papers. This has been going on for some time, as many science bloggers emphasize paper critique. But high profile incidents like the arsenate bacteria bring the value of post-publication review to light. To paraphrase a quote by Carl Sagan which was mentioned at the press conference held when the paper was published: “extraordinary claims attract extraordinary blogging”.
Wolfe-Simon F, Blum JS, Kulp TR, Gordon GW, Hoeft SE, Pett-Ridge J, Stolz JF, Webb SM, Weber PK, Davies PC, Anbar AD, & Oremland RS (2010). A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus. Science (New York, N.Y.) PMID: 21127214
“Post-publication commentators do not have the luxury of the official peer-reviewers of hiding their identity.”
In a year, let’s email Rosie Redfield, Larry Moran, and Jim Hu to see if they think their signed critiques of the paper o their blogs have been pluses or minuses. Because so far, I haven’t seen anyone doing anything but praise them.
well, one reason for not allowing post-publication edits is for reproducibility: if I carry out an analysis based on the results published in a paper, and then the paper gets modified and I don’t notice it, then it will be difficult to understand what I was doing.
Moreover, if a paper gets modified, all the papers that cite it should be revised as well. It is better if major revisions are published as different documents, to me it feels more reproducible.
Anyway, thank you for the post, I wasn’t following all the debate on the Arsenic-life topic.