Thursday, December 21, 2006

Nature ends public peer-review trial

According to this story, Nature is scrapping its attempt to open up the peer review system.

As the article says,
In the Nature experiment, authors whose manuscripts were selected for traditional peer review could also opt to have them simultaneously posted on the Internet for feedback by rank-and-file scientists. Journal editors then weighed both sides when deciding whether a paper gets published.

The experiment generated high online traffic, about 5,600 page views a week, according to Nature. But it was ultimately canceled because few authors participated and many of the online comments were nothing more than "nice work."

My experience with open peer review (through the journal Atmospheric Chemistry and Physics) is similar. While it seems like a good idea, few articles get any public comments. I suspect this is true for a few reasons. First, it can be dangerous to make negative comments without anonymity (and on ACP comments must be attributed) --- which is why most journals implement anonymous reviewers. Second, it takes a lot of time to write a comment, and most people are simply too busy. Another reason the Nature experiment failed was that they did a terrible job publicizing it.

As a dedicated blogger, I see great value in an open comment system and I suspect that some kind of open system will eventually be adopted. However, I don't think any journal has yet stumbled upon the winning model.