Alternatives to peer review

For those interested in the process of peer review, take a look at this interesting article.

For those not familiar with the concept of "peer review," here's a short explanation. Scientific journals will not publish a paper until it has been critically scrutinized by other scientists (usually two or three) who are experts on its subject. In this process, called peer review, the reviewers’ job is to look for any errors or weaknesses – in data used, calculations, experimental methods, or interpretation of results – that might cast doubt on the conclusions of the paper. The process is usually anonymous, so reviewers are free to give their honest professional opinion without fear of embarrassment or retribution.

Peer review is one of the cornerstones of modern science. And succeeding at peer review counts for everything in a scientific career. For scientific work to attract attention and respect, it has to be published in peer-reviewed journals. Proposals for research funding must also go through peer review. For scientists to get and keep jobs and achieve all other forms of professional reward and status, they must succeed at getting their work through peer review.

Because of its central place in science, I'm quite skeptical that non-peer-reviewed journals will be successful. It seems like likely that non-peer-reviewed journal publications will simply not count the same as peer-reviewed publications for things that matter, like tenure decisions, and that people will not publish first-rate work there. Rather, it will be second-rate work that has been rejected from peer-reviewed journals that will end up in the non-peer-reviewed literature.

One of the complaints against peer review is that good science is sometimes held up or even rejected by stubborn or biased reviewers, thus hurting both the authors and the scientific community. My experience is that this is rarely a real problem: if your paper gets rejected by one journal, you can always submit it to another. And an author can always request that a particular person (or two) not serve as peer reviewer. If a paper gets rejected by several groups of reviewers picked by several journals, then it probably doesn't deserve to be published anywhere.

In addition, implementing a non-peer-reviewed journal simply trades one problem for another. While legitimate science might sometimes be delayed or rejected by peer review, a lot of really bad science is correctly filtered by peer review. By eliminating peer review, you will unleash all of the bad science on the community. This seems to me to be a bad idea.

In any event, it looks like the experiment is going to be run, so we'll all see how this turns out.