January 25, 2008

Scientists Are Satisfied With Current System of Peer Review, Survey Finds

A survey of more than 3,000 academic scientists worldwide has found that an overwhelming majority believe peer review in journals is necessary.

Amid recent debate about the performance of peer review, the survey, performed by Mark Ware, a publishing consultant, found that nine out of 10 scientists believe peer review improves the quality of published papers, and just one out of eight is dissatisfied with the current form of review.

Still, more than half of those surveyed said they would prefer both reviewers and authors to be anonymous; currently, most journals make authors’ names known to reviewers. The opposite type — open peer review, where both names are revealed — has gained some support in recent years, but almost half of respondents to the survey said that a journal’s disclosure of their name would make them less likely to review manuscripts for it.

Those surveyed supported review online after publication, but as an addition to, rather than a replacement for, pre-publication peer review.

The survey also found that the average peer review takes 80 days, that the average number of manuscripts each reviewer reads yearly is eight, and that each reviewer tends to spend five hours on a manuscript over the course of three to four weeks.

The survey was paid for by the Publishing Research Consortium. —Lila Guterman

Comments

1. I am curious to know how the author would respond to the charge of the potential bias in reporting that people who have benefitted from the current peer-review are happy with the current system? Additionally, how does this report intersect with recent discussions about the publication of papers with primarily “positive” findings and the perceived lack of balance in reporting of scientific results?

   — Manuel Barrera    Jan 25, 04:10 PM    #

2. Barrera hits the nail on the head. As in all human enterprise, the great majority of people who have prospered under this system want to retain it, perhaps with a cosmetic touch-up here and there.

   Some form of peer review is essential, but science and society would benefit if we could find a way to avoid the enfeebling conservatism of the current approach. It is too difficult and too expensive to get great ideas — or even good but unusual ideas — through the community of self-satisfied journeymen that does most of the reviewing.

   — S. Britchky    Jan 25, 05:28 PM    #

3. The issue never has been “peer review or no peer review” so these survey results mean nothing at all and the survey itself was not optimally designed from this point of view.

   The issue is often (not always) whether peer review, especially with partial names revealed, stifles good research and spawns academic conformity.

   A second issue, also missed by the method of surveying professors about “their” satisfaction with peer review, is the immense volume of unread journal articles. Entire disciplines now produce research no one at all uses except for gaining tenure. Only professors would be “satisfied” with such an outcome!

   The underlying biggie question (third one treated here) is the role of peer review journals in the overall knowledge flow process of US and other societies. Is knowledge actually flowing at all or just clotting up in unread repositories for peer review?

   Surveys that address these more important questions cannot be restricted to professors alone and cannot care much whether professors enjoy being unread.

   Tired of this continual ineffectual wimpy professorial consideration process on peer review, myself, a year ago I tested one alternative, putting a ridiculous article of mine (200 pages long, very densely written) in PDF form out to 12,000 professors in 20 departments I did not know, all of them somehow interested (as indicated by their past publishings or conference presentations) in the topic of my paper “creativity”. In 2 months, 60 days, of sending 200 emails to strangers per night, I got 12,000 copies of my un-peer-reviewed article out across 41 nations of universities. The results were approximately 100 times results from any paper of mine in past peer reviewed journals: 116 citations and still counting (within 9 months), 7 conference invites, 2 book chapter invites, 1 book invite, 47 paid speech invitations, 3 long term collaborations on existing grant projects, 11 invites to co-propose new grant projects, etc. No peer review would have allowed 200 pages of stuff on a topic even if densely written. No peer review journal would cover 20 different disciplines sharing a topic of interest (a slight but not great exaggeration). No peer review would have left many of the contents of my article that implicitly undermined 3 of the 5 leaders in creativity research. Just for my own satisfaction, I was able with this tiny statistically meaningless experiment to prove to me that self publishing on the net—in some limited and ridiculous cases—outperforms by wide margins, peer review in the extant journals of a field. I suspect there are a few hundred other such tiny experiments out there by others. I would be very interested in hearing their results.

   The question is not “peer review or no peer review” but “peer review only or peer review plus other avenues of publishing less conformist and with topics less chopped down to small enough size to make ridiculously unneeded and un-insightful maths work”.

   — Richard Tabor Greene    Jan 25, 09:19 PM    #

Previous: Judge Reinstates Lawsuit Against UC-Irvine Professor
Next: Pennsylvania Student-Loan Agency Faces Demand for $15-Million Repayment