Brainstorm icon

Previous

The Erstwhile Urbanite

Next

Harvard Scholarship on the Web

February 11, 2008, 05:11 PM ET

Peer Review at NIH: A Lottery Would Be Better

For awarding government money for basic biomedical research, would a lottery out perform the traditional system of peer review? A lottery? Ridiculous, when peer review brings knowledge and experience, rather than chance, to the selection process. Maybe not so ridiculous. Read on.

We’re talking mainly about the National Institutes of Health, which presides over the world’s biggest bankroll for health-related research, nearly $30-billion this year. (See a Chronicle story on the matter here.) About half is for grants to individual researchers. As plump as that sum is, there isn’t enough to satisfy all reasonably competent supplicants for grants.

The insufficiency is so great that it’s generally acknowledged that many winners and losers are indistinguishable in terms of capability and the scientific importance of their proposed projects. Moreover, the process for deciding who gets money and who doesn’t is cumbersome, extremely costly, and often meaningless for making wise choices.

The NIH peer review system is one of the wonders of the scientific world. To award that money, NIH relies on some 18,000 reviewers holding small, face-to-face meetings at which they discuss and rate grant applications. Though a cautious, slow transition to electronic communication is under way, the great majority in this army of reviewers attend three meetings per year, usually of two days’ duration, at NIH headquarters, in Bethesda, Md. Travel time for many reviewers adds two days to meeting time. My request for the costs of travel and lodging for this population movement has been languishing at NIH for several months, but I understand that it’s at least a couple of hundred million dollars per year.

At NIH, nine months is the generally reported time for rendering a verdict on a grant application. Efforts are being made to speed up the process, but optimism is not warranted. Off and on, for a long time, such efforts have been made. The National Science Foundation, a sister federal agency, gets its grants out the door in about six months. NSF mainly relies on telephone and e-mail for obtaining evaluations of grant applications. NSF is also overwhelmed by grant seekers. There’s no reason to believe NSF’s decision making suffers from its relatively speedy process.

The disparity between the volume of worthy applications and the availability of NIH grant money is dispiriting for scientists who take peer-review duty seriously. An unknown but apparently rising number are said to decline peer review service as meaningless, given the similarity of winners and losers in the grant derby.

So why bear the logistical burden and expense of a system that’s plummeting into irrelevance and undermining morale in the biomedical research community?

More money for grants is the obvious solution, but that’s not to be in these hard times. The need, then, is to make the most of the available money.

A quick scan by appropriate specialists could easily sift the null applications from the potentially fruitful proposals for using NIH’s scarce money. A lottery drawing could then disburse the available funds. Reliance on chance wouldn’t be inferior to what’s happening now, which, as it turns out, is a game of chance in the guise of informed selection. Moreover, the savings from a lottery could be recirculated to research, providing many millions of dollars for projects that would otherwise go unsupported.

Project selection by lottery? The concept surely grates on the scientific ethos. But, in the spirit of experimentation, it’s surely worth at least a pilot project.

  • Print
  • Comment

Add Your Comment

Commenting is closed.