January 16, 2008

Researchers Demonstrate 'Publication Bias' in Studies of Antidepressants

Researchers have long worried about “publication bias,” the tendency for negative results of clinical trials never to appear in medical journals. Now a team of researchers has demonstrated the bias in studies of antidepressants: In the research literature, it appears that 94 percent of trials had positive outcomes, but in data submitted to the Food and Drug Administration, just 51 percent did.

In a paper appearing in the new issue of The New England Journal of Medicine, the researchers note that the result of such bias is that antidepressants appear more effective than they actually have proved to be.

Pharmaceutical companies must provide the FDA with data from all studies they perform on a drug they submit for approval, including clinical trials with negative outcomes. The researchers, led by Erick H. Turner, of Oregon Health and Science University, collected reviews from the FDA about 12 antidepressant drugs, which were tested in 74 studies involving more than 12,500 patients from 1987 to 2004.

Dr. Turner’s team found that of the 38 studies that the FDA considered positive, all but one had been published. But of the 36 that the FDA considered either negative or questionable, 22 were not published, and 11 were published with positive conclusions.

The result was that 48 of the 51 published studies appeared to be positive, but only 38 of the 74 reported to the FDA produced positive results. At least one study was unpublished or was published with results contrary to the FDA’s judgment for every one of the 12 antidepressant drugs the researchers considered.

Dr. Turner’s group was unable to determine the underlying cause of the bias: whether study sponsors and researchers were failing to submit articles to journals, or journals were turning down manuscripts that reported negative results. —Lila Guterman

Comments

1. i was abused in a FL mentale hospitle

   — aaron silverstein    Jan 16, 06:33 PM    #

2. Labeled “the file-drawer problem” by Rosenthal (*Psychological Bulletin*, v.86 #3, May 1979, p. 638-41), this problem was noticed even earlier. At least we now require the studies to be turned over to the FDA ... but apparently that’s not enough.

   Worst of all, many people consuming drugs don’t even have a choice — children, or people like Aaron, in mental hospitals (although there’s no telling what kind of abuse he personally suffered). Those of us who leapt at the chance to swallow Vioxx or Vytorin voluntarily should be kicking ourselves for getting suckered by ads and pharma pushers.

   — swish    Jan 17, 09:37 AM    #

3. Besides researchers’ obvious desire to find positive results, are these studies paid for (indirectly/directly) by the manufacturers? Organized bribery if so (and no surprise!)

   — David Zetland    Jan 17, 11:27 AM    #

4. Drug studies are most often financed by the manufacturer (research is often given as the reason for the high cost of drugs). I am glad that they have to give the results to the FDA, but don’t know of any obligation to have it published. I think they have an ethical obligation to do so, but that is just my opinion.

   — Dodd    Jan 17, 12:12 PM    #

5. Do we know for sure that the “nonpublication” resulted from “nonsubmission?” I’m wondering if the failure of something to happen—the “no statistically sigificant effect” is considered “newsworthy.”—as if this would be publishing an experiment that “didn’t work.” This would indeed be some sort of psychological bias to correct for, but certainly not one unique to medically significant pharm questions.

   — Vickie    Jan 17, 12:31 PM    #

6. Vickie, that’s a very interesting question. Once a drug is on the market, negative and null results certainly do become newsworthy … but while in the testing stage? Maybe you’re right — medical journals don’t publish everything, and even if the research paper was never submitted anywhere, the authors could argue that it had little chance of being accepted, and that it wouldn’t add to the universe of scientific knowledge.

   What we need is for all the research the FDA gets to be posted publicly. If the FDA isn’t able to handle the job of evaluating the research before approving drugs for the market, then let the rest of us have a look at it.

   — swish    Jan 17, 04:26 PM    #

7. While it’s good that this all is getting some sunlight on it, the bigger story continues to be Eli Lilly misrepresenting the increased risk of suicide they found during clinical trials of Prozac (check out David Healy). Two decades and numerous SSRIs later, I’m supposed to be wowed by this?

   — Micah    Jan 17, 05:04 PM    #

8. A policy of open publication on a central data base might make the most sense.

   — jms    Jan 20, 11:20 AM    #

Previous: Sean O'Keefe Steps Down as Louisiana State U. Chancellor
Next: Jury Finds U. of Phoenix Parent Company Liable for $280-Million