Medical Journals Fight Burying of Inconvenient Research

A dozen leading medical journals have announced that they plan to refuse publication of clinical studies unless those studies were publicly registered ahead of time. Part of the intention is to prevent researchers from privately doing multiple studies and then selectively releasing for publication only those which yield favorable results. There are many other journals which have not signed on to this plan, however, and it remains to be seen what will happen. Personally, I'm surprised it's taken this long; as Karl Popper wrote, "what distinguishes the scientific approach and method from the prescientific approach is the method of attempted falsification."

Comments

Great idea!

[keiferb]

Now, if only this could be extended to include IT studies. I wonder how many studies people run that show that their product isn't that hot.

On second thought, they're all funding their own studies, so it's probably a non-issue.

Finally

[hubs99]

The is great news. If the JAMA can do it then so should everyone else. In addition to the NIH proclaiming that they are hoping to force Researches to publish to public domain, research and reporting of research is moving in a positive direction.

Re:Finally

[bitingduck]

I've always wanted to start a journal called "Journal of Null Results" where people can publish research that's well done, but came up with a result that doesn't qualify as earthshaking or "sexy".

Publishing null results would help people avoid repeating things that have been done already, as well as help refine research to see if there is a positive result hidden in the null.

In the case of big medical studies it could provide a great source for data mining, where metastudies of a bunch of null results might suggest something that would be hard to see without a lot of overlapping data.

Partial solution

[crow]

So what stops someone who registers a study from still deciding not to publish the results if they aren't favorable to the funding drug company? It does mean that there is public information that a study was started, which can mean some pressure to publish.

And nothing stops a drug company from funding a bunch of studies that aren't registered, and then registering duplicate studies that they then expect to be most favorable. Of course, the registration process would then add expense and delay in getting out this sort of slanted results.

At least even with the current system, I expect the peer reviewed journals are much better than the sort of "studies" that get published regarding the computer industry (e.g., TCO of Windows vs. Linux).

Re:Only 20 studies needed to prove placebo works

[jannesha]

The overwhelming majority of drug trials are corporate-funded. A company that's desperate enough to get its drug to market could easily fund 20 studies, and even if the drug were just placebo, chances are good that 1 in 20 of those studies would turn out positive. ...

Would companies be desperate enough to do this? You bet.

I bet not. These studies can cost millions. Sure, BigPharma has BigMoney, but you're suggesting that they can easily soak a 2000% increase in their phase 4 clinical trial budget!? No way.

Wouldn't u spend billions to prove placebo works?

[KWTm]

Would companies be desperate enough to do this? You bet.

I bet not. These studies can cost millions. Sure, BigPharma has BigMoney, but you're suggesting that they can easily soak a 2000% increase in their phase 4 clinical trial budget!? No way.

If you could take a placebo and fund 20 studies to show that it worked, wouldn't you? That's like saying, I can sink millions and billions into this research to prove that potato chips cure cancer. And I hold the patent on the potato chip. I gain the ability to market this directly to consumers ("scientific studies have shown that it works...") You know how big a market we're talking about? Anything under a billion in expenses is peanuts in comparison.

Btw, it's Phase 3. Phase 1 = safety (does it HARM people?); phase 2 = effectiveness (any sign that it works at all?); phase 3 = efficacy (does it work better than placebo / standard of care?); phase 4 = aftermarket (now that we're selling the drug, have we run into any problems?)