Slashdot Mirror
Medical Journals Fight Burying of Inconvenient Research
A dozen leading medical journals have announced that they plan to refuse publication of clinical studies unless those studies were publicly registered ahead of time. Part of the intention is to prevent researchers from privately doing multiple studies and then selectively releasing for publication only those which yield favorable results. There are many other journals which have not signed on to this plan, however, and it remains to be seen what will happen. Personally, I'm surprised it's taken this long; as Karl Popper wrote, "what distinguishes the scientific approach and method from the prescientific approach is the method of attempted falsification."
The problem with statistics is that they can often be made to say what you want. There was a recent advertisement around here that said 80% of accidents happen within a five mile radius of your home. The ad was pushing to use seatbelts even for a short trips. However, if 90% of driving is done within that same five mile radius, then your actually less likely to get in an accident then when your driving close to home. The message is still good, but the statistics may not actually support what the message is saying.
as wrong as the idea that NIH-funded research does all (or even much) of the work of drug development
No they just do the basic research that results in the drug leads. The companies then do the expensive but scientifically easy trials and rake in all the money (and now it seems, the credit as well).
And since when is an industry spokesman considered a reliable source of information..?
Exactly, all research should be published whether it be good or bad and let the reviewers decide whether the results are valid and the methdology is good and is worth publishing. In that way the researchers will probably gain from the positive critism allowing them to do even better research the next time around.
If they don't know why things are failing, how can they improve?
When the article was talking about companies doing multiple studies and only publishing the best one, though, they were talking about this sort of thing. The companies weren't changing the experiment to make it work. Doing that would raise red flags - maybe not so obvious as creative methods for patient recruiting, but something that other doctors would likely catch. They're just doing the studies several times and publishing the best one (or just doing one big study and only using part of the data set for publishing). Doing a pre-run study as you suggest would help their odds, but would nowhere near assure them good resuls in the second, registered study.
This nicely illustrates Lowe's point: what you're saying is widely believed, but is absolutely, utterly, entirely, absurdly false. (Except for the non-sequitur at the end about "credit" which I don't understand at all.)
And since when is an industry spokesman considered a reliable source of information..?
First, he's a chemist, not a "spokesman". Second, he (and I) do precisely the work you claim doesn't exist and might be thought to have something to say on the matter. But, if you want to limit your "reliable sources of information" to people who don't know what they're talking about, that's certainly your right.
What I'm listening to now on Pandora...
First, I think you missed my main point.
I agree that selective publication has the potential to influence use of an approved drug. My point was that there's no way to get a new drug approved based on selective data submission, unless the company's officers are willing to take a substantial risk of going to jail.
Second, Searle did give the FDA all that data on Celebrex, but the whole mess took place after the drug was already approved.
Celebrex was originally approved in Dec 1998. At that time, FDA told Searle they could not make any claims that Celebrex was safer than other NSAIDs. Their clinical data at the time didn't justify such claims. Note that this was based on data submitted to FDA between June and Dec 1998.
In the studies you're referring to, altbough the original protocols date back the Jan 98, it wasn't until Sep 99 that all patients had been treated for at least 6 mos. The studies ended in Jan, 2000. The 6 month data was published in the 9/13/2000 issue of JAMA. All of the data was submitted to FDA and discussed by the Arthritis Advisory Committee on 2/7/01, because Searle wanted FDA approval to amend their labeling to say that Celebrex had lower risk of GI problems than other NSAIDs.
In fact, that is how the discrepancy between the full data and the JAMA article became public. One of the participants in the advisory committee meeting was a GI named Wolfe, who coauthored the favorable editorial accompanying the 2000 JAMA paper! Of course, his editorial was based on just the 6 mos data. When he saw the full data set at the AC mtg in '01, he was quite unhappy.
To boil it all down:
1) Drug companies cannot get unapproved drugs approved by doing lots of clinical studies, and only submitting the "good" ones.
2) Assuming they can get the drug approved, drug companies can potentially use selective publication to enhance perception & use of their drugs.
P.S. If you really want, I can provide links to support the above Celebrex info. It's almost all from FDA's website.