Disappointing Cancer Study Results Go Unreported

An anonymous reader writes "Science News reports on a new study showing that most cancer drug trial results are never published, probably leaving patients vulnerable to cocktails that have already been shown to be dangerous or useless."

Comments

Negative results

[Anne_Nonymous]

This sounds reasonable. From now on I'll be emailing my pals every time I don't get laid.

Re:Negative results

[eln]

It would be a lot more efficient if you just had a cron job that sent out an email every night.

Re:Negative results

[philspear]

Whoever modded that down as overrated clearly didn't know enough to judge. You don't publish negative results because of the time and effort it takes to write it up.

The reasons vary for different fields. A lot of times, researchers don't bother doing proper controls until after the experiment works or doesn't. If the trial run works, you do the controls afterward to verify your result was real, if you don't get the result you expect you might try it again, doing some troubleshooting, but at some point you have to make a choice between a control that would be particularly onerous or expensive, or giving up on the experiment entirely. If you get a negative result that you're not interested in, you generally don't do the controls to prove to others it was a valid result because you don't care and have better things to do. But that's what you would have to do to publish it.

A negative but true result can also be even more difficult to prove than a positive result would be. If you are expecting one protein to interact with another one, and you get no result, it could be that they are and your test just isn't working. If you do the experiment a different way and still show no interaction that makes it a little more convincing, but doesn't prove that both systems are working. You can't say for sure they don't interact in real cells.

In clinical trials you could think of additional reasons why someone would not care to publish the negative results. The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working. They could spend 5 years refining it before it actually works, then more years before it gets to market, and they only have a few years before the patent runs out. If they don't patent it and aren't sure it's a complete dead-end, another company might take the results and make a working drug from it, effectively stealing the expensive work to get up to that point.

Not a lawyer or an expert on the pharmecutical industry obviously, but publishing means making it known, and they're only going to do that if they're sure they're done with it, if then.

It makes sense that they're not going to be published, and while it's less than ideal, I think it would be worse to force the pharmecuticals to publish negative results of trials. If you make the clinical trial phase that risky, companies would be more reluctant to develop new drugs that haven't already proven effective, and advances in cancer treatment would slow.

Re:Negative results

[j0nb0y]

The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working.

Except clinical trials are not required to attain patent protection. Any drug company with good patent practitioners is going to have a patent application filed *before* clinical trials. You can always abandon it later if the clinical trials don't work out. It's much cheaper to file and then abandon an application on a drug than to miss the opportunity to get a patent on a successful drug.

Re:Negative results

[jamesh]

I have been told by someone who would know (psychiatrist) that drug trials (maybe just here in Australia) have to be announced before the trial begins otherwise the outcomes can't be published. The idea is that you have to announce a trial before you begin, and so if the outcomes are bad and you don't publish, the bad outcome can be inferred from the lack of publication, even if the specifics remain unknown.

A one sentence

[DanZ23]

Summary? You're practically forcing me to read TFA

Re:More complaining and second-guessing

[BabyDave]

The point of the story is that companies publish the successful trials on a drug, but don't publish failed trials on that same drug - i.e. they cherry-pick the results.

Financial incentive

[Stickerboy]

The obvious reason is that it takes time and money to publish study results, neither of which are recouped currently if the study showed negative results.

The obvious fix is to reward pharmaceutical companies financially for publishing all results. Form a subentity within the NIH with the power to purchase study data and results that can be published by the government or a peer-reviewed journal.

Cargo Cult Science

[McGregorMortis]

This very subject was addressed, very eloquently as usual, by Richard Feynman in his famous Cargo Cult Science lecture.

Nothing new

[edcheevy]

It's called the "file drawer problem" and impacts every field of science. If you don't find significant results, you don't get published, and you stick your "failed" study in the file drawer. As a result, "failed" studies on ANY topic usually get swept away. It's unfortunate, but there's nothing particular sinister about it (as the article seems to imply). There's just no incentive to publish the trials and studies that didn't work.

Re:Nothing new

[ceoyoyo]

The pharmaceutical industry HAS been caught doing some sinister things with their studies. They do ten studies and publish the three that can be interpreted positively. The seven that can't never see the light of day. Unless of course they accidentally get published on the company web page. Then the sinister part: the drug is approved and sold to treat the condition. The clinical trials registry was formed to try to clean up that particular mess.

This particular story doesn't seem to describe anything sinister. The trials were all in the registry (that's where the authors got the data for the study). Sure, the journals tended not to publish the negative results, but they were all available in the registry.

As far as I can tell from the article they didn't even look at the actual worrisome situation where there are a few published positive studies and many more unpublished negative studies, for the same treatment.

Re:Nothing new

[ceoyoyo]

"Published" in a major journal. ALL of the studies were published in the registry. As I said, you can't ask the major journals to publish every failed study.

If you want to try and standardize trials, go for it. I wish you luck. Speaking as someone who actually does them, it won't happen. There are all sorts of reasons why trials are difficult to compare. Different centres, different technology, different standards of care. Ethics committees in different places demand different things. Not to mention you rarely have a situation where many trials are testing exactly the same thing. Most meta-analyses look at trials that are designed to look at slightly different things and then try to compensate for the differences in study design.

Re:Cancer treatment is a product

[Anonymous Coward]

I have fresh horse urine for only $9 per ounce. Do you want to be cured, or not?

Nay.

Re:More complaining and second-guessing

[nedlohs]

It's not just cherry picking - which wouldn't matter so much in itself. Drugs are very hit and miss you expect lots of things to just not work.

The problem is they study drugs X, Y, and Z in combination and find that not only does it not help it makes the patients worse. They don't bother publishing since they didn't get anything useful out of it and no one is going to cite them...

A year later another group decides to study X, Y, and Z in combination. So a pointless study that harms patients is done because this second group never found out about the previous study in their literature search.

Re:First Post ... sadly

[moderatorrater]

First of all, the mod who gave you "redundant" apparently doesn't realize that this has been widely known for a long time.

Second, this is new because someone's done the actual study and shown the degree to which studies don't go reported. Even if only half of the unreported studies were because of poor results, that's enough to skew things very, very badly.

Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

Re:More complaining and second-guessing

[Davak]

As somebody who was previously in the academic medical field, this is not just a problem with cancer drugs. "Positive Outcome Bias" or "publish bias" is a huge problem.

http://www.ama-assn.org/public/peer/7_15_98/jpv71042.htm

A negative study should be just as important as a positive study. If done well... obviously.

Published negative studies dissuade doctors from using certain offlabel treatments. Published negative studies prevent other docs from wasting time and money to discover the same results.

More importantly, many clinical changes are based on meta-studies... which as basically studies which combine all the available data. If negative studies are not published, it throws off these metastudies... and thus bad care occurs.

Re:I'm publishing the results of a failed cancer t

[adamofgreyskull]

My mother went through a 6 week series of trials of epsom salts against colo-rectal cancer.

The mixture was ineffective.

My, how very useful this information is for cancer patients!

Yes, because that is exactly the same thing. *sigh*

Would my mother have received the quality of the care she eventually did get if her doctors had missed relevant articles in medical journals thanks to the massive signal to noise ratio?

If every failure were published, the cancer research community would suffer the same "eternal september" the usenet community did.

What if the researchers developing new drugs and treatments had access to the failures of others so that they knew what *not* to try. Outside of your pathetically childish and facetious example about Epsom salts, this information could be invaluable. Would you have wanted your mother to die because scientists working for Pfizer didn't tell the community about a failed treatment that they had already tried which GlaxoSmithkline then spent 2 years replicating, at the expense of another possibly more fruitful avenue of research?

Re:First Post ... sadly

[stranger_to_himself]

Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

That's a bit simplistic - often nobody is directly responsible for not publishing results, it comes about because major medical journals are not interested in pubishing negative findings. There are of course exceptions, when the negative findings come as a big surprise or there is a lot of anticipation about them, like the reports of the Alzheimer's disease vaccination not working earlier this year.

The medical journals themselves know this is a problem, but they aren't sure what to do about it. One suggestion has been the journals should make a 'commitment to publish' for a trial based on its design and importance, and then publish the findings whatever they are, but that would of course mean they'd be publishing a lot less interesting stuff.

Also scientists can't be bothered to write up negative findings, because there are proportionally so many more of them than positive ones.

There is a problem of pharma companies suppressing bad findings or writing misleading journal articles, but that's a wholly independant issue and should be dealt with by the schemes in develpment for enforcing results to be left in public databases after the trials are finishsed. The problem with that is that not many GPs will be searching through these databases

The real solution is probably for doctors not to rely on scientific journals for information (which they often don't really understand), but to wait for advice from bodies like the National Institute for Clinical Excellence or the Cochrane Collaboration who do very thorough searches and synthesis.

Plus.

[Wolfier]

There's a wide assumption that the researchers themselves really want to publish all results.

Unfortunately, as in almost the entire field of "science" nowadays, it's not the case.

Researchers themselves have a tendency to hide failures - given that most experiments result in failure, they tend to focus on reporting the ones reporting success.

This use of time simply makes most sense - they don't have the time to report all the failures, and reports of failures not as valuable as reports of success only makes it worse - think about what kind of views your peers will have towards you if most of your publications are negative results.

Sadly, this thinking is parasitic and is very prevalent across all research fields.

Journals are very selective given the limited number of pages they have. If I were a journal, I'd pick reports of success first. It's the evil of centrally-controlled publication, and the mindset that, "if a research is of any good value, it must appear on some journal".

Granted, peer review is a good thing, but there must be a way to give researches credibility without getting published on some journal.

Compounded with big-pharma-sourced funding with very fine strings attached...we have a really screwed up system.