Slashdot Mirror


Disappointing Cancer Study Results Go Unreported

An anonymous reader writes "Science News reports on a new study showing that most cancer drug trial results are never published, probably leaving patients vulnerable to cocktails that have already been shown to be dangerous or useless."

24 of 77 comments (clear)

  1. Negative results by Anne_Nonymous · · Score: 3, Funny

    This sounds reasonable. From now on I'll be emailing my pals every time I don't get laid.

    1. Re:Negative results by eln · · Score: 4, Funny

      It would be a lot more efficient if you just had a cron job that sent out an email every night.

    2. Re:Negative results by philspear · · Score: 5, Informative

      Whoever modded that down as overrated clearly didn't know enough to judge. You don't publish negative results because of the time and effort it takes to write it up.

      The reasons vary for different fields. A lot of times, researchers don't bother doing proper controls until after the experiment works or doesn't. If the trial run works, you do the controls afterward to verify your result was real, if you don't get the result you expect you might try it again, doing some troubleshooting, but at some point you have to make a choice between a control that would be particularly onerous or expensive, or giving up on the experiment entirely. If you get a negative result that you're not interested in, you generally don't do the controls to prove to others it was a valid result because you don't care and have better things to do. But that's what you would have to do to publish it.

      A negative but true result can also be even more difficult to prove than a positive result would be. If you are expecting one protein to interact with another one, and you get no result, it could be that they are and your test just isn't working. If you do the experiment a different way and still show no interaction that makes it a little more convincing, but doesn't prove that both systems are working. You can't say for sure they don't interact in real cells.

      In clinical trials you could think of additional reasons why someone would not care to publish the negative results. The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working. They could spend 5 years refining it before it actually works, then more years before it gets to market, and they only have a few years before the patent runs out. If they don't patent it and aren't sure it's a complete dead-end, another company might take the results and make a working drug from it, effectively stealing the expensive work to get up to that point.

      Not a lawyer or an expert on the pharmecutical industry obviously, but publishing means making it known, and they're only going to do that if they're sure they're done with it, if then.

      It makes sense that they're not going to be published, and while it's less than ideal, I think it would be worse to force the pharmecuticals to publish negative results of trials. If you make the clinical trial phase that risky, companies would be more reluctant to develop new drugs that haven't already proven effective, and advances in cancer treatment would slow.

    3. Re:Negative results by j0nb0y · · Score: 4, Informative

      The most obvious is that the drug company doesn't want to make it known that they're working on drug X. Not sure how that works, but you could imagine that they might have to patent it to keep others from using it, and then the clock on the patent starts before they actually get it working.

      Except clinical trials are not required to attain patent protection. Any drug company with good patent practitioners is going to have a patent application filed *before* clinical trials. You can always abandon it later if the clinical trials don't work out. It's much cheaper to file and then abandon an application on a drug than to miss the opportunity to get a patent on a successful drug.

      --
      If you had super powers, would you use them for good, or for awesome?
    4. Re:Negative results by jamesh · · Score: 4, Interesting

      I have been told by someone who would know (psychiatrist) that drug trials (maybe just here in Australia) have to be announced before the trial begins otherwise the outcomes can't be published. The idea is that you have to announce a trial before you begin, and so if the outcomes are bad and you don't publish, the bad outcome can be inferred from the lack of publication, even if the specifics remain unknown.

  2. A one sentence by DanZ23 · · Score: 4, Funny

    Summary? You're practically forcing me to read TFA

  3. More complaining and second-guessing by Kohath · · Score: 2, Insightful

    Cancer patients are already vulnerable to cancer.

    Not sure what the point of this story is. Sometimes things don't work out the way everyone wishes they would. Apparently every decision to say something or not say something always has to be second-guessed by third parties who have no responsibility or accountability -- but they get to demand things anyway.

    I'm sure a lot more of these failed trials would be published if there was a financial incentive. The complainers should start a foundation and start paying the people who have better things to do than to write papers and publish info that's of no use to them. They should do that instead of complaining.

    1. Re:More complaining and second-guessing by BabyDave · · Score: 5, Insightful

      The point of the story is that companies publish the successful trials on a drug, but don't publish failed trials on that same drug - i.e. they cherry-pick the results.

    2. Re:More complaining and second-guessing by Anonymous Coward · · Score: 2, Informative

      Maybe try reading the article?

      FTFA:

      "Of particular concern," they argue, is the especially poor showing by industry-sponsored trials, since they tended to probe the value of patented drugs -- "many of which are in clinical use."

      I would say that's newsworthy if not all that surprising. Just another case of big drug companies screwing over the general population to make a buck. They run studies to test their drugs and if the results are unfavorable they just bury the study while keeping the drug in circulation.

    3. Re:More complaining and second-guessing by nedlohs · · Score: 5, Insightful

      It's not just cherry picking - which wouldn't matter so much in itself. Drugs are very hit and miss you expect lots of things to just not work.

      The problem is they study drugs X, Y, and Z in combination and find that not only does it not help it makes the patients worse. They don't bother publishing since they didn't get anything useful out of it and no one is going to cite them...

      A year later another group decides to study X, Y, and Z in combination. So a pointless study that harms patients is done because this second group never found out about the previous study in their literature search.

    4. Re:More complaining and second-guessing by Davak · · Score: 3, Interesting

      As somebody who was previously in the academic medical field, this is not just a problem with cancer drugs. "Positive Outcome Bias" or "publish bias" is a huge problem.

      http://www.ama-assn.org/public/peer/7_15_98/jpv71042.htm

      A negative study should be just as important as a positive study. If done well... obviously.

      Published negative studies dissuade doctors from using certain offlabel treatments. Published negative studies prevent other docs from wasting time and money to discover the same results.

      More importantly, many clinical changes are based on meta-studies... which as basically studies which combine all the available data. If negative studies are not published, it throws off these metastudies... and thus bad care occurs.

  4. Financial incentive by Stickerboy · · Score: 3, Interesting

    The obvious reason is that it takes time and money to publish study results, neither of which are recouped currently if the study showed negative results.

    The obvious fix is to reward pharmaceutical companies financially for publishing all results. Form a subentity within the NIH with the power to purchase study data and results that can be published by the government or a peer-reviewed journal.

    --
    Light a fire for a man and he'll be warm for a day. Light a man on fire and he'll be warm for the rest of his life.
  5. Cargo Cult Science by McGregorMortis · · Score: 4, Insightful

    This very subject was addressed, very eloquently as usual, by Richard Feynman in his famous Cargo Cult Science lecture.

  6. Nothing new by edcheevy · · Score: 4, Insightful

    It's called the "file drawer problem" and impacts every field of science. If you don't find significant results, you don't get published, and you stick your "failed" study in the file drawer. As a result, "failed" studies on ANY topic usually get swept away. It's unfortunate, but there's nothing particular sinister about it (as the article seems to imply). There's just no incentive to publish the trials and studies that didn't work.

    1. Re:Nothing new by ceoyoyo · · Score: 4, Informative

      The pharmaceutical industry HAS been caught doing some sinister things with their studies. They do ten studies and publish the three that can be interpreted positively. The seven that can't never see the light of day. Unless of course they accidentally get published on the company web page. Then the sinister part: the drug is approved and sold to treat the condition. The clinical trials registry was formed to try to clean up that particular mess.

      This particular story doesn't seem to describe anything sinister. The trials were all in the registry (that's where the authors got the data for the study). Sure, the journals tended not to publish the negative results, but they were all available in the registry.

      As far as I can tell from the article they didn't even look at the actual worrisome situation where there are a few published positive studies and many more unpublished negative studies, for the same treatment.

    2. Re:Nothing new by ceoyoyo · · Score: 3, Informative

      "Published" in a major journal. ALL of the studies were published in the registry. As I said, you can't ask the major journals to publish every failed study.

      If you want to try and standardize trials, go for it. I wish you luck. Speaking as someone who actually does them, it won't happen. There are all sorts of reasons why trials are difficult to compare. Different centres, different technology, different standards of care. Ethics committees in different places demand different things. Not to mention you rarely have a situation where many trials are testing exactly the same thing. Most meta-analyses look at trials that are designed to look at slightly different things and then try to compensate for the differences in study design.

  7. Re:Cancer treatment is a product by Anonymous Coward · · Score: 5, Funny

    I have fresh horse urine for only $9 per ounce. Do you want to be cured, or not?

    Nay.

  8. Re:Cancer treatment is a product by zippthorne · · Score: 2, Insightful

    No, they'd much rather have those 10 years worth of payments up front, so they can invest or re-invest it.

    You seriously don't think people would pay more for a week-long cure than for a week's worth of "treatment-in-perpetuity?"

    --
    Can you be Even More Awesome?!
  9. Re:Cancer treatment is a product by wiggles · · Score: 2, Interesting

    Horse urine can't cure cancer, although it is used in hormone replacement therapy. I suppose some cancer patients driven to early menopause due to radiation and chemotherapy do use it, though...

    http://en.wikipedia.org/wiki/Premarin

  10. Re:First Post ... sadly by moderatorrater · · Score: 4, Insightful

    First of all, the mod who gave you "redundant" apparently doesn't realize that this has been widely known for a long time.

    Second, this is new because someone's done the actual study and shown the degree to which studies don't go reported. Even if only half of the unreported studies were because of poor results, that's enough to skew things very, very badly.

    Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

  11. Re:I'm publishing the results of a failed cancer t by adamofgreyskull · · Score: 3, Insightful

    My mother went through a 6 week series of trials of epsom salts against colo-rectal cancer.

    The mixture was ineffective.

    My, how very useful this information is for cancer patients!

    Yes, because that is exactly the same thing. *sigh*

    Would my mother have received the quality of the care she eventually did get if her doctors had missed relevant articles in medical journals thanks to the massive signal to noise ratio?

    If every failure were published, the cancer research community would suffer the same "eternal september" the usenet community did.

    What if the researchers developing new drugs and treatments had access to the failures of others so that they knew what *not* to try. Outside of your pathetically childish and facetious example about Epsom salts, this information could be invaluable. Would you have wanted your mother to die because scientists working for Pfizer didn't tell the community about a failed treatment that they had already tried which GlaxoSmithkline then spent 2 years replicating, at the expense of another possibly more fruitful avenue of research?

  12. Re:First Post ... sadly by stranger_to_himself · · Score: 3, Informative

    Anyone doing this should get put in jail for a long, long time. It may not be fraud in the sense that they're publishing fraudulent results, but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

    That's a bit simplistic - often nobody is directly responsible for not publishing results, it comes about because major medical journals are not interested in pubishing negative findings. There are of course exceptions, when the negative findings come as a big surprise or there is a lot of anticipation about them, like the reports of the Alzheimer's disease vaccination not working earlier this year.

    The medical journals themselves know this is a problem, but they aren't sure what to do about it. One suggestion has been the journals should make a 'commitment to publish' for a trial based on its design and importance, and then publish the findings whatever they are, but that would of course mean they'd be publishing a lot less interesting stuff.

    Also scientists can't be bothered to write up negative findings, because there are proportionally so many more of them than positive ones.

    There is a problem of pharma companies suppressing bad findings or writing misleading journal articles, but that's a wholly independant issue and should be dealt with by the schemes in develpment for enforcing results to be left in public databases after the trials are finishsed. The problem with that is that not many GPs will be searching through these databases

    The real solution is probably for doctors not to rely on scientific journals for information (which they often don't really understand), but to wait for advice from bodies like the National Institute for Clinical Excellence or the Cochrane Collaboration who do very thorough searches and synthesis.

  13. Plus. by Wolfier · · Score: 3, Interesting

    There's a wide assumption that the researchers themselves really want to publish all results.

    Unfortunately, as in almost the entire field of "science" nowadays, it's not the case.

    Researchers themselves have a tendency to hide failures - given that most experiments result in failure, they tend to focus on reporting the ones reporting success.

    This use of time simply makes most sense - they don't have the time to report all the failures, and reports of failures not as valuable as reports of success only makes it worse - think about what kind of views your peers will have towards you if most of your publications are negative results.

    Sadly, this thinking is parasitic and is very prevalent across all research fields.

    Journals are very selective given the limited number of pages they have. If I were a journal, I'd pick reports of success first. It's the evil of centrally-controlled publication, and the mindset that, "if a research is of any good value, it must appear on some journal".

    Granted, peer review is a good thing, but there must be a way to give researches credibility without getting published on some journal.

    Compounded with big-pharma-sourced funding with very fine strings attached...we have a really screwed up system.

  14. you're dangerously wrong by jipn4 · · Score: 2, Interesting

    Even if only half of the unreported studies were because of poor results, that's enough to skew things very, very badly.

    The basic idea is that you should only use drugs or drug combinations for which there is evidence that they work and are not harmful. If there's nothing published, don't use it!

    but by not publishing results they're creating fraudulent overall data, with possibly deadly results. This needs to stop.

    There's nothing "fraudulent" about it. Studies often fail for many reasons completely unrelated to the drugs themselves.

    Scientific experiments are usually one-sided: a positive result tells you something, a negative result tells you nothing.