Slashdot Mirror
Majority of Landmark Cancer Studies Cannot Be Replicated
New submitter Beeftopia writes with perhaps distressing news about cancer research. From the article: "During a decade as head of global cancer research at Amgen, C. Glenn Begley identified 53 'landmark' publications — papers in top journals, from reputable labs — for his team to reproduce. Begley sought to double-check the findings before trying to build on them for drug development. Result: 47 of the 53 could not be replicated. He described his findings in a commentary piece published on Wednesday in the journal Nature (paywalled) . ... But they and others fear the phenomenon is the product of a skewed system of incentives that has academics cutting corners to further their careers."
As is the fashion at Nature, you can only read the actual article if you are a subscriber or want to fork over $32. Anyone with access care to provide more insight?
Update: 04/06 14:00 GMT by U L : Naffer pointed us toward informative commentary in Pipeline. Thanks!
But they and others fear the phenomenon is the product of a skewed system of incentives that has academics cutting corners to further their careers.
As I've said before, back when I was in academia, there were always grant-whores and academics more interested in their own interests than science around. Too many people have come to treat science with a reverence more appropriate to a religion than a system of knowledge and discovery, however. And so when I point out that there are scientists out there willing to cook the numbers, exaggerate, play to politics and/or public opinion, etc. I inevitably run into those who say "Science wouldn't allow that" (like my friend who's still in the field). But science is only as good as the people practicing it. And, in any field, there are always those willing to put their own personal interests ahead of the greater good.
I just hope this doesn't cast a shadow over those out there who *are* doing good work and *are* trying to do honest work. Sadly, some of the best researchers out there are the ones who make the least noise, get the least attention, get the least grants, and are least likely to get tenure.
What political party do you join when you don't like Bible-thumpers *or* hippies?
See this discussion of the same paper on In the Pipeline, a blog devoted to organic chemistry and drug discovery. http://pipeline.corante.com/archives/2012/03/29/sloppy_science.php
There is no greater "absolute power" than knowing that if you say or write something that others will like, they will pay you lots of money and make you famous.
It's not that money corrupts. Money is not the root of all evil; the full quote is "the love of money is the root of all evil." When our society decided that money was more important than truth, we surrendered truth to the void.
A research scientist thinks about his day. He can slightly fudge his cancer study, make big headlines, get a ton of grant money and get appointed chair at the university. Or he can go down the long hall to his boss and say, "Nope, this one didn't work either, and while I'd like to start a religion based on false hope, this isn't the false hope you're looking for." If he does that, he can then watch one of his subordinates fudge the cancer study, make big headlines, and be his boss at the same time next year.
Which choice would you make?
Most people pursue careers in atmospheric science for the dollars
My second thought is "Hmmm, academics/scientists skewing results for the sake of their own careers. Global warming, anyone?"
Your second thought is completely off because every single time someone actually tried to replicate global warming research, they DID get the same results. Unlike in the case of those medical papers TFA is about.
Given the expense, I flat don't believe that a private company just decided to replicate 53 studies.
And he claims that authors "made" his team sign confidentiality agreements. How do authors force that?
So, he claims, he can't even tell us which studies failed.
Now he works at a different cancer research company. Conflict of interests?
I don't doubt that we've got problems in the "medical industry", but the linked article reeks of bullshit.
Has anyone looked at Nature?
Sheesh, evil *and* a jerk. -- Jade
While certainly there are those who will publish findings they know to be false, that's not really the big issue I see here. Good science demands that studies be replicated so they can be upheld or refuted. Sure, there's confirmation bias in science all over the place - the bigger problem I see is that there's very little incentive to publish a paper that simply refutes another. Busting existing studies should be a glorious field, but it's not. If big-name scientist A publishes a result in nature, and no-name scientist B publishes a paper in the journal-of-no-one-reads-it, everyone just assumes scientist B is just a bad scientist (assuming he even managed to actually get published at all).
Another major issue is that the null hypothesis is a very un-enticing story. No one wants to publish the paper: "New Drug does nothing to cure cancer". If you spent a year and a ton of money researching New Drug, you're damn sure going to try and make it work. It's unfortunate, because often the null hypothesis is very informative, but it doesn't get you paid or published. Or how about the psychology paper: "Brain does not respond to stimulus A in any meaningful way", don't remember that paper? That's because it never got published.
I think this is less about malicious behavior, and more about a lack of interest (which can somewhat be blamed on the way universities/journals/grants handle funding, notoriety, etc) in replicating and refuting studies.
Do you want to be the guy who cured cancer, or the guy who disproved a bunch of studies?
Ze Atomic Device! It iz Ztolen!
A recent PLOS article (free to view!) analyses modern research, coming to the conclusion that most research findings are false.
TLDR: Because of the nature of the statistics used and the fact that only positive results are reported.
http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124
Physicists get cranky if you show them research with p values higher than one in ten thousand, but medical research routinely produces p values as high as 0.2. On top of that, you have a large number of wealthy corporations who have powerful financial motivations to influence the results of medical research fraudulently, which probably doesn't come up as often in other fields. There has been a lot of news about problems with medical research lately, mostly because the medical research community itself is discussing these issues, which is a promising sign.