Slashdot Mirror
Plan To Replicate 50 High-Impact Cancer Papers Shrinks To Just 18 (sciencemag.org)
Five years ago, researchers set out to replicate experiments from 50 high-impact cancer biology papers. Now, due to various challenges relating to a lack of funding and expertise, the project only expects to complete just 18 studies. Science Magazine reports: The Reproducibility Project: Cancer Biology (RP:CP) began in October 2013 as an open effort to test replicability after two drug companies reported they had trouble reproducing many cancer studies. The work was a collaboration with Science Exchange, a company based in Palo Alto, California, that found contract labs to reproduce a few key experiments from each paper. Funding included a $1.3 million grant from the Laura and John Arnold Foundation, enough for about $25,000 per study. Experiments were expected to take 1 year.
Costs rose and delays ensued as organizers realized they needed more information and materials from the original authors; a decision to have the proposed replications peer reviewed also added time. Organizers whittled the list of papers to 37 in late 2015, then to 29 by January 2017. In the past few months, they decided to discontinue 38% or 11 of the ongoing replications, Errington says. (Elizabeth Iorns, president of Science Exchange, says total costs for the 18 completed studies averaged about $60,000, including two high-priced "outliers.") One reason for cutting off some replications was that it was taking too long to troubleshoot or optimize experiments to get meaningful results... So far, the project has published replication results for 10 of the 18 studies. "Five were mostly repeatable, three were inconclusive, and two studies were negative, but the original findings have been confirmed by other labs," reports Science Magazine. "In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:CB's critics have pointed out."
Costs rose and delays ensued as organizers realized they needed more information and materials from the original authors; a decision to have the proposed replications peer reviewed also added time. Organizers whittled the list of papers to 37 in late 2015, then to 29 by January 2017. In the past few months, they decided to discontinue 38% or 11 of the ongoing replications, Errington says. (Elizabeth Iorns, president of Science Exchange, says total costs for the 18 completed studies averaged about $60,000, including two high-priced "outliers.") One reason for cutting off some replications was that it was taking too long to troubleshoot or optimize experiments to get meaningful results... So far, the project has published replication results for 10 of the 18 studies. "Five were mostly repeatable, three were inconclusive, and two studies were negative, but the original findings have been confirmed by other labs," reports Science Magazine. "In fact, many of the initial 50 papers have been confirmed by other groups, as some of the RP:CB's critics have pointed out."
Replicating the results of experiments, is of course, the scientific method.
Doing so for about $25K each is not garnering much interest in the scientific community, since so many research and development grants far exceed that budgetary bottleneck, particularly when a directed outcome is encouraged.
Happiness in intelligent people is the rarest thing I know.
Ernest Hemingway
High-impact cancer papers are distributed on Snapchat. That’s why every interesting new approach that we hear about in this forum disappears and is never seen again.
Come the fuck on Ivan, you're better than this Trump-level English fail.
I have a cancer paper for you: stop eating sacks of sugar. Cured.
I object to power without constructive purpose. --Spock
are working on how to get google ads past your blocker, improving high frequency trading and better missile guidance systems to defend us from enemies we wouldn't have if we'd stop meddling in other countries to their obvious detriment.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
that I suppose it could be worse. Until the ruling class realized they could make better bombs we used to send physicists off to die in the trenches.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
I remember reading somewhere that the number one reason for the stagnation of quality replication/validation work being done is that the really good researchers don't want to validate somebody else's work, as they won't get any credit for those discoveries... Theyd much rather be performing their own research. That in itself is quite serious. So unless there are fundamental changes in the rewards system these researchers have then the pace of progress in these fields will hardly accelerate.
As a biomedical research, 25k per experiment to replicate key experiments seems like a massive underestimation of cost. 60k is a bargain. 100k budget for each would have been more appropriate. I think they screwed themselves over with that price estimate and setting expectations. Even contracting the work out, I bet every project ran into unforeseen issues that should have been budgeted for. That is just my perspective, biomedical research is an extremely difficult and challenging field, even repeating those classic papers of the past- without having years of specialized expertise in the relevant sub-fields puts the project at a disadvantage.
Yes, the study found a correlation, but how to interpret that ?
The BMI *provokes* cancer ? (more correctly, that high BMI is a cause that increases cancer risks ?)
For some cancers (like gastro-oesophagal tract) that might be the case (e.g. due to chronic damage by an overflowing stomach, chronic damage to the liver by fat, chronic damage by crystal formation, hormonal disbalance due to the endocrine system getting fucked up, etc.).
But it might be that BMI and cancer happen to appear in the same people due to some cofounding factor ?
People eat shit. Unhealthy diet habbit increases obesity, unhealthy diet increase cancer risks due to high concentration of harmful substances (hormonal disbalance but this time due to them being in abnormal quantities in the food, well documented carcinogens present in the food, etc.).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
And I am still convinced that it doesn't exist. Simulations show that the effect is actually very small. And looking into the details of the experiments shows that it can only be shown by making samples of materials with a different composition. These different compositions can always cause other effects though, which change what is measured. For some papers I can even find other sources that show that the other differences in the materials cause the measured effect. So people are always comparing apples and oranges.
Yet it is 10 peer reviewed papers that show the same effect, which is not even measurable in theory. It was also an area with lots of funding and possibly high effect. This makes me really worry about these cancer studies.
about 20 years ago I picked some medical studies more or less at random to see how the medical folk handled their statistics. They were all peer reviewed published papers. In almost all cases, the sample sizes were shockingly small and there were huge risks of Type I and Type II errors (basically false positives and false negatives). I found papers that were using the wrong statistical techniques, such as when calculating the probability of the means of two sample populations being different. In almost all cases, my feeling was that the authors (and apparently the editors) were not talented enough in statistics to recognize the extra lack of information you'll get from a small sample sizes and that you have to be extra careful to make sure you are doing the statistics correct. This was done a long time ago and I wish I remembered more specifically or had the report I did. (Actually I'd be fun to see my report because my own standards of quality have risen as I published over the decades.) Anyway, long story short. Be extra skeptical when you read the media reporting on the latest study. The studies themselves often has large uncertainty to them and the media often runs with them as if they are gospel.
A paper in a peer-reviewed journal is supposed to give all the information required to reproduce the experiment. Obviously they're failing that.
It will be very interesting to see just how many are actually confirmed. The reproducibility crisis ( https://en.wikipedia.org/wiki/Replication_crisis ) is a well-known thing.
Why not wait to see if this recent Australian discovery that puts cancer to sleep works. Or do a study on the long ago discovered cures that are taking place in Mexico.
https://www.youtube.com/watch?...