Slashdot Mirror
Why Most Published Research Findings Are False
Hugh Pickens writes "Researchers have found that the winner's curse may apply to the publication of scientific papers and that incorrect findings are more likely to end up in print than correct findings. Dr John Ioannidis bases his argument about incorrect research partly on a study of 49 papers on the effectiveness of medical interventions published in leading journals that had been cited by more than 1,000 other scientists, and his finding that, within only a few years, almost a third of the papers had been refuted by other studies. Ioannidis argues that scientific research is so difficult — the sample sizes must be big and the analysis rigorous — that most research may end up being wrong, and the 'hotter' the field, the greater the competition is, and the more likely that published research in top journals could be wrong. Another study earlier this year found that among the studies submitted to the FDA about the effectiveness of antidepressants, almost all of those with positive results were published, whereas very few of those with negative results saw print, although negative results are potentially just as informative as positive (if less exciting)."
At the risk of being modded down to oblivion, I am still curious to how this effects popular theories like global warming. We already has people claiming that the science is wrong and they are generally mocked and ignored because their works are published in major journals. Well, this story seems to indicate that publishing those claims will give them a larger change of it being incorrect.
Anyways, it seems that if you don't tow the line on climate change, there is no room for you anywhere. So where does this leave the accuracy of the claims in light of how common it seems that they can be wrong even when published in a respectable scientific journal. I know the IPCC looked at them, but they didn't validate any of the claims, they only looks at whether or not Humans were the cause (that was their charter and they acknowledged this in their reporting).
I would think that "Publish or Perish" must contribute to a lot of crappy papers getting published. Shovel it out the door, somebody else says it's wrong, write another grant for a study to verify that, shovel that one out the door, rinse, lather, repeat...
For all intensive purposes, "whom" is no longer a word. That begs the question, "who cares"?
No question that the corporatizing of research can lead to conflicts of interest, but what are the alternatives?
You mention drugs that kill people. Well, that would be all of them - including sugar pills. In fact, if you created two groups of 1000 people and gave half of them purple sugar pills, and the other half green sugar pills, you'd find that one group vs the other would have a statistically significant increase in heart attack rates some percentage of the time. If you consider the green pills placebos then you'd erroneously prove that purple sugar pills are dangerous. Statistics with 95% confidence are wrong one time in 20...
The same applies to efficacy - especially in stuff like antidepressants where we have almost no understanding of the physiology of the brain. A psychiatrist I was chatting with speculated that it is like treating "cough" back in the 1400s - even if you had penicillian back then it wouldn't be effective against "cough" since doctors of the time had no way of evaluating what the cause of "cough" was and consequently what the appropriate treatment was. It would be a complete mystery to them why one person might miracuously recover with an antibiotic and another would not benefit at all.
I'm all for having independantly-funded clinical trials to test the safety/efficacy of pharmaceuticals, but that would cost taxpayers a fortune. Also - how do you decide what drugs are to be tested? Will we see lobbyists briging congressmen to make sure their companies products get tested before their competitors (leading to huge profits for them)? The problem with publicly funded R&D is that it politicizes research. The private R&D system combined with patents at least prioritizes research that will lead to the largest number of people using a drug/device/procedure (even if affordability becomes an issue).
I'm not actually convinced that corporate malfeasance is the reason for the recent string of drug safety problems. I think a few issues are more significant:
1. Existing drugs work moderately well - so it raises the bar for new drugs in terms of efficacy.
2. Clincial trials are becoming more and more effective at detecting side effects.
3. Doctors tend to assume that well-established drugs are safe. So, even a tiny increase in risk with a new drug leads doctors to avoid it (even if there isn't any strong evidence that older drugs are any better).
4. The tort system ensures that doctors are better off undertreating a disease than risking a side effect. When a patient dies of cancer due to less aggressive treatment it is the cancer's fault. When a patient dies from a side-effect it is the doctor's fault or the drug companies. This neglects the risk/reward tradeoffs that all treatment decisions involve.
5. All the "easy" drugs have already been discovered. This leads to increasing costs for drug R&D and less selectivity for drugs that enter trials.
6. The large number of drugs in development creates enormous demand for clinical trial subjects. This leads to doctors inappropriately enrolling patients and massive costs. Doctors are basically paid by the subject so that have incentive to commit fraud, and more demand means higher fees which means more expensive drugs.
I'm not sure what the solutions to these problems are. Industry consolidation would probably help - fewer companies competing mean that clinical trial costs would drop, there would be less rush, and drug prices would rise so companies have less need to cut corners. Government funding of drug development might not hurt (from start to finish) - the patent issues go away if government just picks up the tab (with all the issues of politicized medical research).
There really are no easy answers though. Sure, people offer easy answers to the pharmaceutical problem, but they don't seem any better than the easy answers to crime, world peace, and all those other things taht people oversimplify...