Slashdot Mirror
Meta-Research Debunks Medical Study Findings
jenningsthecat writes "From The Atlantic comes the story of John Ioannidis and his team of meta-researchers, who have studied the overall state of medical research and found it dangerously and widely lacking in trustworthiness. Even after filtering out the journalistic frippery and hyperbole, the story is pretty disturbing. Some points made in the article: even the most respected, widely accepted, peer-reviewed medical studies are all-too-often deeply flawed or outright wrong; when an error is brought to light and the conclusions publicly refuted, the erroneous conclusions often persist and are cited as valid for years, or even decades; scientists and researchers themselves regard peer review as providing 'only a minimal assurance of quality'; and these shortcomings apply to medical research across the board, not just to blatantly self-serving pharmaceutical industry studies. The article concludes by saying, 'Science is a noble endeavor, but it's also a low-yield endeavor ... I'm not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life.' I've always been somewhat suspicious of research findings, but before this article I had no idea just how prevalent untrustworthy results were."
That fat in your diet is bad for you.
If medical research were really as close-to-useless as The Fine Summary claims, we'd be hardly better off with modern Western medicine than with homeopathy and prayer. Clearly, we are, refuting the idea that medical research doesn't do a huge amount of good. I'm not saying it isn't flawed, but give it some credit.
Le français vous intéresse?
... it's the most useless way to progress, except for all others.
Be a skeptic, but don't confuse skepticism with truthiness. Unfortunately, I expect a rise in the use of truthiness over science when people will investigate reality.
Those who can, do. Those who can't, sue.
In 2001, rumors were circulating in Greek hospitals that surgery residents, eager to rack up scalpel time, were falsely diagnosing hapless Albanian immigrants with appendicitis. At the University of Ioannina medical school’s teaching hospital, a newly minted doctor named Athina Tatsioni was discussing the rumors with colleagues when a professor who had overheard asked her if she’d like to try to prove whether they were true—he seemed to be almost daring her. She accepted the challenge and, with the professor’s and other colleagues’ help, eventually produced a formal study showing that, for whatever reason, the appendices removed from patients with Albanian names in six Greek hospitals were more than three times as likely to be perfectly healthy as those removed from patients with Greek names.
Okay - so I only bothered getting this far into TFA.
Now - I'm no medical junkie, I didnt' even take Bio in high school, but I have occaisonally watched Scrubs and House and ER and a bunch of other medical dramas from time to time.
One thing that always seems to surface in these TV shows is the patients history, like their religion, nationality, where they work, etc. This leads me to believe that maybe - JUST MAYBE - there is actually some correlation between something in the Albanian culture and society that has an increased chance of appendicitis, and that its entirely possible that this pushes doctors towards diagnosing that when some of the symptoms appear. (Not that this is particularily the best course of action, but what else would you do? Run every test?)
But at the same time I know how incredibly innaccurate a lot of television can be about portraying a subject. However, the IT Crowd has basically mimicked my life, but thats another story for another time. Anyways, so if I'm absolutely and completely wrong, feel free to mod me down - but I just thought I'd interject.
I was going to suggest just the opposite. Medical studies aren't like doing physics experiments in the lab: you can't control the minutae of the experiment to anywhere near an ideal degree. You need to have control groups, you need to factor out all possible other causes (and even then, you can be sure you won't catch them all), you need to have long-term observations and follow-up studies. Sometimes you'll see a trend and it turns out it was pure chance. Everyone is different, both in terms of genetic makeup and environment, and that's going to mean that everyone has different reactions to just about anything.
There's clearly room for a great deal of improvement here, but people might need to accept that results of medical studies are never going to be as clear-cut as math papers. I think, and now I'm moving into personal opinion, the most important step-- and this applies to both the media and to scientists who might want to cite or cross-reference something-- is that nothing is ever, ever proved after the first paper. Ever. The first paper on some hypothetical connection really should just be ignored, except for prompting further investigation Like I said above, it needs follow-ups, and experiments with different variables controlled for.
Dislike the Electoral College? Lobby your state to join the National Popular Vote Interstate Compact.
Because he has documented statistical analysis backing his points up and if you want to verify them you just have to read his studies, look up all the studies he cites and then run statistical analysis on them yourself. That's the beauty of the scientific method (when it's implemented correctly at-least...).
I've heard a lot of disgusted (perhaps accurate, perhaps not) complaints about how politicians (usually this complaint is aimed at Republicans, as in the book title "The Republican War on Science") have "cut money for basic, fundamental research -- the kind that benefits all of us." It's a slight paraphrase, and you might consider it a straw man, but I don't think these gripes are hard to find.
Meta studies like this are worth thinking about when someone says we need more "basic research" in any given area: it might be sincere, well-intentioned (or who knows, even perfectly accurate, in some platonic universe where "needs" could be accurately, objectively assessed and compared), but it might also mean that well-intentioned money would be spent on stuff that is very sciency, but not very helpful.
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
What? Doctors have to be convinced to follow evidence-based medicine? What were they practicing before? And why are they against it?
You know, there's never been a randomized control trial of the effectiveness of parachutes versus placebo when jumping out of planes at high altitude. Would you care to volunteer?
Some things are just obvious or can be deduced from retrospective analysis or theoretical modeling. And some times it would be unethical or impractical to conduct a randomized control trial of a treatment (e.g., testing an AIDS drug versus placebo).
The idea that doctors are ALLOWED to prescribe drugs for off-label use horrifies me (it's off-label because there's no proof it works for the off-label condition).
No, there is often tons of proof. Off-label just means that the manufacturer hasn't gone through the (expensive, time-consuming) process of proving its effectiveness to the FDA. If the patent has expired there's often no financial incentive to go through the trouble.