Meta-Research Debunks Medical Study Findings

jenningsthecat writes "From The Atlantic comes the story of John Ioannidis and his team of meta-researchers, who have studied the overall state of medical research and found it dangerously and widely lacking in trustworthiness. Even after filtering out the journalistic frippery and hyperbole, the story is pretty disturbing. Some points made in the article: even the most respected, widely accepted, peer-reviewed medical studies are all-too-often deeply flawed or outright wrong; when an error is brought to light and the conclusions publicly refuted, the erroneous conclusions often persist and are cited as valid for years, or even decades; scientists and researchers themselves regard peer review as providing 'only a minimal assurance of quality'; and these shortcomings apply to medical research across the board, not just to blatantly self-serving pharmaceutical industry studies. The article concludes by saying, 'Science is a noble endeavor, but it's also a low-yield endeavor ... I'm not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life.' I've always been somewhat suspicious of research findings, but before this article I had no idea just how prevalent untrustworthy results were."

Comments

Reality check

[koreaman]

If medical research were really as close-to-useless as The Fine Summary claims, we'd be hardly better off with modern Western medicine than with homeopathy and prayer. Clearly, we are, refuting the idea that medical research doesn't do a huge amount of good. I'm not saying it isn't flawed, but give it some credit.

Re:Reality check

[zrbyte]

If medical research were really as close-to-useless as The Fine Summary claims, we'd be hardly better off with modern Western medicine than with homeopathy and prayer.

True.

Top notch research is what makes all the medical breakthroughs, but this is only the top few percent of ALL medical research. IMHO one of the main reasons there are so much bogus papers out there is because of the publish or perish attitude in academia, which requires researchers to have a set number of papers published to be eligible for research funding, tenure, other career advancements. I know from experience (although not in medical research, but natural sciences) that sometimes you have to publish a paper even if you know that the results aren't meaningful, or of value to anyone. Then there are people who publish things that were not subjected to rigorous testing, double checking of data, etc. which can easily turn out to be wrong. Lastly there are the cheats. All I'm trying to say is that it's more of a science policy problem than a problem with the integrity of researchers. If the number of publications has to go up, then their quality will surely decrease. Very few research groups (the ones which have good funding) have the luxury of publishing only every now and then. But when they do it's usually a Science or Nature paper. This problem os quality VS quantity is most serious in China. However, not even journals such as Nature are immune to this.