Slashdot Mirror
Dozens of Recent Clinical Trials May Contain Wrong or Falsified Data, Claims Study (theguardian.com)
John Carlisle, a consultant anesthetist at Torbay Hospital, used statistical tools to conduct a review of thousands of papers published in leading medical journals. While a vast majority of the clinical trials he reviewed were accurate, 90 of the 5,067 published trials had underlying patterns that were unlikely to appear by chance in a credible dataset. The Guardian reports: The tool works by comparing the baseline data, such as the height, sex, weight and blood pressure of trial participants, to known distributions of these variables in a random sample of the populations. If the baseline data differs significantly from expectation, this could be a sign of errors or data tampering on the part of the researcher, since if datasets have been fabricated they are unlikely to have the right pattern of random variation. In the case of Japanese scientist, Yoshitaka Fuji, the detection of such anomalies triggered an investigation that concluded more than 100 of his papers had been entirely fabricated. The latest study identified 90 trials that had skewed baseline statistics, 43 of which with measurements that had about a one in a quadrillion probability of occurring by chance. The review includes a full list of the trials in question, allowing Carlisle's methods to be checked but also potentially exposing the authors to criticism. Previous large scale studies of erroneous results have avoided singling out authors. Relevant journal editors were informed last month, and the editors of the six anesthesiology journals named in the study said they plan to approach the authors of the trials in question, and raised the prospect of triggering in-depth investigations in cases that could not be explained.
claims another study
Authors of the first study were promptly sacked
“He’s not deformed, he’s just drunk!”
There are lots of TV commercials from law firms advertising class action lawsuit settlements from the side effects of drugs. Is it surprising to see so many dangerous drugs and treatments if there's so much dishonesty in the testing process?
Thanks for that! Now I can use that tool to generate data for my upcoming fabricated studies.
Everything I write is lies, read between the lines.
That's... less than 2%. Naturally, we want it to be 0%, but 1.8% is nothing to generate scare headlines over.
"I don't know, therefore Aliens" Wafflebox1
controls is. They refuse to allow new drugs and procedures to be used.
hates this people, this is no surprise. They always stand against medical advances since it hurts their profits.
So 90 of the 5,067 were outlier-like data and this is concluding results on these outliers.
I knew that publish or perish was ruining science, but this is actually the most heartening news I've heard of its credibility.
I've learned that less than 1.8% of these studies used non-extreme crazy data. My faith in science is restored!
>>"ad space available -- low rates!!!"
It is fortunate that fraud (or incompetence) like this never occurs in other areas. For example think of the implications of this happening in Climate Science papers and studies. Luckily we can trust those implicitly, especially the model based ones.
A study should be repeated by an org that has no skin in the game per results. They should be paid to test and get the same amount of compensation regardless of outcome. A random lottery should decide the head managers/researchers for any given repeat.
Table-ized A.I.
That word strikes again. "May". There is never a time when recent studies definitely had no falsified data. The opposite of this would be far more newsworthy, which is usually a good sign that someone has been wasting their time. There always has to be skepticism of studies, that's why replication is required.
I do not want your cheap brainburning drugs. They are useless for work. And I am a working man today.
Most scientists 'can't replicate studies by their peers'
http://www.bbc.co.uk/news/science-environment-39054778
Majority of landmark cancer studies cannot be replicated:
https://science.slashdot.org/story/12/04/06/139231/majority-of-landmark-cancer-studies-cannot-be-replicated
http://www.nature.com/nature/journal/v483/n7391/full/483531a.html
etc.etc.
...But, did he use a bonferroni correction to compute his p-values, since this is a classic data dredge? Sure, his method will turn up true positives (and did, for at least one known offender) but what remains to be seen is the false positive rate and the lawsuit rate, since skewed distributions could have many causes some of which are benign and this is pretty serious defamation of character if one casts aspersions without secondary supporting evidence of malpractice.
In other words, are his "positives" really malefactors or is he picking out acne-causing green jellybeans: https://xkcd.com/882/
Worse, the study appeals to my own confirmation bias on the matter, as I'm sure that the rate of wrongdoing in the research is if anything higher than he finds it (he "detects" just under 2% possible/probable bad articles -- I would have guessed more like 5% to 10% just from sheer incompetence and inadequate power, but perhaps he corrected somehow for inadequate power although TFA doesn't really say). So I WANT to believe him, but sans bonferroni, I don't know what to to make of his p-threshold of 1/10000 applied to 5000 samples and testing multiple statistics per sample. He really needs bonferroni twice, as he dredges for out-of-bounds statistics PER article as well, for thousands of articles.
Even when the experts all agree, they may well be mistaken. --- Bertrand Russell.
more news at eleven
Scott Gottlieb, the current head of the FDA, wants to end drug trials. "The free market will put the bad actors out of business."
Most of those studies are probably not blinded. I wouldn't be surprised if all of the flagged ones are not. There doesn't have to be any fraud at all. If you know what the groups are, your brain will introduce its own bias, without you even knowing about it.
Claim their study is related to Climate Change.
Then they can jank the data all they want and the scientific community won't care.
http://pdfernhout.net/to-james... "The problems I've discussed are not limited to psychiatry, although they reach their most florid form there. Similar conflicts of interest and biases exist in virtually every field of medicine, particularly those that rely heavily on drugs or devices. It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine. (Marcia Angell)"
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
I aint no fancy science nerd, but if there isnt one, there ought to be an international body of scientists who's sole purpose is to analyze and replicate past studies.
The usual threshold of statistical certainty used for publishing scientific results is 95% (sometimes 98%). That is, a result becomes noteworthy enough to publish if there's a 5% or lower chance of it happening simply due to random chance.
90 studies out of 5,067 is 1.8%. Which is below the 5% you'd expect from a 95% threshold, and even the 2% you'd expect with a 98% threshold. When you're looking at five thousand studies, about 100-250 of them will report results which aren't real, but simply happened due to chance. That only 90 such studies were found seems to indicate scientists are using an even stricter standard than 98% certainty before publishing. And those 90 aren't necessarily due to fraud, but are within the number you'd expect to find purely due to chance.