Dozens of Recent Clinical Trials May Contain Wrong or Falsified Data, Claims Study (theguardian.com)

John Carlisle, a consultant anesthetist at Torbay Hospital, used statistical tools to conduct a review of thousands of papers published in leading medical journals. While a vast majority of the clinical trials he reviewed were accurate, 90 of the 5,067 published trials had underlying patterns that were unlikely to appear by chance in a credible dataset. The Guardian reports: The tool works by comparing the baseline data, such as the height, sex, weight and blood pressure of trial participants, to known distributions of these variables in a random sample of the populations. If the baseline data differs significantly from expectation, this could be a sign of errors or data tampering on the part of the researcher, since if datasets have been fabricated they are unlikely to have the right pattern of random variation. In the case of Japanese scientist, Yoshitaka Fuji, the detection of such anomalies triggered an investigation that concluded more than 100 of his papers had been entirely fabricated. The latest study identified 90 trials that had skewed baseline statistics, 43 of which with measurements that had about a one in a quadrillion probability of occurring by chance. The review includes a full list of the trials in question, allowing Carlisle's methods to be checked but also potentially exposing the authors to criticism. Previous large scale studies of erroneous results have avoided singling out authors. Relevant journal editors were informed last month, and the editors of the six anesthesiology journals named in the study said they plan to approach the authors of the trials in question, and raised the prospect of triggering in-depth investigations in cases that could not be explained.

Comments

Re:"90 of the 5,067"

[SharpFang]

Seems like artifact of randomness - Prosecutor's Fallacy.

Yes, some will be genuine falsifications. But some WILL be genuine results.

You write a paper on a list of 1000 tosses of a coin, noting each result. The chance for the coin to land on edge in one toss is around 1 in 100,000.

Then your paper is reviewed along with 100,000 others. If you have the coin land on edge more than once in your dataset, it's flagged as a falsified dataset.

Roughly 10 papers in the 100,000 tested flagged as falsified will be false positives.

------

Statistical results are subject to the same randomness as single tests contributing to these results. The scale of the randomness is reduced by a factor related to the number of tests, but still exist. And take enough correctly obtained statistical results, and you WILL find outliers.

Marcia Angell & Skepticism on Mainstream Scien

[Paul Fernhout]

http://pdfernhout.net/to-james... "The problems I've discussed are not limited to psychiatry, although they reach their most florid form there. Similar conflicts of interest and biases exist in virtually every field of medicine, particularly those that rely heavily on drugs or devices. It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine. (Marcia Angell)"