Slashdot Mirror
Dozens of Recent Clinical Trials May Contain Wrong or Falsified Data, Claims Study (theguardian.com)
John Carlisle, a consultant anesthetist at Torbay Hospital, used statistical tools to conduct a review of thousands of papers published in leading medical journals. While a vast majority of the clinical trials he reviewed were accurate, 90 of the 5,067 published trials had underlying patterns that were unlikely to appear by chance in a credible dataset. The Guardian reports: The tool works by comparing the baseline data, such as the height, sex, weight and blood pressure of trial participants, to known distributions of these variables in a random sample of the populations. If the baseline data differs significantly from expectation, this could be a sign of errors or data tampering on the part of the researcher, since if datasets have been fabricated they are unlikely to have the right pattern of random variation. In the case of Japanese scientist, Yoshitaka Fuji, the detection of such anomalies triggered an investigation that concluded more than 100 of his papers had been entirely fabricated. The latest study identified 90 trials that had skewed baseline statistics, 43 of which with measurements that had about a one in a quadrillion probability of occurring by chance. The review includes a full list of the trials in question, allowing Carlisle's methods to be checked but also potentially exposing the authors to criticism. Previous large scale studies of erroneous results have avoided singling out authors. Relevant journal editors were informed last month, and the editors of the six anesthesiology journals named in the study said they plan to approach the authors of the trials in question, and raised the prospect of triggering in-depth investigations in cases that could not be explained.
Thanks for that! Now I can use that tool to generate data for my upcoming fabricated studies.
Everything I write is lies, read between the lines.
That's... less than 2%. Naturally, we want it to be 0%, but 1.8% is nothing to generate scare headlines over.
"I don't know, therefore Aliens" Wafflebox1
So 90 of the 5,067 were outlier-like data and this is concluding results on these outliers.
I knew that publish or perish was ruining science, but this is actually the most heartening news I've heard of its credibility.
I've learned that less than 1.8% of these studies used non-extreme crazy data. My faith in science is restored!
>>"ad space available -- low rates!!!"
A study should be repeated by an org that has no skin in the game per results. They should be paid to test and get the same amount of compensation regardless of outcome. A random lottery should decide the head managers/researchers for any given repeat.
Table-ized A.I.
Luckily we can trust those implicitly, especially the model based ones.
Trust is not required. Full source code and input data is available for your inspection and verification.
It is fortunate that fraud (or incompetence) like this never occurs in other areas. For example think of the implications of this happening in Climate Science papers and studies. Luckily we can trust those implicitly, especially the model based ones.
Because when confronted with the evidence that 90/5,067 studies in one field (likely) contain fabricated data the obvious implication is that an entire field is fabricated?
Your conspiracy theory seems to be missing a few steps.
I stole this Sig
Those are only the ones that are easy to prove fake. There has been a lot of research over the years whose results simply cannot be reproduced even in an identical experiment. Big Pharma has been caught red handed several times now - at one point even publishing their studies in their own "peer review" magazine.
Seven puppies were harmed during the making of this post.
Hmm...
I am not a climate scientist. I am a retired scientist. What did I do? I modeled traffic. As strange as it might sound, there is a lot of similarity between the two. I will try to give some history, as it may help this make more sense. Sorry for the lack of brevity.
In my case, I helped bring traffic modeling to the age of computers. In this process, it was learned that you could improve the model results, significantly, by increasing the amount of data available. Even seemingly trivial things can impact throughput. Simple things, such as signage fonts, can impact throughput. Even the frequency of lane markings, reflectivity of lane markings, and coloration all have an impact on throughput.
To try to put this in perspective, I was working with data sets in the full TB size, before the turn of the century. We did distributed computing, before it even really had a name.
Why is that important?
Well, traffic is a bit like climate. It is a chaotic system. To be clear, a chaotic system is not a system that is random. It appears random but, with more data, you can tease out patterns and make deterministic predictions based on a variety of variables, with some levels of consistency and success.
I am not suggesting, for the record, that the climate science models are 100% accurate. In fact, they have confidence ratings. That goes underreported, but they will tell you how confident they are in the results.
Anyhow, that's besides the point. I just want to make it clear and avoid confusion.
What is important is that you have to massage the data. You have to make corrections to the data. You have to remove outliers.
See, we'd collect data and then run it against the models. We'd compare the model output with what was really happening. Sometimes, the results are pretty close. This means you can have greater confidence in the results. Sometimes, it isn't even remotely close.
At that point, you usually start by poking at the model itself. However, you will also poke at the data. You will throw some of that data right into the trash. You will normalize the numbers, and adjust the impact factor. You will also probably swear, like a lot. You will invent whole new languages, just to swear in them.
Either way, you will massage that data until you get the results that most closely match reality. You take existing data and run your models to see how well they match reality. When you get it to the point where you're confident, you use those methods to make predictions about the future, given new variables. This will have varied confidence levels, and pinpoint accuracy isn't expected by anyone versed in the science.
The truth is, you can model all you want but some drunk guy is still going to drive, in reverse, the wrong direction down a one way street. So, you only have so much confidence in the predictions.
The whole point is, you have to massage the data. If you don't, you get horrible results that don't match reality. The expected outcome isn't certainties. The expected outcome is predictions for which you can assign a confidence level.
I suspect part of the problem is poor communication and bad journalism. I've taken some time to examine the models, methods, and reasons. I am not a climate scientist, but I have taken a reasonable amount of time to study it in a scholastic manner. You can download their data AND their models, for free, and run them yourself. You can massage that data any way you want, too. You can apply all the adjustments you want and run the models yourself - for just the cost of hardware you already own and electricity.
Anyhow, I hope this clears a few things up. Correcting and massaging data is pretty normal. It's pretty much required, if you want meaningful results. I am pretty sure the uncorrected data sets are also available. You can get so many data sets, for free. They'll even give you the models. Hell, they'll even give you the source code for the models.
I do want to make it clear, the goal isn't a perfect pre
"So long and thanks for all the fish."
Scott Gottlieb, the current head of the FDA, wants to end drug trials. "The free market will put the bad actors out of business."
That's not fraud. Most of those studies are primary biology or animal studies, non-blinded. They tend to have sample sizes of around 10, and use sketchy stats. It's not particularly surprising they can't be replicated.
The stats should be improved, and they need to be more cautious in their conclusions (as does anyone reading them), but the scientific literature is supposed to be more about "hey guys, look at this, what do you think?" and less about "this is the truth!".
How the hell does he even have that many publications?!?
Probably doing a study on the effects of amphetamines.
“He’s not deformed, he’s just drunk!”
http://pdfernhout.net/to-james... "The problems I've discussed are not limited to psychiatry, although they reach their most florid form there. Similar conflicts of interest and biases exist in virtually every field of medicine, particularly those that rely heavily on drugs or devices. It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine. (Marcia Angell)"
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
Not OP but I'll give you a better example. Let's say you are processing the temperature data of all of the US over 20 years, tens of thousands of weather stations with multiple readings per day. It's likely that some of those malfunction every now and then. How do you know which readings are correct and which are not? Well you can eliminate the outliers. You take an area and you look at the temperature readings - if they are all around 75F plus or minus 3 degrees, but you have one that's showing 90F, there is pretty good change that one station is busted. That's exactly how they deal with the urban island effect. Temperature readings in cities are higher due to high area of pavement/concrete, so instead they use the readings from a rural area nearby. In the end you don't care about the ups and downs, you smooth those over and you look at direction of the average. I
I am not a climate scientist - I feel this needs to be made clear. I have made a quasi-scholastic study of climate science, largely to see for myself what the fuss was about. I've read a whole lot of papers, a whole lot of research, and watched a whole lot of talks. Oddly, I never watched the Gore movie. I prefer to listen to the scientists.
Which leads me to...
The theory behind it is pretty sound. It can be reduced to some pretty simple physics and math. If you put more energy into a system, it's going to transfer some of that energy in the form of heat. The effects of greenhouse gases are, fundamentally, well understood.
And then it gets complicated as all hell.
I am unqualified to opine on the affects from human output. However... The research that I've read makes a pretty good case for it and I didn't see (once I dug deep enough) anything that reeked of bad science. Note: Some does exist but, for the most part, I found that coming from proselytizers and journalists, as well as some who appear to have moved from academia to publicist.
There's a pretty big difference between what I read for research and what is parroted by those who have decided men in lab coats are the new priests. The research is more nuanced and actually lists things like confidence ratings. Where the research may say 2, 4, or even 6 standard deviations, the journalists will take even the weakest confidence conclusions and present them as factual certainties. And, of course, this gets parroted and treated like the kid's game of Telephone.
What baffles me is the amount of alarmists who claim to support science while parroting bad science - or just plain inaccurate science.
Not that long ago, I had someone telling me that the oceans were going to rise 27' within the next fifty - and that this was a fact. So, I handed down citations and demonstrated that there's actually no research supporting such absurdities. They immediately decided I was anti-science and a Trump voter. I am still dumbfounded.
I, who seems to actually understand the science to some extent, don't deny that the climate is changing. In fact, it's pretty evident that it is changing. Hell, I can even understand the science that suggests it is caused by releasing CO2 into the atmosphere. Yet, because I didn't toe the line - and agree with their kinda strange assertion, I was lumped in with the deniers and, even more strangely, assumed to be a Trump voter.
I don't know? I just don't know, at that point. I will say, I can sure as hell understand why people would be skeptical. I can sure as fuck understand why people would negative in their responses. The oceans aren't gonna rise 27' in the next fifty years. Not even the most dire of predictions suggests that.
Again, I'm not a "denier." I've done a lot of reading and researching. I'm not a client scientist, but I am a scientist. I didn't see anything wrong with their conclusions - and there are many, many studies. Some of those studies have data that hasn't been manipulated, to any great degree (as far as I saw). There's a whole lot of research on this.
If I were qualified to opine, I'd suggest that they're on the right track and that we are releasing enough CO2 to have an impact. How much? What will the results of our impact be? Is there a tipping point? Those are things I can't really say. I can say that it does look like the science is fundamentally good.
Why yes, yes I am being careful with my verbiage. ;-)
"So long and thanks for all the fish."