Study Shows One Third of All Studies Are Nonsense

SydShamino writes "CNN has a report on new research to confirm claims made in initial, well-publicized studies. According to the new study, about a third of all major studies from the last 15 years were subsequently shown to be inaccurate or overblown. The study abstract is available."

Comments

Missing the point

[kebes]

I feel like alot of posters are not understanding what the study is... this is probably because the abstract (or, if you have access, the actual article) is much more meaningful than the CNN report.

First, notwithstanding the many good jokes about a self-referrential study that will proven to be exaggerated, this study specifically checked whether highly cited clinical studies had claims that were later contradicted or softened due to other research. This study was not claiming that 1/3 of all scientific studies published were wrong in some way. It's worth noting that doing clinical research is very difficult, and that the error bars will always be quite large. It's also important to keep in mind that sometimes clinical research may be unduly influenced by financial pressures... and that clinical research undergoes very heavy scrutiny.

So having 1/3 of all clinical studies be later contradicted should not make us worry that clinical research is being done wrong. We should be happy that so much verification occurs, that any erroneous conclusions will (probably) be checked again. One line from the CNN article rings true:
Experts say the report is a reminder to doctors and patients that they should not put too much stock in a single study and understand that treatments often become obsolete with medical advances.

I think that should be the take-home message for the casual reader. Science is doing its job of verification, but people need to stop jumping to conclusions (or worse, changing their life habits) based on the results of a single study. The results need to be double-checked. The study may have been a fluke, or have flaws, or the data may have been manipulated. Whatever the reason, we should not trust single experiments, especially where human lives are at stake!

Having (partially) read the JAMA article, I think their result is sobering and useful. It really shows how intense the competition is in that field (which leads both to people making exagerrated claims, but also alot of pressure to dis-prove other's claims and get at the "right answer").

Re:Overblown

[uhoreg]

Where did you get 26.4% from? They looked at 49 studies. 45 of them reported that intervention was effective. Of those 45, 7 were subsequently contradicted, and 7 were found to report stronger effects. So that's 14 that are "inaccurate or overblown". 14/45 (which follows the percentages given in the abstract) is 31% (which is pretty close to 1/3). If you want to do 14/49 (the total number of studies they looked at), that gives you 28.6%.

Re:Science by press conference

[krbvroc1]

The slashdot abstract is incorrectly summarized. This study is reported in the Journal of American Medicical Association and relates to 'clinical studies'. The summary infers a wider group of studies.

Re:The topic is stated toooo broadly!

[flosofl]

[...]its worth a perusal to a skim

GAH! Someone needs to look at the definition for "perusal"

Perusal - To read or examine, typically with great care.

And don't get me started on your use of "its" (its=possesive it's=contraction of "it is")

I know, I know... Mod me +5 Pedantic. I've had a long day, and that felt good. :)