Slashdot Mirror
Studies Suggest Massive Increase In Scientific Fraud
Titus Andronicus writes "Scientific fraud has always been with us. But as stated or suggested by some scientists, journal editors, and a few studies, the amount of scientific 'cheating' has far outpaced the expansion of science itself. According to some, the financial incentives to 'cut corners' have never been greater, resulting in record numbers of retractions from prestigious journals. From the article: 'For example, the journal Nature reported that published retractions had increased tenfold over the past decade, while the number of published papers had increased by just 44 percent.'"
That is old news. Research in many areas of academic science has been mostly unreproducible for some time. http://dissention.wordpress.com/2011/02/06/why-all-publicised-breakthroughs-are-lies/
Discover something marginal with real research, then use photoshop and obscure statistical methods to make it look like you have a real discovery. Make outlandish claims about the prospect of your discovery revolutionizing everything.
This is so true, particularly in small or relatively new fields, and particularly in the "softer" sciences. I took a course a few years back concerning a relatively small subfield of cognitive studies (an area which intersects with another obscure discipline), and the instructor assigned a half dozen papers to read each week, and class members would present a summary.
Basically, the instructor ended up using the primary literature of the field to show us how not to do good scientific research. About 90% of the time someone would point out a major "significant" correlation, the instructor would ask: but how many correlations did they try? Sometimes, there would be dozens and dozens of potential correlations checked in the article, and the one or two that actually worked would be touted as of "major significance."
Except when you try that many things, chances are something's going to correlate with something else. If you set your threshold at 95% confidence (common in soft science experiments where you don't have enough funding to get a lot of subjects), you'll get a correlation from random data about 1 out of 20 times. If you do dozens of correlations, you'll always find something.
But that wasn't the worst of it. The experiments were often poorly designed, because as an interdisciplinary subfield, most of the researchers didn't actually understand both areas that well. But the ambiguous manipulation of data then was generally used to justify the most absurd claims in the discussion section -- sweeping generalizations about how these findings might revolutionize our understanding of how the brain works or some other incredibly broad statement (usually false on its face, because the experiment was almost always so badly designed that it couldn't even say anything about the tiny subfield itself).
And then -- the worst part. Future articles would propagate the absurd sweeping conclusions from the discussions sections as if they were fact. A decade later, many of these claims had become "accepted knowledge" in the field.
I'd say about 75% of the articles we looked at -- and almost all of them were frequently cited and published in the central journals of the field -- were guilty of some sort of extreme bias in experiment design, data manipulation, or grossly exaggerated conclusions.
I know these things are far less frequent in the "hard" sciences, but the things I took away from this course were (1) how to read scientific articles carefully, and (2) there's a lot of crap being published out there that is barely "scientific."