Studies Suggest Massive Increase In Scientific Fraud

Titus Andronicus writes "Scientific fraud has always been with us. But as stated or suggested by some scientists, journal editors, and a few studies, the amount of scientific 'cheating' has far outpaced the expansion of science itself. According to some, the financial incentives to 'cut corners' have never been greater, resulting in record numbers of retractions from prestigious journals. From the article: 'For example, the journal Nature reported that published retractions had increased tenfold over the past decade, while the number of published papers had increased by just 44 percent.'"

Comments

Surpised?

[Vinegar Joe]

There's more money in it now.

Re:Surpised?

[Anonymous Coward]

I am not sure about how much money is in it now compared to previous times. I work at a place that does scientific research and I know that the people around me regularly put a lot of work into proposals for which they have no guarantee of funding. These are good researchers with good projects. But they have to compete for most funding opportunities. Then you add in the issue of the politicization of funding. No, I am not going to make this about global warming. What I mean is that some who hold the purse strings have a pet subject and will put large amounts of funding into it while starving other, equally worthy subjects. So you have an excess in one area while another is not getting proper attention.

Re:Surpised?

[pz]

There's more money in it now.

Actually, it's quite rather the opposite: there's not enough money, so competition for scant funding is intense. Two days ago I saw a presentation by Dr. Francis Collins, head of the NIH, who was really trying very hard to put a positive spin on resarch and the source that it provides for economic recovery. He was trying really, really hard. Why? Because if you look at the inflation-adjusted budget of the NIH, it's been going down ever since 1978, and is currently closing in to about 20% off the peak. In the meantime, the number of applications has skyrocketed to the point that fewer than 25% of applications are being funded. In my subfield, that number is closer to between 7 and 9%. When competition is that fierce, the temptation to fudge data is huge.

But his arguments were solid: there has been rarely a better ROI on governmental programs than the NIH budget with a factor of at least 2x overall (each $1 in NIH budget results in $2 in GDP), and individual cases that are well over 100x (like the Human Genome Project). Research, nationally funded research, is one of the basic means for seeding long-term economic growth. If you are in biomedical science or its related basic fields, you should contact your congressmen and insist that the NIH and NSF budgets be increased: we need another doubling, like we saw during the Clinton administration.

Re:Surpised?

[PingPongBoy]

There's more money in it now.

On a different tack, rather than money, it may be due to another theory of economics, the law of diminishing returns. As more discoveries are made, it becomes harder to make discoveries, but with the human population growing at least linearly and the population of researchers keeping pace, the rate of good research results is under great pressure to keep up. Add to this the specter of funding cuts and people not wanting to lose their research jobs, and the sheer volume of research results being reported. Human nature completes the syllogism: there will be more falsification.

Re:Surpised?

[Smidge204]

I would say it's just the opposite. Contrary to what might seem obvious in other situations, MORE money could actually DECREASE fraud in science. There is less money - at least public, "no strings attached" money - available. It would not surprise me at all if societies that invest more public money into research see less fraud.

If you're a scientist you are increasingly pressured to get the results your funding corporation/institution is paying for and to do it within crushing schedules and shoestring budgets. That's not to say all studies are trying to reach a particular conclusion (though some clearly are like that), but often a study is simply inconclusive... but inconclusive studies don't make the bean counters happy.

Science needs materials, equipment, staff and time. Give them what they need, stand back, and you'll get good results. Might not be the results you want, but that's a risk you'll have to accept.
=Smidge=

Dystopic Reward System

[Advocatus Diaboli]

That is old news. Research in many areas of academic science has been mostly unreproducible for some time. http://dissention.wordpress.com/2011/02/06/why-all-publicised-breakthroughs-are-lies/

Re:Dystopic Reward System

[jd]

This is why I would argue for a shake-up of how science is funded and how papers are refereed, published and post-publish reviewed.

Science should NOT be corporate-funded, it should be grant-funded -- directly from a scientific organization like NIST, or indirectly via university (or other educational) departments. Corporations should be entitled to push money into a grant pool and should also be entitled to suggest problems to study, but there should be absolutely NO link between the providers of the money and the providers of the science. Scientists MUST be free to say a claim is wrong, obtain negative results or otherwise get results corporations aren't going to like. Sorry, the universe doesn't give a flying what your CEO says.

A paper should NOT be considered as having been refereed until the work has been reproduced. But what constitutes reproduction of a result? At least some forgeries have involved people taking prior published papers and doing a cut-and-paste on the tables of results. The values now necessarily agree. Is that reproduction of results? No. Conclusion - a copy of the lab notes during the experiments should be placed in escrow with the journal. Once the peer reviewers have also submitted their lab notes, the complete collection is released to a second-stage peer review to determine if the collection suggests anyone "cooked the books". Only when a paper passes second-stage review is it published.

Next, there need to be central scientific libraries that collect ALL journals (regardless of obscurity), ALL reviewed lab notes, etc, making that information available to absolutely anyone, with PROPER linkage between research (Semantic Web has nothing on this!). Journals will claim they need to make a profit -- fine, embargo new publications for N months after pay-per-view publication. Since I'm arguing for quality indexing, and given that takes time, such a library can't publish instantly anyway.

What to do with negative results, though? Journals hate publishing those. So, have the central funding agencies ALSO fund an "open journal" that ONLY publishes negative results. Journals can't complain that it's competing, since there's no overlap.

Ok, but even with all of that, nobody has time to read every paper and certainly nobody has time to go back and correlate current science with past papers even if all this information was available. Doesn't matter. If there's a central store of everything, and that everything is properly linked up, the reasoners that have already been written for Semantic Web logic will work on those links to determine if the data is internally consistent. That information can be passed back to the funding agencies to determine what experiments are needed (if any) to identify what results are good, what ones are fraud and what ones are merely incompetent.

This sort of framework is relatively open (anyone can join as a publisher, anyone can join as a researcher, anyone can throw money into the pool), but more importantly the information is open and the information lifecycle is a closed loop. Even if the majority of past data is bad in any given field, this system would make bad data unsustainable because it can't pass through a two-stage review anything like as easily as it can a one-stage because the criteria differ, and even if it did get through, it then has to handle an automated consistency check.

Yes, this is serious infrastructure we're talking. However, science journals cost many times more to publish in than open journals (roughly, $8,000 an article less, assuming the typical conversion rates). You don't need to hand that many papers being published before the cost of all the infrastructure needed matches the amount saved. The money then saved from eliminating the bad science then becomes pure profit, which can be ploughed into new work.

Re:Dystopic Reward System

[jd]

Much, perhaps most, scientific fraud in published studies has little to do with corporate R&D.

Cite, please. All references I can find are specifically to do with corporate sponsorship with journals and corporate sponsorship of reseach.

It's this "business/capitalism-is-the-root-of-all-evil" Marxist reductionism that is getting really tiresome to read. I get it. You don't like business.

This must be why I mentioned corporations putting money into the central pot. If there's any ideology here, it's yours, since you have evidently taken a few things utterly out of the context in which they were placed and imposed your own idea of what I "must have" meant according to some fantastically inaccurate wall-chart of phrases-to-politics.

And what did you even want to propose, anyway?

I said what I wanted to propose. In detail.

No private scientific R&D?

Plenty of private R&D in this framework. Private but decoupled.

Are you mad?

Those who have marked me as "foe" on Slashdot would say so. Those, like you, who simply don't read what I write and prefer to imagine what you want me to have written - well, that used to make me mad. These days, it makes me wish I could emigrate to Mars on the basis that microbes and amoeba offer better conversation.

nope

[geekoid]

retractions is a bad measurement.

More and more data is open and available, so when 1 person committed fraud, it impacts many papers that come after it. The paper aren't committing fraud, there the victim of the first guy.

So I could commit frauds, and after 10 year it could impact 100 papers.
So retraction is a very poor way to determine this.

Re:nope

[sphealey]

= = = Indeed. I'm beginning to suspect these claims of widespread fraud have more to do with some pretty bizarre metrics on the part of those making the claim. It makes great headlines, but I think there's something rather fishy about it. = = =

Lot of pushback on the so-called "fraud epidemic" on the academic science blogs. The emerging concensus is that the campaign is part of a softening-up process for anti-climate science actions.

sPh

Re:nope

[Black Parrot]

retractions is a bad measurement.

You do have to consider other explanations, e.g. maybe the internet makes it easier for scientists to get their hands on sufficient information to detect fraud, or maybe even journals have become more responsible about retracting articles one they're shown to be bad.

Orthogonal lines of evidence would indeed be useful for understanding what is going on.

Re:nope

[MightyMartian]

I suspect an investigation into journalistic fraud would be far more fruitful. But I tend to agree with you, only because only those miserable fuckers at the Heartland Institute would spread around a lie like "retraction == fraud".

Re:nope

[WhatAreYouDoingHere]

I think this study itself may perhaps be an example of the scientific fraud that they are describing.

Is that irony?

Plausible...a trend that is self amplifying

[ShooterNeo]

The more scientists who commit fraud and outcompete honest scientists for funding, the higher the bar becomes for the honest scientists. With dwindling tenure positions (and far more scientists competing for those positions), in order to be considered for tenure you have to meet very high productivity standards : a large number of peer reviewed papers in high-impact journals.

Well, real research takes time, money, and if it's good research, it will FAIL most of the time. It HAS to fail...to find something truly new you have to leave the bounds of existing knowledge, and most solutions anyone attempts are going to fail. The only way to guarantee an experiment will succeed is to :

            1. Research something you really already know the answer to. Hence the popularity of further research on the dangers of smoking. Throw a dart at a picture of a human body, check if someone else has researched it, if not, check. You will "discover" that cigarette smoke is quite harmful to or increases the prevalence of . This kind of research is not fraud, per say, but is really boring to high impact journals SO
            2. Discover something marginal with real research, then use photoshop and obscure statistical methods to make it look like you have a real discovery. Make outlandish claims about the prospect of your discovery revolutionizing everything.

And so on. The problem is, there ARE real discoveries made, every now and then, that would be huge IF large sums of money were spent to develop the REAL advances. But, if you have 10 fakers for every legitimate discovery, and you try to fund them all equally, most of the money gets wasted and so we live in a society without effective treatments for cancer, without a cost effective way to reach low earth orbits, without any of the other things that technology theoretically could make possible.

Re:Plausible...a trend that is self amplifying

[AthanasiusKircher]

Discover something marginal with real research, then use photoshop and obscure statistical methods to make it look like you have a real discovery. Make outlandish claims about the prospect of your discovery revolutionizing everything.

This is so true, particularly in small or relatively new fields, and particularly in the "softer" sciences. I took a course a few years back concerning a relatively small subfield of cognitive studies (an area which intersects with another obscure discipline), and the instructor assigned a half dozen papers to read each week, and class members would present a summary.

Basically, the instructor ended up using the primary literature of the field to show us how not to do good scientific research. About 90% of the time someone would point out a major "significant" correlation, the instructor would ask: but how many correlations did they try? Sometimes, there would be dozens and dozens of potential correlations checked in the article, and the one or two that actually worked would be touted as of "major significance."

Except when you try that many things, chances are something's going to correlate with something else. If you set your threshold at 95% confidence (common in soft science experiments where you don't have enough funding to get a lot of subjects), you'll get a correlation from random data about 1 out of 20 times. If you do dozens of correlations, you'll always find something.

But that wasn't the worst of it. The experiments were often poorly designed, because as an interdisciplinary subfield, most of the researchers didn't actually understand both areas that well. But the ambiguous manipulation of data then was generally used to justify the most absurd claims in the discussion section -- sweeping generalizations about how these findings might revolutionize our understanding of how the brain works or some other incredibly broad statement (usually false on its face, because the experiment was almost always so badly designed that it couldn't even say anything about the tiny subfield itself).

And then -- the worst part. Future articles would propagate the absurd sweeping conclusions from the discussions sections as if they were fact. A decade later, many of these claims had become "accepted knowledge" in the field.

I'd say about 75% of the articles we looked at -- and almost all of them were frequently cited and published in the central journals of the field -- were guilty of some sort of extreme bias in experiment design, data manipulation, or grossly exaggerated conclusions.

I know these things are far less frequent in the "hard" sciences, but the things I took away from this course were (1) how to read scientific articles carefully, and (2) there's a lot of crap being published out there that is barely "scientific."

Re:Plausible...a trend that is self amplifying

[jd]

Agreed. However, it's all about "survival of the fittest". The current system favours the least work (since doing less means you can write more, and writing more means a higher citation score, which in turn means more funding), the work least likely to fail (negative results don't get published) and the work least likely to contradict prior work (repeat studies also don't get published).

In order for quality science to survive, it HAS to be the fittest for purpose, which means we've got to change the purpose so that the above three flaws are selected against and not for.

Retraction != Fraud

[yesterdaystomorrow]

In my experience as a scientist, what has increased is the pressure to publish quickly. So, people publish results that haven't been checked as much as they perhaps should be. But this is not fraud, and perhaps it's even healthy. Better to get crazy results out there than bury them in notebooks: sometimes they turn out to be major discoveries.

The fish rots from the head down...

[ibsteve2u]

When the most corrupt people in America include so many of our most powerful politicians, corporate CEOs, and Wall Street barons it is unreasonable to expect any facet of American society to remain unaffected. The only and only thing you can be sure will "trickle-down" is corruption as the system has been rigged by the corrupt to ensure that it is corruption that pays the big bucks in America.

not necessarily fraud

[Anonymous Coward]

Perhaps the journals just don't do enough due-diligence anymore? The rush to publish in a world with 24-hour news and the internet...

Fraud is not the only cause of retraction

[damn_registrars]

Even good studies can have aberrant results that start with promising findings and end in retraction. The fact that retractions are up is not inherently indicative of more fraud, it could just as well be indicative of more pressure and more thorough peer review.

Re:asia is real big on tech the test and cheating

[Black Parrot]

asia is real big on tech the test and cheating

Since NCLB linked money to performance on tests, US school districts have become big fans of cheating.

Re:Numbers are blown out of proportion

[ilguido]

So there's 196 papers retracted since 2001?

What? They put a nice graph to make it clear even to condensed matter physicists. There are 742 retracted papers in ten years (2000-2009), in the PubMed database and they increased from 3 in 2000 to 180 in 2009. 196 were fraudulent papers, 235 included some mistakes (they can't tell if those were intentional or not) and 311 were retracted for other reasons (including: those poor guys that based their work on prior forged papers).

what planet are you living on?

[damn_registrars]

There's more money in it now.

While more money is spent in science, the scientists themselves have in general not had a meaningful raise in some time. Anyone who goes in to science to make money is, to say the least, misguided. Scientific research is often the least profitable venture you can pursue with a PhD.

The additional money being spent in science is largely going to keep the lights on in the lab. Scientists need to pay for their utilities and consumables, all of which have risen in price while their wages generally have not.

Re:what planet are you living on?

[sandytaru]

They also have to fight and squabble for that money in ways they never had to do before, and they're under severe pressure to produce results, any results, within a certain length of time. Not that any scientist at a research institute should automatically get full funding, but they should be funded on a per project basis, instead of for a specific amount or a specific length of time. A hundred thousand dollar grant sounds great, but that's money the scientist doesn't see - it goes to pay the graduate assistants (who are eking out a living at near minimum wage while they finish their own degrees), the materials, the lab fees to the university, etc. A hundred thousand dollar grant will cover perhaps a year of research. The researcher is thus pressured to publish the results of the experiment within that one year, even if the experiment isn't actually done.

Re:what planet are you living on?

[Niedi]

I honestly cannot remember when I've heard so much misguided pseudoscience stuff for the last time.
Let's start.
No papers sceptic on global warming? I was able to find a couple of 100 released since 2011 by just spending 5 minutes on scholar.google.com.
Non-coding DNA? There's an even simpler explanation: Regulatory sequences. LOADS of them. Each cell only needs a VERY small subset of the proteins encoded. How does the cell know which ones it should express and how many of them? How do the controlling proteins know which sequences they should control? Regulatory elements. (That was was oversimplified, but you get the idea). There is no high-level/low-level DNA, no compiler, no linker. There are epigenetic modifications and posttranslational modifications, but these are for ensuring correct amount and function/transport of the proteins.
The creationist thing is a strawman. Why would evolution set up a compiler as you suggested it? Because it would allow quicker changes to adapt to different environments which is beneficial for the one having such a thing. In fact HUGELY beneficial. In the same way you could argue that having eyes, ears, etc... is an argument for creationists
The way too complex parts of the genome? Sorry, citation needed.
The bacteria thing? They are much simpler in what they can do with their toolset. The thing they can do a lot better due to their simpler construction and high mutation rate (when you take bacteria as a group) is to adapt almost any condition. And in that respect they already "won". In our very own body, bacteria outnumber our own cells by a factor of 10 (wikipedia). Bacteria exist in practically any environment on this planet. So what was your point again?
I'm not into the transmitter/reciever thing so I can't comment on that other than "citation needed".

And of course denialists use globalwarming tag

[Ranger]

on this story. Asshats. Seriously? How many of those retracted papers dealt with the studies relating to climate change?

Re:And of course denialists use globalwarming tag

[Daniel Dvorkin]

And some other asshat has to drag out the term "denialist".

There's no other shoe that fits. If you want to propose a different word, feel free to do so -- but you can't have "skeptic," because that means something different, sorry.

information about information and leverage

[slew]

It might be interesting to think about the ways that increases in scientific fraud parallels the recent financial industry meltdown that resulted from the mortgage industry mess.

In the mortgage industry back in the old-old days, when you wanted to borrow money, you took your information (w2, bank account statements, etc,) down to the local bank which analyzed your finances and issued you a loan based on thier "gut" feeling on your credit worthiness. This was found to be a very non-scaleable, often discriminatory system, however the risk was localized therefore immediate feedback was available (banks that issued too many bad loans failed).

Then the industry evolved. Credit reporting agencies and credit scores were created to reduce discrimination, and automate decision processes and help quantify risk, and packaging was created to securitize loans which effectively aggregated and anonymized both borrowers and banks and attempted to present an abstract risk profile to folks investing in debt. The risk/return profile of this investment created a high demand for more securitized loans, creating a scarcity. What happens when demand exceeds supply? Either the price goes up (the yield of the debt investment goes down when the price goes up), or some risk takers will attempt to increase the supply by substituting marginal quality goods (loans that aren't well vetted). Then when others see their success with marginal quality goods, even the regular suppliers take the plunge and drop their quality to maintain their market share. Large coalitions enter the field and start to game the system. The lack of information available to the investors due to anonymization and aggregation amd increased leverage (firms started using derivatives and CDOs to invest in mortages) set us up for the financial industry fall. Then the cards all fell down.

Historically, scientific publishing when you wanted to get your paper published, you sent a pre-print to a journal and they attemped to referee the paper based on the "gut" feeling of their reviewers. This was fairly unscalable and often discriminatory system, but the risk of a poor quality paper was localized to the journal (basically journals that published too many bad papers would lose credibility).

We are in the midst of an evolution in scientific publishing. Now there are many mroe researchers and many more journals. Many journals don't have the staff to do a good job a vetting the papers, and the specialization, cost and expense of many research fields make peer-review "santity" checking across different research groups difficult. Ironically, as we have more information about science, we have less information about the quality of that information. Since published results attract scarce research dollars, the cost of doing good research that results in published papers go up (reducing the ROI on research dollars), or some risk takers will attempt to attract scarce research dollars with sub-quality work... and so on...

Let's hope that large coalitions don't enter to game the system, nor research grants are anonymized from author and institution as researchers move around and institutions do joint projects, nor that large research projects leverage questionable earlier research w/o information or verification or we may be building a similar house of cards with scientific research literature. Isn't scientific literature supposed to all be about leverage (standing on the shoulders of giants)? Aren't certain publication too-big-to-fail? Aren't large research coalitions monopolizing areas of grant money in certain fields and effectively owning the available peer-review resources? Maybe we've already set the table and just don't know it yet.

Some food for thought...

Re:This Week's Conservative Nonsense on Slashdot,

[Anonymous Coward]

I'll answer that for him!
A conservative is a middle-aged white male strawman who hates Science, thinks the Earth is 6,000 years old, believes in every major religion, murders at least 5 black people a day, gets to vote 100 times in each election, is richer than Scrooge McDuck, has an IQ equal to their geographical latitude, and is secretly gay. This demographic composes 50% of the population of America.

Re:Decent validation

[SomeKDEUser]

Gahh. Pet peeve of mine: I have come to the conclusion that people on the American side of the Atlantic don't understand what a model or a theory is for.

Science is not a collection of facts: it is a collection of theories supported by facts. When someone tries to publish something without model or explanation, it is your duty as a reviewer to reject the article with great prejudice. Because it it the theories that advance science.

This is because although the experiments will not get repeated (sure, they might if we scientists had job security and enough funding -- won't happen), the theories and models will get tested with new experiments. And this is really how science advances. Real science is the formulation of theories and not testing randomly new drugs: this also has marginal utility, but can never be as solid as, say, the theory of gravitation. Therefore, don't be surprised when people publish results that turn out to be a fluke, when they are pressed for time: this is because the reviewers accepted papers which were not framed within theories. Models and theories based on first principles are the only thing one can be reasonably certain of...

I suspect that the reason climate science and evolution are misunderstood in America more than elsewhere is that the education system here does not emphasize systematic knowledge and the power of models. Evolution cannot be "experimentally proven". but it can be used to formulate a great number of hypotheses which can then be verified experimentally.

Re:This Week's Conservative Nonsense on Slashdot,

[damn_registrars]

I can say that you're wrong. Do you even know what "conservative" means?

In this country the conservatives are often looking for excuses to further decimate the already very lean scientific research budget. This article provides another one of those excuses.

Re:asshats vs stupes and crooks

[Ranger]

No attempt at humor. I'm not going to be offended by you calling me willfully ignorant, because I didn't give you enough information. I've just noticed there seems to be an editorial bias on slashdot towards stories that deny anthropogenic global warming. I'd seen the science paper retraction story on another site. It is indeed disturbing, but it does show that the process of science still works. If those bad papers weren't being retracted then we'd be in a much worse situation. As soon as I saw the globalwarming tag, I knew the asshats who put them there were still trying to cast doubt on that humans were changing the climate. I'm not seeing anything in the article about retractions of papers on global warming, or rather about the study of ice cores, of tree rings, of sediment cores, of analysis of weather records over time--all the disciplines that are studied to form a picture of our climate and to predict what is happening.

The rise in retractions seems to have almost become exponential over the past 6 years. That we knew and had predicted humans were causing climate change by increasing the warming of the climate was pretty well understood by the late 1980's. In fact many of the predictions have been born out since then. We aren't seeing retractions of papers from the 1980's, but the late 2000's.

Anyway, here is an excellent resource called The Discovery of Global Warming (See the timeline: http://www.aip.org/history/climate/timeline.htm). It's also a book. I hope that clarifies where I stand on the issue and why I called them asshats. Because only an asshat would try to make a connection. And I'm still waiting for that list of retracted papers about global warming.