Slashdot Mirror
How Common Is Scientific Misconduct?
Hugh Pickens writes "The image of scientists as objective seekers of truth is periodically jeopardized by the discovery of a major scientific fraud. Recent scandals like Hwang Woo-Suk's fake stem-cell lines or Jan Hendrik Schön's duplicated graphs showed how easy it can be for a scientist to publish fabricated data in the most prestigious journals. Daniele Fanelli has an interesting paper on PLoS ONE where she performs a meta-analysis synthesizing previous surveys to determine the frequency with which scientists fabricate and falsify data, or commit other forms of scientific misconduct. A pooled, weighted average of 1.97% of scientists admitted to having fabricated, falsified or modified data or results at least once — a serious form of misconduct by any standard — and up to 33.7% admitted other questionable research practices. In surveys asking about the behavior of colleagues, admission rates were 14.12% for falsification, and up to 72% for other questionable research practices. Misconduct was reported more frequently by medical/pharmacological researchers than others. 'Considering that these surveys ask sensitive questions and have other limitations, it appears likely that this is a conservative estimate of the true prevalence of scientific misconduct,' writes Fanelli. 'It is likely that, if on average 2% of scientists admit to have falsified research at least once and up to 34% admit other questionable research practices, the actual frequencies of misconduct could be higher than this.'"
Scientists are humans too and a job won't change some humans from being cheats.
It is often cited that crappy, broken or incomplete code is often shoved out the door by business in order to meet deadlines. Quality or even truth are sacrificed for business reasons.
Why would R&D be any different? Big business often exhibit quota and other incentives for patent filing and the like. Outside funding sources pressure even pure research activities so that they can get their hands on new technology or even for silly things like a name being recorded as "first to" do something.
I am actually a bit surprised that the numbers aren't a bit higher.
If we accept that scientists are human like anyone else, we accept that scientists, like others, will make mistakes that get bigger and go more wrong than they anticipated. Some may intentionally commit fraud.
How common is scientific misconduct relative to other types of misconduct seems a more relevant question.
Also: What can we do to decrease it and how can we lessen its impact.
But if there are so many examples of scientists providing fake data how do i know the results of the survey in the FA are correct?
And how exactly are we supposed to believe her study?
The television will not be revolutionized.
and up to 33.7% admitted other questionable research practices.
I wonder if this refers to shortcuts taken because its common knowledge, Such as, if you use water as a control lubricant, you might test its wetness, density, purity, viscosity, etc, to compare against water with a slippery polymer in it. I wonder if these "questionable" practices involved taking distilled water, making sure its pure distilled water, and then pulling the other factors off of charts for distilled water or if "Questionable" means something far worse.
The reason i bring this up is because hindsight is 20/20 and everybody knows every mistake that they've made, if they're smart and that's what they're fessing up to.
Is it sad that I am more likely to recognize you and your posts by your sig than your name or UID?
True, giving a certain "spin" to you interpretation of correctly presented data is common - but not necessarily a terrible thing. As you said, it it will be scrutinizied and filed in the big "misinterpretation" folder. As for active misconduct - it probably happens more often than reported, but thankfully gets caught internally most of the time before it is published. I can only offer anecdotal evidence, but while doing my PhD work, one of my colleagues tried to get away with made-up results. Head of department smelled a rat, checked the data and promptly fired the guy without hesitation. PhD student one day, unemployed with revoked visum the next day....
Ubi solitudinem faciunt, pacem appellant.
Disclaimer: I'm a scientist.
Scientist will behave much better as soon as society (or perhaps the government at least) understands that if you want reliable information, you actually have to treat your scientist well.
Now, do not got me wrong, some countries, especially the US, invest quite a lot in science. But the problem is that the whole system is rotten to the core. It makes almost no sense at all for a young graduate to stay in a University/Institute. Pay will be low, and you have (in most countries) no job security. In Europe you either get a nice job at a company, or you go around taking post-docs for 5-10 years, hoping to get lucky. Working crazy hours with no holidays. For most, in the end, they go to a company anyway (having lost quite a lot of money in the process).
Often you are expected to go abroad, and unless you are lucky this leaves you with no good way to take care of your pension. Then if you want to return, somebody else took your place at university.
There is 2 ways to stay in the system: either you are lucky or you lie like hell.
Now, people may say that if your good you do not need luck. But remember that for high impact publications you need a lot more then good ideas and good skills. In research it is perfectly normal to conclude after 2 years that your hypothesis is false. This is great science, it also is hardly publishable in a good journal. People like positive results, and the reviewer system actually encourages you to confirm generally accepted ideas, not to falsify them.
Well, I could go on but I am sure others will.
To be honest, I do not even get angry anymore when I suspect someone may have done something "questionable". It's just sad.
The truth is the way that scientific institutions are set up isn't very scientific. There is definitely an attempt at oversight and impartiality but it's very easily corrupted by a wide variety of people with a wide variety of interests and ulterior motives. There aren't nearly enough checks and balances.
There are many things wrong with the system. Some include:
- Almost anyone can commission a study, write a book etc. and it's left to the scientific community to place value on that work. Viewed on it's own, without knowledge of the scientific community's opinion it can be difficult to tell how valid the work is. For example Wolfram's "New Science" has been largely debunked as mostly a rehash of old ideas (minus accreditation) but it took some time for this to become clear and in the meantime it was popularized in the press as a breakthrough work.
- The only real form of moderation is whether or not work has made it into a respected journal. Other scientists are then expected to publish corroborating work etc. However, until this is done, it is very difficult to judge the validity of the work, and papers get published that are later discredited. (Cold fusion anyone?) Likewise, work that should be published is often initially rejected. The primary motivation of a lot of the scientific journals is financial gain. In fact the entire publishing system is an antiquated remnant of the last 2 centuries and doesn't belong in an Internet connected world, yet publication is still the primary tool by which a scientist's work gets recognized.
- Speaking of antiquated the institutions, committees and governing bodies of science are about as scientific as a mother's group - it's all professional bitching and posturing for status. Real monkey hierarchy stuff. A lot of decisions get made on the basis of status. It's particularly bad for applied science professions like the medical profession where you hear stories about doctors who should have been prevented from practicing continuing for many years before being disciplined or quietly removed. At the senior level, scientists are often more politician than anything else as then need to secure funding and approval from political bodies. Then you see students who have to work their way up in status being treated like crap "paying their dues" as noted in a story posted a few days ago about a student who died in a chemical fire.
- Speaking of status, there is an emphasis on using scientific jargon to exclude the community at large. Some scientific ideas require complex specialized language and university post graduate mathematics to understand, and so require such specialized language. However even simple concepts must be described in overly complex specialized language to be accepted for journal publication. This is absolutely backward. We should have a system that requires simplified language where possible and a layman's overview attached early in the document. Instead, reading a scientific paper if you're not a specialist in the field is an art that you learn when you do post graduate work. If you assess a published article for readability you'll find the statistics you generate tell you that it's dense and difficult to understand. There are journals and subjects that allow simpler and informal
language but they are the exception rather than the rule and usually apply as addendum publications for applied fields. (Again I'm thinking of medicine. My own post grad work is in astronomy so I'm very much a lay reader when it comes to medicine, and when I've tried to read medical papers it's usually been an interesting excercise). Any real simplified content seems to get presented in slide form at conferences and presentations are often a better way of getting an overview.
I could go on about the shortcomings of various scientific institutions but I won't.
My point is that when you have a system that is so open to corruption, with so few checks and balances, and so much baggage inherited from institutions that began in the dark ages, it's no surprise that you end up with science that's much less than perfect.
These posts express my own personal views, not those of my employer
Definitely they sometimes fudge their data so that it will support their theories. Scientists are human and not perfect, it's part of human nature. That is where peer review comes in. A true scientist s work has to stand up to peer review and this is where the fudging of data is often uncovered. The problem is that much of the research going on is cloaked in secrecy by governments and corporations and proper peer review doesn't happen.
This brings to mind an incident in history where the scientist was right but his data was just too good. I'm talking about Gregor Mendel and his work on genetics. Later statistical analysis of his data indicates that it was very unlikely that he got that data. He probably got very close to the experiment result that he predicted but it was not good enough so he fudged his results. It wasn't until long after that this inconsistency in the data was uncovered. Was he right? Absolutely he was but his data is suspect nonetheless.
At least in science there is a built-in way of self-correction. Publish all the made up crap you want, but when no one can duplicate the feat don't be surprised when the community calls you out on it. Tell me where you go to find the guy double checking the work of the corrupt police officer or judge when they perjure themselves to ruin your life and your ability to defend yourself. Find me the people replicating every aspect of your grafty mayor's work to make sure he's not full of shit...
I can't think of anywhere else in life that there are as many checks and double checks and accountability as in the field of scientific research. Just because no one catches it immediately means nothing. If it was fake no one will be able to replicate it. A single study proves very little and likewise does very little damage, so if no one cares enough to replicate it chances are slim that it will cause harm.
when I did chemistry at 6th form college (UK term, in US I suppose you'd call it senior high?), I recall doing a practical test in chemistry (titration) where you had some mystery chemicals and a colour change. the experiment was rigged so that it was somewhat like a reaction we'd already seen, but was in fact something quite different. the instructions were to make accurate measurements first, draw the appropriate graphs and *then* speculate on the mystery ingredients.
it turned out that we'd never encountered the particular reagents before, and if you did the test accurately you'd have realised it wasn't the old familiar reaction, but had to be something new - the figures would simply not add up. however, a significant number of people rejigged their results to match the known reaction and failed the test totally for two reasons, first being for failing to make accurate measurements and secondly for faking the results.
Can we please put a stop to all these people citing peer review as a sort of wonder cure?
I peer review a lot of papers. And yes, it catches a lot of bad science. But most of that is just, bad experimental design, bad writing skills, wrong conclusions, uninteresting stuff, etc.
There is nothing I can do against some smart guy who makes up all the numbers, but knows enough of statistics to make it look plausible. It is often not feasible, or even impossible to redo the experiments. I never heard anybody do that anyway (maybe because you get 2-3 weeks to do your review, whereas the work would take half a year at least).
There's a big gray area. For instance, the Millikan oil drop experiment, which established quantization of charge, was arguably fraudulent. Millikan threw out all the data he didn't like, and then stated in his paper that he had never thrown out any data. His result was correct, but the way he went about proving it was ethically suspect.
Find free books.
Check your math. I got 83%.
From:
http://www.its.caltech.edu/~dg/crunch_art.html
"""
The crises that face science are not limited to jobs and research funds. Those are bad enough, but they are just the beginning. Under stress from those problems, other parts of the scientific enterprise have started showing signs of distress. One of the most essential is the matter of honesty and ethical behavior among scientists.
The public and the scientific community have both been shocked in recent years by an increasing number of cases of fraud committed by scientists. There is little doubt that the perpetrators in these cases felt themselves under intense pressure to compete for scarce resources, even by cheating if necessary. As the pressure increases, this kind of dishonesty is almost sure to become more common.
Other kinds of dishonesty will also become more common. For example, peer review, one of the crucial pillars of the whole edifice, is in critical danger. Peer review is used by scientific journals to decide what papers to publish, and by granting agencies such as the National Science Foundation to decide what research to support. Journals in most cases, and agencies in some cases operate by sending manuscripts or research proposals to referees who are recognized experts on the scientific issues in question, and whose identity will not be revealed to the authors of the papers or proposals. Obviously, good decisions on what research should be supported and what results should be published are crucial to the proper functioning of science.
Peer review is usually quite a good way to identify valid science. Of course, a referee will occasionally fail to appreciate a truly visionary or revolutionary idea, but by and large, peer review works pretty well so long as scientific validity is the only issue at stake. However, it is not at all suited to arbitrate an intense competition for research funds or for editorial space in prestigious journals. There are many reasons for this, not the least being the fact that the referees have an obvious conflict of interest, since they are themselves competitors for the same resources. This point seems to be another one of those relativistic anomalies, obvious to any outside observer, but invisible to those of us who are falling into the black hole. It would take impossibly high ethical standards for referees to avoid taking advantage of their privileged anonymity to advance their own interests, but as time goes on, more and more referees have their ethical standards eroded as a consequence of having themselves been victimized by unfair reviews when they were authors. Peer review is thus one among many examples of practices that were well suited to the time of exponential expansion, but will become increasingly dysfunctional in the difficult future we face.
We must find a radically different social structure to organize research and education in science after The Big Crunch. That is not meant to be an exhortation. It is meant simply to be a statement of a fact known to be true with mathematical certainty, if science is to survive at all. The new structure will come about by evolution rather than design, because, for one thing, neither I nor anyone else has the faintest idea of what it will turn out to be, and for another, even if we did know where we are going to end up, we scientists have never been very good at guiding our own destiny. Only this much is sure: the era of exponential expansion will be replaced by an era of constraint. Because it will be unplanned, the transition is likely to be messy and painful for the participants. In fact, as we have seen, it already is. Ignoring the pain for the moment, however, I would like to look ahead and speculate on some conditions that must be met if science is to have a future as well as a past.
"""
A 21st century issue: the irony of technologies of abundance in the hands of those still thinking in terms of scarcity.
If you think being a scientist sucks, try working on a factory floor.
Leaving aside your questionable assertion that being a factory worker is far worse than being a scientist, the more relevant comparison would be to jobs that require similar levels of education and competence. A compelling case can be made that the education and competence required to be a scientist is similar to that of a medical doctor, lawyer or high level engineer.
The problem with a career in science is that it is like a career in acting. Sure, there's the super stars at the top who are doing extremely well for themselves but then pretty much everyone else is struggling just to feed their families.
Of course, there are struggling actors who obviously don't have what it takes to be actors and there are struggling scientists who obviously don't have what it takes to be scientists. There are also, however, huge numbers of actors and scientists who are doing everything right and who are just as talented as the guys at the top but who somehow just didn't get their big break - and who, as a result, are struggling to feed their families.
So, what's the problem? Well, a lot of young are encouraged to embark on the long and arduous path to become scientists with the belief that they will eventually command salaries on par with careers that require similar levels of education and competence (medical doctors, lawyers, etc.). Unfortunately for them, when they final complete the dozen or so years of training to become scientists, they realize that they are overwhelmingly likely to command a salary on par with mid-level factory workers.
Eventually as knowledge about expected science salaries becomes more widespread, "the market" will probably adjust and young people who are considering careers in science will have enough information about expected salaries to choose other careers such as medicine, law, or management.
If the USA, for example, doesn't want to be a world leader in scientific research then that's totally fine. It's unfortunate for people who have already committed to a career in science but, with any luck, today's young people will choose other careers and complaints about low pay for scientists will go away because there won't be any scientists left to complain.