Slashdot Mirror
Scientist Says Most Scientific Papers Are Wrong
An anonymous reader writes "According to epidemiologist John Ioannidis, the majority of published scientific papers are wrong. If Ioannidis's own paper is right, a randomly chosen scientific paper has less than a 50% chance of being true. He also says that many papers may only be accurate measures of the prevailing bias among scientists. However, a senior editor of a scientific journal says that scientists are already aware of this: 'When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas. So even if something is wrong with the paper, if they have the kernel of a novel idea, that's something to think about.'"
"Science" is NOT the same as "fact" or "truth". It is a METHOD -- a PROCEDURE one follows in an attempt explain some event or phenomenon. It should hardly be surprising that "Scientific" papers are mostly wrong. There may be only one "right" or "correct" theory for a given phenomenon -- but there are countless wrong ones.
Wow, I thought Study Shows One Third of All Studies Are Nonsense is bad enough, who knows scientific papers are worse!
I patiently await the next article: "Research Shows Three-Quarters of All Researches Are Bullshit".
Rock that crushes, Paper & Scissors that don't matter.
What I do see as harmful is the attitude towards bad papers. To many academics try to accumulate more and more published papers the same way that slash-dotters try to build up karma. I understand that having papers published can reflect well on someone, but we need more accountability. Journals need to create a more strict system for reviewing papers that are to be published to weed out more of the crap plain and simple. If the evidence does not reflect the claims throw it away. If the research was conducted on a population that was too small or specific for a grand generalized claim about the topic as a whole, throw it out.
I understand that you will always have people just trying to throw their names around, but this needs to be looked at from the grander perspective.
"When I read the literature, I'm not reading it to find proof like a textbook.
Sure there are probably many scientists that think of it this way. But the problem is that bad research (or a bad paper) rarely dies after being published. They are often cited as evidence for years to come in other papers until enough evidence to the contrary comes out to raise questions. Plus, you have crazy professors giving this bad research for their classes to read, and often they don't explain to their classes where research is possibly flawed--so we find ourselves training generations of new scientific minds that run around spouting out bad research. I understand that we all need to take research with a grain of salt when we read it, however bad scientists trying to become famous with their bad ideas or bad papers can be very detrimental to any field.
"If we knew what we were doing, it wouldn't be research"
That's what my supervisor used to say to me when I got depressed about lack of progress.
Whether anything anyone says is right or wrong, it's a matter of opinion first and foremost.
No, it's not.
Our biology does not provide us
Our biology provides us with excellent truth detectors: throughout most of primate evolution, if you were wrong about whether your food was poisonous or whether there was a lion hiding in the bushes, you didn't get to pass on your genes. You didn't get to debate social relativism with the lion before he made a tasty meal out of you.
Most of science is still ultimately about matters like that, matters that have good answers, at least in principle.
Some science has veered off course, however. Every major scientific discipline (physics, biology, chemistry, etc.) has subareas where people start conflating experimental facts with opinion, aesthetics, and prejudice.
So, scientific truth is not a matter of opinion, but a lot of what is published in science is not about scientific truth.
Therefore anything that anyone says is simply an opinion.
That's just your opinion.
Who ordered that?
If atoms aren't billard balls, how can you be sure we're monkeys?
If fish aren't landmines, how can you be sure we're really elk?
But, more to the point, should you be really drawing any conclusions?
No. Not until the smoke clears, and I can once again tell the difference between billiard balls, atoms, monkeys, landmines, and elk. And neither should you.
Science progresses when well thought out hypothothies based on a good data are replaced by more inciteful reasoning based on more complete data. Lamarck wasn't guilty of faulty reasoning. He just didn't have a complete enough data set.
But the article at hand, isn't talking about that kind of "wrong". He is talking about conclusions that can not be supported by the data presented. Either the reasoning is faulty or the data collection methods are so faulty that no meaningful conclusions can be drawn.
When a theory is proven wrong in the scientific sense, it is a good thing. We learn something new and that be the basis for further developments. But if a theory is proven "wrong" in the mechanical sense, we have no new insights, just a relief from further time wasting.
Did anyone actually RTFP? It's one of the most spurious pieces of "research" I've ever read. And with a biophysics degree, I have read quite a few. The author actually didn't investigate any actual papers, but he builds a mathematical model out of his own biases, statistical projections, and some back of the envelope computation. Even then, his conclusions are much less stringent than the submitter makes them out to be. He "proves" that under all his assumptions, half the research papers *might* be wrong, but shows not even statistical evidence that they are.
I think PLoS is peer reviewed, but that paper should never have survived peer review. Occasionally, bad papers slip through, even in the so called hard sciences. This one seems to be one of them. Since PLoS Medicine is pretty well respected for an open access publication, lets assume that this was a lark and more on.
But it makes me curious what the fraction of bad papers looks like in an open access publication like PloS versus a traditional journal like, say, Nature, The Lancet, or New England Journal of Medicine. One reservation people have about open access (or author pays) models was that since PLoS gets paid about $1500 from they authors, they might be accepting vanity papers, or don't triage as well as traditional journals. I don't think they are, but if this paper is any indication, PLoS might take a second look at their peer review process.
After reading the summary, I was under the impression that this artical was about a high rate misleading papers published by scientists.
Based on all of the anti-creationalist comments I thought maybe I had misread the summary, so I looked at the article itself.
Not once did I see mention of the universe's creation in the summary or in the linked artical, in fact the example stated was "such as whether a particular gene influences a particular disease."
It seems to me that lately a lot of comments on slashdot have been trying to start a witch hunt for advocates of ID. Can we please knock it off and stop screaming wolf every time some thing that is related to science is mentioned on slashdot.
If science can be wrong, then why trust it?
It is the only objective process for assessing facts from fiction.
In other words, if the best a scientist can tell you today is that, he might be wrong tomorrow, why even bother listening to him?
No one is forcing you to listen. You ignore the information provided by science at your peril.
So you can use science for real things, like physics and design of military weapons and consumer goods, but the rest of it is so much speculative nonsense.
Quantum physics is speculative, but you don't seem to be throwing your computer out the window.
The consequences of guessing wrong about the origin of humanity are completely immaterial to most people's lives.
Dead wrong.
Stalin believed that Darwinian evolution was just a bouguoise concept. He believed in Lamarckian evolution and directed his agricultural ministry to ignore studies that supported Darwinain evolution. Their agricultural industry suffered and people went hungry in the process.
You can't show people evolving any more than someone else can show God making something.
I can show a progression of hominid fossils leading to homo sapien sapien. The Bible is silent about these fossils.
It's immaterial, unprovable, and so why fight over it?
It may be immaterial to you, but the theory is consistent with the evidence we possess. You may not choose to believe it, but that the only thing immaterial about this discussion.
Yeah you can roll out the eliptical argument that evolution is somehow necessary for medicine but most doctors are concerned with the human species, here and now, and now plants and people are related.
Why bother? You obviously believe that the scientific method works differently for investigations related to the origin of humanity than it does when applied to chemistry.
To wit, you can get a Chem E degree and still get into Med School.
You are correct.
Just don't whine to me when you have difficulty making sense of the data you gather without using evolutionary theory.
You will never amount to anything more than a glorified technician.
I can live with that.
"Rocky Rococo, at your cervix!"