Scientist Says Most Scientific Papers Are Wrong

An anonymous reader writes "According to epidemiologist John Ioannidis, the majority of published scientific papers are wrong. If Ioannidis's own paper is right, a randomly chosen scientific paper has less than a 50% chance of being true. He also says that many papers may only be accurate measures of the prevailing bias among scientists. However, a senior editor of a scientific journal says that scientists are already aware of this: 'When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas. So even if something is wrong with the paper, if they have the kernel of a novel idea, that's something to think about.'"

Comments

This is FUD

[nigham]

From the article:

Traditionally a study is said to be "statistically significant" if the odds are only 1 in 20 that the result could be pure chance. But in a complicated field where there are many potential hypotheses to sift through - such as whether a particular gene influences a particular disease - it is easy to reach false conclusions using this standard. If you test 20 false hypotheses, one of them is likely to show up as true, on average.

Odds get even worse for studies that are too small, studies that find small effects (for example, a drug that works for only 10% of patients), or studies where the protocol and endpoints are poorly defined, allowing researchers to massage their conclusions after the fact.

So this guy means that every scientific study is simply putting together some numbers from experiments? A good scientific paper - which follows the scientific method, if the author has heard of it - first presents a theory, after which it conducts experiments to estimate whether or not the theory is correct. Also, a good scientific paper will talk about limitations of accuracy - statistical limitations as the author suggests have been extremely well studied and documented.

So while it may be said that statistical studies may not always be as conclusive as they're made out to be, generalising 50% of published scientific literature to be wrong is simply Troll.

Re:Lamarck and Darwin were wrong too

[Saint Stephen]

Yeah but I would think you would look at the fundamental flaws in Newtonian physics as very, very instructive about how not to put too much faith in Evolution. People used to use Newtonian physics to justify determinism, all on perfectly sound (at the time) basis.

If atoms aren't billard balls, how can you be sure we're monkeys? But, more to the point, should you be really drawing any conclusions?

The analogy of "atoms are billiard balls" -> "determinism" :: "humans are monkeys" -> "there is no plan for the universe"

Re:Lamarck and Darwin were wrong too

[Saint Stephen]

I don't know why you're marking the guy funny. If you parse his statement you'll see it amplifies my point.

"Atoms behave like billiard balls." is the statement which was once thought to be true. So atoms not being billiard balls is the suprising thing. Fish not being landmines is the "obvious" thing. The surprising thing would be if Fish were indeed landmines.

His statement should have been:

If Fish are landmines, how can you be sure we're really elk?

Parsing his statement out merely reinforces my point!

Re:Lamarck and Darwin were wrong too

[PsiPsiStar]

I'm just saying it's highly likely that any philosophy or world-view a particular science-technique appears to imply at this point in time (with Evolution, a philosophy of No God, No Plan) is HIGHLY likely to be .... Dead wrong.

Not "dead wrong." Partially wrong. The world is less than totally deterministic, but it is not totally non-deterministic. Causes have reasonably predictable effects. A deterministic universe, while inaccurate, was more useful than the totally non-deterministic universe which existed before Newton, where anything could happen.

Asimov spoke brilliantly to this line of reasoning in his essay "the relativity of wrong."
To quote him;

when people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.

http://chem.tufts.edu/AnswersInScience/Relativityo fWrong.htm

and yet

[objwiz]

everyone believes global warming is a reality