Scientist Says Most Scientific Papers Are Wrong

An anonymous reader writes "According to epidemiologist John Ioannidis, the majority of published scientific papers are wrong. If Ioannidis's own paper is right, a randomly chosen scientific paper has less than a 50% chance of being true. He also says that many papers may only be accurate measures of the prevailing bias among scientists. However, a senior editor of a scientific journal says that scientists are already aware of this: 'When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas. So even if something is wrong with the paper, if they have the kernel of a novel idea, that's something to think about.'"

Comments

Re:Blinded by Science

[blamanj]

And, as the article mentions (but doesn't go into detail on), this is why reproducibility is so important.

Let's say a scientist comes up with a new idea, does the research, and publishes a result. OK, assuming we buy the article's premise, there's a 0.5 chance s/he's made a mistake. Now another scientist and then a third duplicate the experiment and get the same result. The odds that the original proposition is in error drops to 0.25, then to 0.125. The odds are now 8:1 the result is valid.

See cold fusion for an example of the converse.

Sadly, they did publish

[edremy]

Stephen Meyer got an ID paper published in the peer-reviewed Proceedings of the Biological Society of Washington last year.

Skeptic had their take on it in the last issue. In a nutshell

• The journal is in the bottom 20% of all journals for impact, but it is a legit peer-reviewed journal with a long history
• The current head editor is a noted creationist who's on the editorial board of another journal that only publishes papers that are in agreement with a literal interpretation of Genesis.
• The editor won't say who the reviewers were, only that they were biologists at well known institutions.
• The paper's sponsoring society was not happy, and put out a press release saying that none of them would have agreed to publish if they had known.

Re:Lamarck and Darwin were wrong too

[jc42]

... Galilean relativity, which was proven wrong by Einstein and friends.

Actually, strictly speaking, Einstein (and friends) didn't prove Galileo or anyone else wrong. That had already been done by others. Thus, precise measurements of the orbit of Mercury and turned up discrepancies with Newton's and others' laws of orbital mechanics. The Michaelson-Morley experiments produced the apparently-absurd result that light moved at the same speed relative to all observers, even if those observers were moving relative to each other or the light source. Etc.

What Einstein did was develop a new theoretic approach that could explain a number of these anomalies. It was then up to the scientific community to viciously attack Einstein's theories, and attempt to prove him wrong. They've been at this for a century now, and all of their tests so far have end up with results consistent with Einstein's theories, to within the error bounds of the measurements. In scientific circles, this constitutes "proof" that Einstein's theories are either correct, or are very close to correct.

Even then, the earlier theories hadn't really been proven wrong. Rather, they were shown to be merely good approximations. After all, if your instruments can measure something to 12 places, but Einstein's and Newton's equations predict a difference in the 20th place, you can't show either set of equations to be wrong. This is why those earlier "disproved" theories are still taught in science and engineering schools. Newton's equations are a lot simpler than Einstein's, and in situations where you can't measure the difference, you might as well use the simplest equations. You just have to be careful not to apply the simpler equations in situations where they aren't good enough.

But note that Einstein himself didn't disprove those earlier theories; that had been done by the others that found the anomalies. And Einstein didn't prove his own theories; that has been done by a century of tests by the entire scientific community. He did the really hard job: He came up with his wild new theories of a universe that behaved rather differently than anyone thought. But his theories were consistent with those strange observations. And his theories included equations that could be tested against the real universe. And his theories keep passing every test that anyone comes up with.

Now if we could get some other would-be scientists to present us with versions of their theories that can be tested against the real universe ...