Scientist Says Most Scientific Papers Are Wrong

An anonymous reader writes "According to epidemiologist John Ioannidis, the majority of published scientific papers are wrong. If Ioannidis's own paper is right, a randomly chosen scientific paper has less than a 50% chance of being true. He also says that many papers may only be accurate measures of the prevailing bias among scientists. However, a senior editor of a scientific journal says that scientists are already aware of this: 'When I read the literature, I'm not reading it to find proof like a textbook. I'm reading to get ideas. So even if something is wrong with the paper, if they have the kernel of a novel idea, that's something to think about.'"

Comments

Lamarck and Darwin were wrong too

[geomon]

Wow! Science can be wrong.

That is how the system works.

But just because these two scientists were wrong about the precise mechanics of evolution doesn't mean that they were wrong about how the data should be interpreted. The data shows that life has progressed to meet the demands of its environment. Survival of the fittest is correct, but there is no straight-line progression of lifeforms leading one from another as was supposed when these authors first penned their ideas.

Scientific ideas may come and go, but the data set just gets larger. That is why this guy can claim the others are wrong: he has a better data set.

Re:Lamarck and Darwin were wrong too

[vanka]

If you received a PhD in 2003 and were taught that natural selection is the driving force of evolution, then I am sorry that you wasted your money.

Darwin's original theory of evolution was based on two very obvious things; variations among organisms and natural selection. Natural selection was not something that Darwin "discovered", the idea had been around for a bit and is quite intuitive - those that are better adapted to their environment will survive; rather he proposed a mechanism through which beneficial changes in a population are preserved in the offspring. But he did not do a good job of how these changes are brought about. We all know that the offspring of an organism will not be identical to the parent (unless it is a clone); so that children are taller, stronger, lighter/darker (you get the idea) than their parents. What Darwin proposed was that those variations were unfavorable to an organism were weeded out by natural selection (i.e. the organism was killed by predators, could not feed, etc); while the advantageous were preserved. Darwin assumed that the variation that he saw in a species (or between closely related species) would continue indefinitely; that if an organism that can run 40 mph has offspring that can run 1 mph faster and the offspring's offspring continue the trend, this will continue until in several generations these organisms will be able to reach speeds of 100 mph.

This is where Darwin went wrong, although we must not be too hard on him because he had no knowledge of genetics. The genetic code of an organism puts an upper limit on the variation that is possible. Mendel demonstrated this in his famous experiments with peas; which incidentally took place a little after Darwin wrote "Origin". He crossed peas with red and white flowers and was able to get red, white, and pink flowers; but he never got orange, blue, or yellow flowers. Mendel could only select for those traits that were encoded in the genes. So if the population as a whole did not already have the genes for 100 mph speed (recessive or otherwise) there is no way that natural selection can select those genes. A good rule of thumb to remember is that natural selection cannot select what does not exist. So natural selection cannot drive evolution by itself as it cannot produce new traits, and the traits of an organism are determined by its' genes. So there needs to be a mechanism or process that can create new genes; and this is where mutations come in. Mutations by definition are copying mistakes that change the genes. So if there are new genes, natural selection can do its' job of weeding out the bad traits and leaving the good ones. This is known as Neo-Darwinism as the primary idea of primitive organisms evolving into complex ones remains but the process driving it is not natural selection but mutations. In Neo-Darwinism, natural selection is seen as just an obvious footnote.

Re:Lamarck and Darwin were wrong too

[instarx]

If you received a PhD in 2003 and were taught that natural selection is the driving force of evolution, then I am sorry that you wasted your money.

Oops, how embarrassing for you, because a REALLY top-flight education would have taught you to be more precise. Natural selection IS the driving force that propogates desirable characteristics (and therefore evolution). You forgot that natural selection operates on mutations just as it does on intra-species genetic variation. Perhaps you meant "Darwinism" instead of natural selection, but even then some clear thinking would show that Darwin's theories are very compatable with evolution via mutational variation even though he did not recognize it. After all, where do you think all those intra-species variations came from except mutation, interbreeding and natural selection?

Imagine a species that has only genetic mutation and no intra-species variability (a population of clones, perhaps). Any genetic mutation will be either good for offspring or bad. Natural selection "decides" which is which. Without natural selection mutations are, by definition, neither good nor bad, nor can they result in any evolutionary advancement. If there is no disadvantage to change then neither is there any advantage. Without natural selection random mutational variation would eventually populate the universe with a near-infinite number of completely different individuals.

Far from being relegated to the scientific scrap heap, natural selection is still the prime mover (in fact the only mover) in evolution.

Re:Reach

[cahiha]

Whether anything anyone says is right or wrong, it's a matter of opinion first and foremost.

No, it's not.

Our biology does not provide us

Our biology provides us with excellent truth detectors: throughout most of primate evolution, if you were wrong about whether your food was poisonous or whether there was a lion hiding in the bushes, you didn't get to pass on your genes. You didn't get to debate social relativism with the lion before he made a tasty meal out of you.

Most of science is still ultimately about matters like that, matters that have good answers, at least in principle.

Some science has veered off course, however. Every major scientific discipline (physics, biology, chemistry, etc.) has subareas where people start conflating experimental facts with opinion, aesthetics, and prejudice.

So, scientific truth is not a matter of opinion, but a lot of what is published in science is not about scientific truth.

Re:groan

[david.given]

Unless someone in the ID camp is willing to admit that God is falsifiable, their theory will not be considered science.

God's irrelevant to whether ID is true or not --- it's whether ID is falsifiable or not that's important.

Which, AFAICT, it isn't, so it's still not science. But let's at least be precise when slagging them off...

Re:Bad research==dangerous.

[BenjyD]

It's not just psychology that has so much publication list padding. During my PhD I was asked to do a detailed review of about ten papers in my area (process scheduling algorithms). They were all peer-reviewed and published in reputable journals by well-known researchers. IIRC:

- About half had very little novel content. Maybe one equation changed, a few different examples added
- Two or three had basic mathematical errors
- About half omitted details that were required to easily replicate their results or actually use their methods. I spent weeks piecing together what the authors meant from various clues scattered across appendices, tables and figures.
- Several had gaping holes in the method that were apparent to me, a first year PhD with no experience.
- All of them cherry-picked examples to show their methods in the best light, completely omitting any bad results.

Re:Peer Review

[LurkerXXX]

The article is about "Published Research Findings". It doesn't specify that all the papers analyzed were from peer reviewed journals. There are a lot of non peer reviewed journals out there. Usually you only publish in those if you have a short paper, or one that's not extremely novel, or just not of great general interest. Many times researchers will publish in those journals when they can't get the paper published in a peer reviewed journal. I'm sure the percentage of false findings in those journals is much higher, and may have altered the ratio of found false papers significantly.