Can Bad Scientific Practice Be Fixed?

HughPickens.com writes: Richard Horton writes that a recent symposium on the reproducibility and reliability of biomedical research discussed one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with science (PDF), one of our greatest human creations. The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. According to Horton, editor-in-chief of The Lancet, a United Kingdom-based medical journal, the apparent endemicity of bad research behavior is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world or retrofit hypotheses to fit their data.

Can bad scientific practices be fixed? Part of the problem is that no-one is incentivized to be right. Instead, scientists are incentivized to be productive and innovative. Tony Weidberg says that the particle physics community now invests great effort into intensive checking and rechecking of data prior to publication following several high-profile errors. By filtering results through independent working groups, physicists are encouraged to criticize. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. "The good news is that science is beginning to take some of its worst failings very seriously," says Horton. "The bad news is that nobody is ready to take the first step to clean up the system."

Comments

Can bad journalism be fixed?

[khchung]

The case against journalism is straightforward: much of the news articles, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, journalists has taken a turn towards darkness. The apparent endemicity of bad journalist behavior is alarming. In their quest for telling a compelling story, journalists too often sculpt facts to fit their preferred narrative of the world or retrofit hypotheses to fit their data.

Unlike journalists, however, science will always have to bow to reality. So, yeah, bad science practice will eventually run aground when reality hits, no matter how many epicycles one add to the model. But bad journalism will persists as long as it attracts eyeballs.

Science is fine... Academic institutions are not

[jellomizer]

"Publish or Perish", Degrees that require new original ideas, Strict hierarchy structure...
Academic institutions are culturally stuck in victorian times. So if you want to work up, get the choice projects and research, you need to publish. The more your publish, the higher the chances you will move up. Because there is so much published material, people don't read it much, so they found that they can get credit for half ass work.
Your name becomes your brand, so when you try to get a grant your name+institution you will work for will get you the grant money.
There isn't any reason why Say State University of New York Buffalo can't get a grant to study seismology, but chances are it will go to University of California Berkeley not because they will do a better job, but because of the name.
Finally institutions haven't learned how to deal with today's political climate with the attempt for breaking news. Every Hypothesis is sold to the public as a new Theory... Then if that Hypothesis is shown false (as it is common in science) then the media who may have a political slant will go and say see Science is Wrong again, just like our political stance has predicted!

Science for the most part is quite work, collaborating with like minded people, with checks and balances to try to filter out strong egos. But it has gone commercial so these checks and balances are weaken as strong egos will win out.