Poor Scientific Research Is Disproportionately Rewarded (economist.com)

A new study calculates a low probability that real effects are actually being detected in psychology, neuroscience and medicine research paper -- and then explains why. Slashdot reader ananyo writes: The average statistical power of papers culled from 44 reviews published between 1960 and 2011 was about 24%. The authors built an evolutionary computer model to suggest why and show that poor methods that get "results" will inevitably prosper. They also show that replication efforts cannot stop the degradation of the scientific record as long as science continues to reward the volume of a researcher's publications -- rather than their quality.
The article notes that in a 2015 sample of 100 psychological studies, only 36% of the results could actually be reproduced. Yet the researchers conclude that in the Darwin-esque hunt for funding, "top-performing laboratories will always be those who are able to cut corners." And the article's larger argument is until universities stop rewarding bad science, even subsequent attempts to invalidate those bogus results will be "incapable of correcting the situation no matter how rigorously it is pursued."

Comments

Re: management

[Anonymous Coward]

No, a lot of this work is done in academic institutions that rely on grants to fund research. The funding cycles tend to be about three years long, and there's pressure to generate lots of publications rather than do good work. Institutions also tend to skim a lot of money off the top through F&A costs, and there's a lot of corruption involved. As a result, money isn't spent well and there's not enough to go around.

Re:Change the funding cycles

[NotAPK]

Are you serious?

"Then you fund graduate students, who in my experience tend to rush their work at the end and don't produce research anywhere close to the value of what they are paid."

Grad students are paid barely above minimum wage, if that. They actually aren't expected to produce *any* research output, and anything they get out of their project is regarded as a bonus. Remember, a PhD is a *training* exercise and students are *learning* how to become scientists, no matter how "good" they may seem. This doesn't stop many grad students being exploited. You'd be hard pressed to find a smarter more "capable" (I put that in scare quotes since some grads can't even tie their shoes) group of people being treated like dirt and generally undervalued. They only tolerate it because they're clueless or they just want to tough it out and get their qualification and move on. For yourself, if you are running your research group on the output of grad students (and yes, I know many are) then you're bound to be sunk sooner or later. Remember: pay peanuts, get monkeys!!

It's a strange claim to make, since hardly anyone in science is overpaid. The discrepancies become apparent once you scale income against level of responsibility, perhaps crudely converted to dollar terms based on the equipment they are using/responsible for. It's not uncommon to find a post-doc managing $2-5 million worth of equipment while being paid $40-60 per year. In the private sector such a management policy would be viewed as fascicle at best and negligent at worst.

I do agree with you entirely on one point: the administrative overheads charged against grants are disgustingly inflated by parasitic policies.

Also happens in CS research

[gweihir]

I have seen quite a bit of it and know of several CS PhDs that are based on bogus results. The tragedy is that people doing their research properly will take significantly longer and have much diminished chances at an academic career. And this effect propagates: First PhD students advance on bogus results, then they become professors on fraud and finally the whole research field is broken.