Slashdot Mirror
Poor Scientific Research Is Disproportionately Rewarded (economist.com)
A new study calculates a low probability that real effects are actually being detected in psychology, neuroscience and medicine research paper -- and then explains why.
Slashdot reader ananyo writes:
The average statistical power of papers culled from 44 reviews published between 1960 and 2011 was about 24%. The authors built an evolutionary computer model to suggest why and show that poor methods that get "results" will inevitably prosper. They also show that replication efforts cannot stop the degradation of the scientific record as long as science continues to reward the volume of a researcher's publications -- rather than their quality.
The article notes that in a 2015 sample of 100 psychological studies, only 36% of the results could actually be reproduced. Yet the researchers conclude that in the Darwin-esque hunt for funding, "top-performing laboratories will always be those who are able to cut corners." And the article's larger argument is until universities stop rewarding bad science, even subsequent attempts to invalidate those bogus results will be "incapable of correcting the situation no matter how rigorously it is pursued."
The article notes that in a 2015 sample of 100 psychological studies, only 36% of the results could actually be reproduced. Yet the researchers conclude that in the Darwin-esque hunt for funding, "top-performing laboratories will always be those who are able to cut corners." And the article's larger argument is until universities stop rewarding bad science, even subsequent attempts to invalidate those bogus results will be "incapable of correcting the situation no matter how rigorously it is pursued."
No, a lot of this work is done in academic institutions that rely on grants to fund research. The funding cycles tend to be about three years long, and there's pressure to generate lots of publications rather than do good work. Institutions also tend to skim a lot of money off the top through F&A costs, and there's a lot of corruption involved. As a result, money isn't spent well and there's not enough to go around.
Are you serious?
"Then you fund graduate students, who in my experience tend to rush their work at the end and don't produce research anywhere close to the value of what they are paid."
Grad students are paid barely above minimum wage, if that. They actually aren't expected to produce *any* research output, and anything they get out of their project is regarded as a bonus. Remember, a PhD is a *training* exercise and students are *learning* how to become scientists, no matter how "good" they may seem. This doesn't stop many grad students being exploited. You'd be hard pressed to find a smarter more "capable" (I put that in scare quotes since some grads can't even tie their shoes) group of people being treated like dirt and generally undervalued. They only tolerate it because they're clueless or they just want to tough it out and get their qualification and move on. For yourself, if you are running your research group on the output of grad students (and yes, I know many are) then you're bound to be sunk sooner or later. Remember: pay peanuts, get monkeys!!
It's a strange claim to make, since hardly anyone in science is overpaid. The discrepancies become apparent once you scale income against level of responsibility, perhaps crudely converted to dollar terms based on the equipment they are using/responsible for. It's not uncommon to find a post-doc managing $2-5 million worth of equipment while being paid $40-60 per year. In the private sector such a management policy would be viewed as fascicle at best and negligent at worst.
I do agree with you entirely on one point: the administrative overheads charged against grants are disgustingly inflated by parasitic policies.
The "massive consensus" has been going down every year, more and more scientists are pulling out of the consensus. You will rarly hear about that because politicians and news organizations make a lot of money in making people think it is real.
Citation please?
All of the climate change data sets are made by computer models which always get out the results desired, and the desired result is confirming climate change, because if it does not, their funding is cut. So politicians, news organizations AND scientists benefit from lying, the ones that disprove it are shouted down. And the results? Billions of tax payer money (all of it that our children will have to pay) get sent over to other countries.
You have it backwards. Models are constructed from data, not the other way around. To paraphrase plasma physicist Kenneth Birdsall, the purpose of models is to generate insight, not data.
36%? Yea, there is a reason why I don't believe in any science study unless it makes sense.
Strawman, and a sloppy one at that. The 36% in TFS refers to reproducibility of psychological studies, not climate studies.
If it weren't for deadlines, nothing would be late.
If your paper confirms that GMOs are as safe as mother's milk, you are also more likely to get funding. Also, if your study shows that vaccines are safe, you are more likely to get funding.
Are those examples of confirmation bias too?
You are welcome on my lawn.
I have seen quite a bit of it and know of several CS PhDs that are based on bogus results. The tragedy is that people doing their research properly will take significantly longer and have much diminished chances at an academic career. And this effect propagates: First PhD students advance on bogus results, then they become professors on fraud and finally the whole research field is broken.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
You're not quite on the mark there. PhD students are indeed learning how to become research scientists, and the way they practice and prove they have learned is by doing original research. A thesis has to have original research in it or it's not a thesis. In almost all cases that is published somewhere peer reviewed as well.
SJW n. One who posts facts.
There are also "taught" graduate degrees opposed to research degrees.
Yes, but you're talking about graduate degrees, not PhDs. A PhD is a research course. Some have a taught component, sometimes even a whole year, but that's to bring the student up to speed, and so is simply pass/fail with no further effect after a pass. I've examined/viva'd a couple of PhDs, and original research was a major part of the criteria for examination.
SJW n. One who posts facts.