Slashdot Mirror
Results Are In From Psychology's Largest Reproducibility Test: 39/100 Reproduced
An anonymous reader writes: A crowd-sourced effort to replicate 100 psychology studies has successfully reproduced findings from 39 of them. Some psychologists say this shows the field has a replicability problem. Others say the results are "not bad at all". The results are nuanced: 24 non-replications had findings at least "moderately similar" to the original paper but which didn't quite reach statistical significance. From the article: "The results should convince everyone that psychology has a replicability problem, says Hal Pashler, a cognitive psychologist at the University of California, San Diego, and an author of one of the papers whose findings were successfully repeated. 'A lot of working scientists assume that if it’s published, it’s right,' he says. 'This makes it hard to dismiss that there are still a lot of false positives in the literature.'”
More to the point, this is a problem of funding in all fields.
No one wants to pay for basic research, even if it yields other useful ideas for further research. Unless it hyped to high heavens, the possibility of getting dollars is nil.
Gent I know was a decent researcher who got demoted to teaching community college. After a year of not being able to produce the "right" (read: able to secure further funding), he was canned, and another researcher who was more accommodating to fudging results got the position.
It's not like the experiments were going to be reproduced anyway. Just fodder for additional grants because you produce "results".
Just taking this quick opportunity to post a link to my favorite journal, the Journal of Articles in Support of the Null Hypothesis: http://www.jasnh.com/ .
JASNH is one of the few places where you can submit a paper that says "we tested for X effect on Y and found no evidence that X affects Y". Generally this research is unpublishable and people will tweak parameters to get something career-advancing out of their research; I like JASNH because of the reminder that "falsifiability" can really happen.
We recently had heard in the office over one of the Yellow Machine that's made by Anthology Solutions.
You missed the GP's point: the problem is not that true negatives were found; the problem is that they were not published. Because they were not published, future researchers might waste more effort re-discovering them.
"[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz