Slashdot Mirror
Study: More Than Half of Psychological Results Can't Be Reproduced
Bruce66423 writes: A new study trying to replicate results reported in allegedly high quality journals failed to do so in over 50% of cases. Those of us from a hard science background always had our doubts about this sort of stuff — it's interesting to see it demonstrated — or rather, as the man says: 'Psychology has nothing to be proud of when it comes to replication,' Charles Gallistel, president of the Association for Psychological Science. Back in June a crowd-sourced effort to replicate 100 psychology studies had a 39% success rate.
Real science is reproducible, that's the crucial part. That which can't be reproduced is flawed, discarded and ends up being treated like cold-fusion rubbish.
Psychology has always been bunk from a scientific point of view. It changes it's tune to whatever happens to be in vogue that decade. It's nothing like a real science as we define it, but has its own values elsewhere in life.
You know nothing of Psychological research.
It sounds like you know very little about cancer or cancer research.
So is medical science not "real science" because we've had quite a few stories over the last few years that a ton of results from medical research and drug trials can't be reproduced.
A large percentage of medical studies are funded by manufacturers, and it's fairly well understood that most of those don't get published unless they produce the "right" results. And those that are published are often really "preliminary", based on too little data to be considered reliable. But if a test on 10 or 20 patients gives the "right" results, there is a lot of marketing pressure to get the paper published right away.
This easily explains the growing problem of medical products that are found to be worthless (or even harmful) to the patients, after years of heavy marketing has produced large profits.
There's also the age-old problem that studies with "negative" results usually don't get published at all. As usual, there's a good xkdc comic that explains the methodology in a way that even the minimally numerate reader can understand.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
That sounds like a pretty weak test, but not a bad one. To my mind, this crosses one of my areas of expertise since, I have had his job as a professional sysadmin. I worked in a shop where, for better or worst, we decided that all free software we used on Solaris would be compiled from source.
This quickly became a huge mess as updates would sometimes bring changes and there was always the question "who built it last time and what options did they choose", so quickly we found a need to fix that, and I started scripting. (its where my competence with shell really began)
Once you have even solved the easy part, then you have to think about versions and dependencies.
In fact, later on we were getting involved in research computing, that wasn't my project but one of the topics that came up was... researchers will build this software, just like we are talking about, and use the data.,...now someone wants to audit it down the road....
What happens if the libraries have changed and the old code doesn't compile? What if there is an error in a calculation that was introduced by a particular library version being used?
The reality is, you write the code, but it gets run in an environment. That entire environment has the potential to have an effect, a full specification needs to capture at least some of that as well.
"I opened my eyes, and everything went dark again"
Exactly. This applies to "hard" sciences too. The jackass submitter couldn't get his head out of his ass long enough to read the article and understand the implication. One of the problems the article mentioned was journals not willing to publish null results and research that just replicates other research. That's rather important because you need more than one researching publishing similar findings to feel confident about the results. The other point is that some the results, though valid, were valid for very narrowly defined scenarios and therefore not generalization.
Yo Bruce66423, RTFA sometime.
There has been no great art or literature or music in decades.
Oh piss off, hipster.