Psychology's Replication Battle

An anonymous reader sends this excerpt from Slate: Psychologists are up in arms over, of all things, the editorial process that led to the recent publication of a special issue of the journal Social Psychology. This may seem like a classic case of ivory tower navel gazing, but its impact extends far beyond academia. ... Those who oppose funding for behavioral science make a fundamental mistake: They assume that valuable science is limited to the "hard sciences." Social science can be just as valuable, but it's difficult to demonstrate that an experiment is valuable when you can't even demonstrate that it's replicable. ...Given the stakes involved and its centrality to the scientific method, it may seem perplexing that replication is the exception rather than the rule. The reasons why are varied, but most come down to the perverse incentives driving research. Scientific journals typically view "positive" findings that announce a novel relationship or support a theoretical claim as more interesting than "negative" findings that say that things are unrelated or that a theory is not supported. The more surprising the positive finding, the better, even though surprising findings are statistically less likely to be accurate."

Comments

WTF?

[Oidhche]

it's difficult to demonstrate that an experiment is valuable when you can't even demonstrate that it's replicable

Duh. That's because an experiment that is not replicable has *no* value.

Re:WTF?

[Oligonicella]

Recording supernovae

Not an experiment.

Dissecting passenger pigeons

Not an experiment.

Studying the medical complications of Thalidomide babies

You got one.

Any scientific analysis of an event which occurred once may not be directly replicable.

Actually the analysis can be replicated ad nauseam.

Re:Freud's problem too

[Anonymous Coward]

When psychologists stop producing so many studies with obvious bias, subjective terminology, subjective conclusions, and stop arbitrarily coming to conclusions based on data flawed for those reasons, maybe it could be taken seriously. Obviously, replication is needed, too.

But so many people are fooled by it. Want a study that says video games cause people to be aggressive? There's a psychology study for you, but there's also one for your opponents. And all of them are bad science.

Who writes this crap

[awol]

"Those who oppose funding for behavioral science make a fundamental mistake: They assume that valuable science is limited to the "hard sciences." Social science can be just as valuable, but it's difficult to demonstrate that an experiment is valuable when you can't even demonstrate that it's replicable."

No, those of us that oppose the funding of this crap recognise that if you cannot replicate your "study" then it is not an experiment. If what you are doing cannot be proved (one way or the other) by experiment then IT IS NOT SCIENCE. I don't really care what it gets called and some of it may even be valuable for some values of valuable however the amount of dross that is produce by social researchers that try and call themselves scientists is truly extraordinary and a plague on our world.

Re:Who writes this crap

[Intrepid imaginaut]

The above comment is precisely why these "social sciences" need to be delegitimised and rubber-roomed until they can figure out the meaning of the phrase "scientific method". Grant them no authority in deciding government policy, massively defund them in academia, get them out of the courtrooms, and generally pillory them for the witchdoctors they are.

If you have to ask why, you're part of the problem.

Re:Freud's problem too

[sjwt]

Yup, like the recent one about men not being able to 'be alone with their own thouhgs'..

That same data can also read 'Men, more willing to put up with pain' or 'Men, more curious and want to know what they may experience'