Slashdot Mirror
Journal Article On Precognition Sparks Outrage
thomst writes "The New York Times has an article (cookies and free subscription required) about the protests generated by The Journal of Personality and Social Psychology's decision to accept for publication later this year an article (PDF format) on precognition (the Times erroneously calls it ESP). Complaints center around the peer reviewers, none of whom is an expert in statistical analysis."
Ouch. Taken down by two Bayesian tests on whether it's more likely that the paper is true or not. They didn't even need to get out of bed or dig out a big maths book to basically disprove the entire premise of the original paper using its own data.
As they hint at in that rebuttal - As a mathematician and someone of a scientific mind, I would just like to see *ONE* good test that conclusively shuts people up. Trouble is, no good test will report a false result and thus you'll never get the psychic / UFO / religion factions to even participate, let alone agree on the method of testing because they would have to accept its findings.
Never dabble in statistics - the experts will roundly berate you and correct you even if you *THINK* you're doing everything right. When PhD's can't even work out things like the Monty Hall Problem properly, you just know it's not something you can throw an amateur towards.
In one sense, it is a historically familiar pattern. For more than a century, researchers have conducted hundreds of tests to detect ESP, telekinesis and other such things, and when such studies have surfaced, skeptics have been quick to shoot holes in them.
I always thought the hole-shooting was an essential step in the scientific process. If you can't patch up the holes, you aren't really doing "science."
In science, you tell a story. Then everyone says, "no, that's wrong because..." Then you say, "I'm afraid I'm right, because..." It is an imperfect process because people have biases that are hard to overcome. But, if the empirical evidence is strong enough, you will overcome these imperfections. In the end, you build a consensus by presenting enough evidence that no one can argue with. I'm not sure what is more democratic than that.
I don't understand this. If the researchers did a proper experiment, respected the rules, followed proper procedures, and did a proper analysis of the data they collected, in a scientific way. Why is it a problem to publish ?
Shouldn't be any. However, this case isn't relevant to your question, since they didn't do a proper analysis. (Or at least that's what the rebuttals say; I haven't read the paper.)
One of the most glaring problems is that they (reportedly) went fishing for statistical significance in their results, without making the correction that is required for rigor when you do that. When you find significance at the traditonal 95% confidence level, there's a 5% chance that you're finding meaning in noise. If you test for 20 different effects and then go fishing to see whether *any* of them show significance at the 95% confidence level, you have 1 - .95^20 = 64% chance of finding "significance" in noise.
There are simple ways to fix that problem, e.g. the Tukey HSD "honestly statistically different" test. Apparently the authors were either ignorant or dishonest, and the reviewers were either ignorant, dishonest, are careless.
Sheesh, evil *and* a jerk. -- Jade
Wait, you've lost me here... what about human beings can't be explained with classical physics?
2 girls 1 cup