Princeton ESP Lab to Close

Nico M writes " The New York Times reports on the imminent closure of one of the most controversial research units at an ivy league School. The Princeton Engineering Anomalies Research laboratory is due to close, but not because of pressure from the outside. Lab founder Robert G. Jahn has declared, in the article, that they've essentially collected all the data they're going to. The laboratory has conducted studies on extrasensory perception and telekinesis from its cramped quarters in the basement of the university's engineering building since 1979. Its equipment is aging, its finances dwindling. Jahn points the finger at detractors as well: 'If people don't believe us after all the results we've produced, then they never will.'"

Comments

Re:Ahem

[BiggerIsBetter]

http://www.princeton.edu/~pear/publications.html

The problems with PEAR

[FreelanceWizard]

The methodology wasn't flawed, so much as the analysis and the conclusions drawn from it.

A PEAR experiment involved a participant attempting to influence a random number generator (essentially) in a pre-specified direction over a large number of trials. Because random events are, by nature, random, you can get streaks that are above or below the mean. If you analyze a large enough sample, these streaks can become statistically significant, even though they're essentially meaningless and practically insignificant -- it's similar to the fact that any deviation from the mean, no matter how small, is statistically significant if you measure the entire population. Additionally, while the probability of any particular streak is low (.5^n is the probability of any number of heads flipped in a row, which gets very small when you talk about enough of them), if you have enough random events, those streaks are pretty much guaranteed to appear.

So, that's the logic of the PEAR data analysis. Collect a huge corpus of random events, look for streaks, then call them statistically significant because of their low base probability of appearance and the fact that they deviated at all from the expected mean. Skeptic magazine has a good discussion of the PEAR lab inanity, and I believe James Randi's commentary addresses it a few times.

The claim that PEAR's research wouldn't be reviewed is probably false, by the way. It's most likely that the papers were rejected from mainstream journals for the very reasons I mentioned earlier, or because the PEAR lab had no theoretical explanation for the "results" they observed. Or, of course, it's because their papers seem rather dubious in their lack of data and explanations of how they've arrived at their stated probability values (which I say from having the experience of reading one in a, how shall we say, less than top tier journal). Additionally, the lab's been extremely difficult with regards to their raw data. Randi, for example, has never been able to get ahold of it.

Re:The problems with PEAR

[ponos]

The whole point of statistics is that some "streaks" are very improbable if they are coming from a really random source. In that sense, if a random number generator displays such a tendency, it is rather probable that it isn't really random. So, yes, the statistical power (ability to discriminate between small differences) increases with huge sample sizes, but a really random source should fail such tests with probability p=0.95 regardless of sample size. That is because the tests ALWAYS compare the sample with one coming from a truly (theoretically) random source. This is the way those things work.

I would also like to remind (not to you, personally) the difference between statistically significant and meaningful. Even if an absurdly small difference can be inferred with certainty, it remains to be seen whether it matters in actual practice. This is a common cause of confusion, especially when medical epidemiological studies demonstrate a .001% reduction in risk for heart attack in those who eat cucumber every day. The .001% may be true, but it doesn't really matter.

P.

no. No. and NO !

[aepervius]

They simply retrofit the data after the fact. And once you retrofit data you can find ANY EVENT which match as long as your criteria is low enough. There is always some bad stuff going around. Especially that they aren't limited by event size, number of people, or geography !! This is again pseudo science at its best. You want to sway us ? Fine ! Set a level of population impacted, a geography limit, event size, then make bloody prediction. Else what you are doing is no better than taking a random bunch of data and finding correaltion between that data and other event. I bet with the same methodology I could take the price variation of potatoe per tons, take only the cent (fractional aprt) and find a corelation with major earth event. As long as I define event as above I am pretty sure any kind of shit can be retrofitted.

Extraordinary evidence is needed

[mangu]

I have little reason to doubt their methodology

Well, if you check one of their papers, you'll find the following sentence, on page 7: "While no statistically significant departures of the variance, skew, kurtosis, or higher moments from the appropriate chance values appear in the overall data, regular patterns of certain finer scale features can be discerned." That's an outright confession of fraud. They are saying they cannot find any evidence if they analyze a statistically significant amount of data, so they pick whatever small sample will suit them. It's as if I threw a coin a million times and said: "Oh look! Here I threw ten heads in sequence!"

Further on, in the next page, they state "Given the correlation of operator intentions with the anomalous mean shifts, it is reasonable to search the data for operator-specific features that might establish some pattern of individual operator contributions to the overall results. Unfortunately, quantitative statistical assessment of these is complicated by the unavoidably wide disparity among the operator database sizes, and by the small signal-to-noise ratio of the raw data, ...", which means they didn't follow a consistent testing protocol and didn't have a standardized method for training their operators. Basically, they are admitting that any statistical correlation in their data is extremely small (which is what "small signal-to-noise ratio of the raw data" means) and they have no way to check if any positive results aren't attributable to insufficient training of their operators.

Of course, if they *did* communicate their results by telepathy, then that would be an extraordinary proof. But what they have published is rather underwhelming, can we assume that if they did have any better results they would have published them?