Crowd Wisdom Better At Predictions Than Top CIA Analysts

First time accepted submitter tkalfigo (1448133) writes "The Good Judgment Project is an experiment put together by three well-known psychologists and some people inside the intelligence community. What they aim to prove is that average, ordinary people in large groups and access just to Google search can predict far more accurately events of geopolitical importance than smart intelligence analysts with access to actual classified information. In fact there is a clearly identified top 1 percent of the 3000 predictors group, who have been identified as super-forecasters: people whose predictions are reportedly 30 percent better than intelligence officers."

Comments

Well yeah

[MichaelSmith]

This is why its better to have elections than let the CIA select the government. AFAIK, anyway.

Seems fishy

[ceoyoyo]

I wonder if they properly controlled for luck. Take three thousand people and get them to make predictions and some of them are going to appear unusually accurate than others even if all of them are just making completely random guesses. You'd be surprised how many people don't correctly account for that. Every paper proposing clinical diagnostic criteria I've ever read, for example.

Reminds me of the Policy Analysis Market

[khallow]

Back in 2003, there was a similar system called the Policy Analysis Market (PAM) that was close to being implemented. It got deep-sixed by some world-class idiots from Congress (see my opinion then). It's too bad that we have to go to a somewhat contrived surveying/polling system rather than use something that we know works.

For example, I think a PAM system would have given us (and I mean everyone not just US policy makers) insight into how the events of the Arab Spring revolutions would evolve even if it couldn't have predicted the original flash point.

Re:Reminds me of the Policy Analysis Market

[TapeCutter]

even if it couldn't have predicted the original flash point.

Funny you should say that, the diplomatic cable leaks showed that high level western diplomats in Syria were concerned about a civil war erupting due to the severe "fertile crescent" drought fuelling internal migration from rural areas to the cities (10% of Syria's total population simply abandoned their farms due to lack of water). The drought caused food prices to rise sharply and food riots became a regular occurrence in cities across the middle east and North Africa.

"flash point" - Have a look at why that protester set fire to himself in the public square and why it resonated so strongly across the Arab world, it wasn't because they all logged on to FB and suddenly realised their governments were tyrannical. Predicting this sort of social unrest is like predicting an earthquake in LA, you can be pretty confident that your prediction will come to pass but have no idea when.

Re:Question of scale

[mikael]

Both the British and Americans used the same government contact for their information, but they didn't tell each other who that contact was. In fact, they had different codenames for the person. When they cross-referenced each others information, they got two confirmations.