Slashdot Mirror
Google Putting Crowd Wisdom to Work
daveperry writes "The Google Blog has a post about their use of prediction markets to forecast certain events that are relevant to their business. From the article: "Our search engine works well because it aggregates information dispersed across the web, and our internal predictive markets are based on the same principle: Googlers from across the company contribute knowledge and opinions which are aggregated into a forecast by the market. Sometimes, just feeling lucky isn't enough, and these tools can help." In related news, some software was recently open sourced that enables people to set up their own prediction markets."
Want a tip on when a stock is going to move? Monitor the number of times your users send email to one another containing the stock's symbol in the message. When the number goes up, activity is sure to follow.
But they wouldn't do that, right? Because...
Because...
Exactly.
--
You didn't know.
Predicting involves extrapolating from a current ** sample** of data to predict the future based on one own interpretation /recognition of patterns.
The better your size and quality of your sample combined with a finely tuned pattern recognition the better your forecast (I won't go into exceptional events which by definition are exceptional).
So what do we have here? A larger sample of better quality for starters.
Also do not underestimate the power of the masses. If your sample=population size this is no longer forecasting (i.e. extrapolation) but the writing on the wall! (as long as people do as they say they are going to do).
So if your sample size increases dramatically, with better quality (smart employees) things will tend to happen as per the survey!
I know I am oversimplifying but but these are the basics of neural networks (or in this case a neural network of neural networks). Look it up.
Of course, predicting is not extrapolating if it is purely a random guess.
Have fun!
Artificial intelligence is no match for natural stupidity
The classic example of "crowd wisdom" is the jellybeans-in-the-jar experiment, often used in introductory MBA classes to convince people that open markets value securities (basically) fairly. The experiment goes like this: the professor brings a jar of jellybeans and asks everyone to guess how many there are in the jar. The individual estimates may vary quite a lot, but the average of the estimates in the class is usually close, in fact often closer than the closest estimate of any of the students depending upon the size of the class. That is in a situation where the students have very little information about the jar and perhaps no experience with such estimates. If there were greater experience and/or they were allowed more information then presumably the individual and average estimates would be even closer. Basically, this can be described as the "Central Limit Theorem" in action- that the standard deviation of averages is smaller than the standard deviation of the individuals by a factor of the square root of the sample size, as illustrated in this applet or in this Mathworld description. The CLT actually says more- that as the sample size increases, the distibution of averages approaches that of the normal ("bell curve") distribution, so the distribution of avergaes is roughly normal, and then techniques designed to analyze the normal distribution can be applied with greater certainty.
It's psychosomatic. You need a lobotomy. I'll get a saw.
Our philosophy is: why bother with all the trappings of the "market". It just confuses people, and leads to all kinds of gamesmanship that has nothing to do with what you're trying to predict. We simply ask people what they think the outcome will be, and we use a mathmatically correct way of ranking their accuracy (called a "proper scoring rule" for those of you who like math).
People who are right get higher weights in our system, can win prizes like Amazon gift certificates, and gain "titles" such as "Senior Political Analyst, Level 3". Bloggers can use our data graphs free, just don't nuke our watermarks. The system has already been more accurate than experts at some predictions, although we are just starting out and need a lot more people.
I see all these dismissals, as if this is a joke. But might I suggest that the researchers at Google aren't trying to get an acceptable sample-size, as many people here are naively suggesting. It seems much more likely that Google is researching the thought-process of prediction itself. It also seems likely that a number of fairly intelligent industry-insiders also believes they're close to figured out how to create a prediction engine that is accurate (ahref=http://slashdot.org/article.pl?sid=05/09/22 /1229238&tid=109&tid=120&tid=217rel=url2html-6813h ttp://slashdot.org/article.pl?sid=05/09/22/1229238 &tid=109&tid=120&tid=217>)...
Think about it; If they have enough information (IE the web), they can hone it by feeding it past events to "predict" other past events and then comparing their prediction to what actually occurred. If it's able to predict events that have occurred given what came before those events, then there's a good chance that it will work with current data to predict future events.
Feel free to label me a wacko... Perhaps I've seen "Terminator" too many times. But I've also worked with a number of freakishly intelligent people; the kind that Google has been hiring right-and-left without any apparent reason. I've always said that technology could someday facilitate society's return to bondage... It may seem far fetched, until one considers the sheer breadth of people that hate "western" culture. Given the past as a predictor, I would be inclined to believe that few Middle Eastern leaders would hesitate to use technology to exterminate as much of western culture as they could... Folks, its not unrealistic to recognize the likelihood that technological-advancements will make sudden and drastic changes to our way of life... This ought to give us pause. What balances are essential to our culture? What imbalances will end it? Someday, somewhere, somebody is going to come up with a way to utilize technology to facilitate his/her agenda. Maybe we don't have to worry about Google, since the magnitude of the consequence is, at least in part, proportionate to the magnitude agenda of the individual(s). But things such as this should be cataloged in our brains as evidence that this mode of societal-failure is plausible.
Ok, flame away. But I hope I never have to say I told you so.