Slashdot Mirror
Google Putting Crowd Wisdom to Work
daveperry writes "The Google Blog has a post about their use of prediction markets to forecast certain events that are relevant to their business. From the article: "Our search engine works well because it aggregates information dispersed across the web, and our internal predictive markets are based on the same principle: Googlers from across the company contribute knowledge and opinions which are aggregated into a forecast by the market. Sometimes, just feeling lucky isn't enough, and these tools can help." In related news, some software was recently open sourced that enables people to set up their own prediction markets."
Want a tip on when a stock is going to move? Monitor the number of times your users send email to one another containing the stock's symbol in the message. When the number goes up, activity is sure to follow.
But they wouldn't do that, right? Because...
Because...
Exactly.
--
You didn't know.
Reminds me of E.T. Jaynes's "Honest Weatherman" example in his book, Probability Theory: The Logic of Science (section 13.5). By making a weatherman's salary proportional to a certain function (logarithm of the probability he predicts, or entropy, or something, I forget), the weatherman has an incentive to make the best possible predictions: his salary will be directly proportional to the quality of his predictions, being maximized when his rain forecast probabilities match the actual probabilities of rain. You could set up a "free market betting system" this way, rewarding the quality of people's guesses regarding the likelihood of various outcomes.
Predicting involves extrapolating from a current ** sample** of data to predict the future based on one own interpretation /recognition of patterns.
The better your size and quality of your sample combined with a finely tuned pattern recognition the better your forecast (I won't go into exceptional events which by definition are exceptional).
So what do we have here? A larger sample of better quality for starters.
Also do not underestimate the power of the masses. If your sample=population size this is no longer forecasting (i.e. extrapolation) but the writing on the wall! (as long as people do as they say they are going to do).
So if your sample size increases dramatically, with better quality (smart employees) things will tend to happen as per the survey!
I know I am oversimplifying but but these are the basics of neural networks (or in this case a neural network of neural networks). Look it up.
Of course, predicting is not extrapolating if it is purely a random guess.
Have fun!
Artificial intelligence is no match for natural stupidity
On the Brunner novel "The Shockwave rider" is described the "Delphi pool" and that was the first thing that came to my mind after reading TFA...
1
http://www.technovelgy.com/ct/content.asp?Bnum=66
The use of intelligence to develop accurate results in a predicative system
You don't even need intelligence, in many cases. Just the "wisdom of the crowd". Read this for more info. A quote:
In an early example, Surowiecki refers to a study conducted by the British scientist Francis Galton. Galton was a believer in the power of the elite, noting "the stupidity and wrong-headedness of many men and women being so great as to be scarcely credible." But at a fair, he noticed a wagering competition in which people bet on the weight of an ox. Eight hundred people participated; some were butchers and farmers, others just idle guessers.
When Galton averaged the estimates, he expected the result to be way off. Instead, the crowd had come within one pound of the ox's weight.
More
Yeah, we know. Just look at the vast majority of people that you have to interact with on a daily basis.
No, I'm not trolling. I'm being serious. Take a good look at those around you and you'll see the truth behind Galtons comments.
The difference between why a crowd can give you better results than asking a single person is because each of us has our biases and preconceptions. However, put us in a situation where others can give input and the groupthink mentality starts to take over.
We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
The classic example of "crowd wisdom" is the jellybeans-in-the-jar experiment, often used in introductory MBA classes to convince people that open markets value securities (basically) fairly. The experiment goes like this: the professor brings a jar of jellybeans and asks everyone to guess how many there are in the jar. The individual estimates may vary quite a lot, but the average of the estimates in the class is usually close, in fact often closer than the closest estimate of any of the students depending upon the size of the class. That is in a situation where the students have very little information about the jar and perhaps no experience with such estimates. If there were greater experience and/or they were allowed more information then presumably the individual and average estimates would be even closer. Basically, this can be described as the "Central Limit Theorem" in action- that the standard deviation of averages is smaller than the standard deviation of the individuals by a factor of the square root of the sample size, as illustrated in this applet or in this Mathworld description. The CLT actually says more- that as the sample size increases, the distibution of averages approaches that of the normal ("bell curve") distribution, so the distribution of avergaes is roughly normal, and then techniques designed to analyze the normal distribution can be applied with greater certainty.
It's psychosomatic. You need a lobotomy. I'll get a saw.
Its "well documented" that financial markets are not normal. They are "fat-tailed"--extreme events happen with regularity. This attempt has been tried thousands of times by people with far more at stake than google. They also had more money, more PhDs, etc. And they failed. They always fail at some time or another. There is no algorithm for the financial markets.
Those who doubt had best read the story of Long Term Capital Management. wikipedia link
What Google is doing is interesting, but it's no magic bullet. And I'm not even sure I'd use this system to place money on the markets. Fads come, fads go. Neural Networks. Genetic Algorithms. Fibonacci numbers. Eliiot Waves. what-have you. They're all nonsense. The big ol boys still make their money the old fashioned way: graft, theft, and deceipt.
Anyone honestly think Google knows something that the trillion dollar financial industry doesn't? They have far more at stake that Google. They have their fair share of MIT PhDs, economic nobel prize winners, sparc computers, access to nearly every price in the world at any time. And they still make the majority of their cash through commission.
Call me a cynic, but the math says this can't work. And I'll trust the math before I trust a company that doesn't even pay dividends any day.
Our philosophy is: why bother with all the trappings of the "market". It just confuses people, and leads to all kinds of gamesmanship that has nothing to do with what you're trying to predict. We simply ask people what they think the outcome will be, and we use a mathmatically correct way of ranking their accuracy (called a "proper scoring rule" for those of you who like math).
People who are right get higher weights in our system, can win prizes like Amazon gift certificates, and gain "titles" such as "Senior Political Analyst, Level 3". Bloggers can use our data graphs free, just don't nuke our watermarks. The system has already been more accurate than experts at some predictions, although we are just starting out and need a lot more people.
I see all these dismissals, as if this is a joke. But might I suggest that the researchers at Google aren't trying to get an acceptable sample-size, as many people here are naively suggesting. It seems much more likely that Google is researching the thought-process of prediction itself. It also seems likely that a number of fairly intelligent industry-insiders also believes they're close to figured out how to create a prediction engine that is accurate (ahref=http://slashdot.org/article.pl?sid=05/09/22 /1229238&tid=109&tid=120&tid=217rel=url2html-6813h ttp://slashdot.org/article.pl?sid=05/09/22/1229238 &tid=109&tid=120&tid=217>)...
Think about it; If they have enough information (IE the web), they can hone it by feeding it past events to "predict" other past events and then comparing their prediction to what actually occurred. If it's able to predict events that have occurred given what came before those events, then there's a good chance that it will work with current data to predict future events.
Feel free to label me a wacko... Perhaps I've seen "Terminator" too many times. But I've also worked with a number of freakishly intelligent people; the kind that Google has been hiring right-and-left without any apparent reason. I've always said that technology could someday facilitate society's return to bondage... It may seem far fetched, until one considers the sheer breadth of people that hate "western" culture. Given the past as a predictor, I would be inclined to believe that few Middle Eastern leaders would hesitate to use technology to exterminate as much of western culture as they could... Folks, its not unrealistic to recognize the likelihood that technological-advancements will make sudden and drastic changes to our way of life... This ought to give us pause. What balances are essential to our culture? What imbalances will end it? Someday, somewhere, somebody is going to come up with a way to utilize technology to facilitate his/her agenda. Maybe we don't have to worry about Google, since the magnitude of the consequence is, at least in part, proportionate to the magnitude agenda of the individual(s). But things such as this should be cataloged in our brains as evidence that this mode of societal-failure is plausible.
Ok, flame away. But I hope I never have to say I told you so.
Thats not really what the main part of the book is about.
Its about distributed decision making. Its not a book about making money from the stockmarket, its about optimal decision making by the market. Indeed, the whole point is that markets are "smarter" than any individual who participates in them. Its actually an idea in opposition to the view that individuals can stand to "beat" the market by rejecting conventional wisdom. But furthermore, the fact the market makes "wise" decisions doesn't necessarily equate with anyone getting rich or otherwise getting what they want either.
I don't think you've actually read the book.
Plays violent online games as: Nerfherder76
This actually has nothing to do with the CLT. The CLT is a statement about the distribution of a sample average in relation to the population average.
OTOH, TFA is about information markets, which concerns comparisions between the average of people's forecasts of events, and the "true" (or natural) frequency of events. We are not comparing a sample average to a population average. (In other words, to apply the CLT here, we'd make a statement comparing the average forecast of 100 people to the average forecast that would be made by all the people in the population. We're absolutely not doing that.)
Instead, the concept to discuss here is what is sometimes called calibration. This is a concept used to measure the accuracy of forecasters (such as meterologists). It works like this: say we look at all the days a weatherman said "chance of rain is 30%". Then, we'd hope to see rain on around 30% of those days in order for that kind of prediction to be accurate. Calibration (roughly) asks: How accurate were all such statements that were made by the weatherman? (I.e., check "chance of rain 0%", 10%, 20%, etc... looking at each subsample of his forecasts.)
Here (pdf) is a randomly chosen article on this kind of calibration.
For those that read TFA, did you see the graph? Calibration asks for those two lines to overlap. Whether they do or not has nothing to do with the Central Limit Theorem.
the crowds don't know shit when it comes to financial markets.
Most of the time, they do. "The Madness of Crowds" is about a relatively rare phenomenon, the speculative bubble. Most of the time markets work very well.
It's also worthn nothing that bubbles are much less spectacular than they used to be, thanks mainly to the crowd learning about bubbles and crashes. If you've read "The Madness of Crowds", you'll know that some of the early ones caused widespread financial disaster. Our most recent ones, the Internet bubble and the current real-estate bubble, are pretty minor by comparison.
um. the internet bubble was the greatest bubble since the great depression. there are many markets in the world. when you consider them all, speculative bubbles happen quite frequently, and they are always nasty.
Comment removed based on user account deletion