Slashdot Mirror
Google Putting Crowd Wisdom to Work
daveperry writes "The Google Blog has a post about their use of prediction markets to forecast certain events that are relevant to their business. From the article: "Our search engine works well because it aggregates information dispersed across the web, and our internal predictive markets are based on the same principle: Googlers from across the company contribute knowledge and opinions which are aggregated into a forecast by the market. Sometimes, just feeling lucky isn't enough, and these tools can help." In related news, some software was recently open sourced that enables people to set up their own prediction markets."
use of prediction markets to forecast certain events that are relevant to their business.
Nothing to see here....hmmmm...bet they didn't see that one coming.
I just recently started participating in it. Doesn't seem too complicated, though I'm still a bit unclear as to the relevancy of the predicitions it generates. It seems to me that for this sort of thing to work right, you'd need a much larger sampling (which maybe Google is hoping to get), but then, maybe I just don't know what I'm talking about?
And once again John Brunner wins. Can't believe the Shockwave Rider was written in 1975... compensated abstinence zone in New Orleans anyone?
We also found that the market prices gave decisive, informative predictions in the sense that their predictive power increased as time passed and uncertainty was resolved.
This is why I like Google. The use of intelligence to develop accurate results in a predicative system, and keep it all flowing -- it shows not only wisdom -- it shows an early level of omnipotence, which has to be the key ingredient to success today! If God is supposed to be omnipotent, why not try it? (haha I'm not a religious nut, FYI... just like to use the data available)
Consider the alternative solution to success and you really must put your investment dollars where you have the most faith. Google stock can only go up, thus breaking the law of Gravity. Only a supra-genius (Wyle E. Coyote) knows how to bend the laws that govern market economy to their favour. Are you really going to bet against that kind of mindpower?
Being geeks, we naturally used information theory to measure the entropy of our probability distributions:
This is a demonstration of wisdom. Knowing the rate of market decay is a HUGE BENEFIT for all Google stockholders. Google keeps proving that time and time again: being a geek puts you in the best position for the continuation of the species, even if you're not getting laid *today*, you will get laid PLENTY when you get lots of money, and therefore sire many children and overwhelm the market with clones of yourself. Then improve the genome of your clones in modular functionality, so that parts become interchangeable. You can now patent genes, so you don't have to patent actual clones to profit.
Seriously, at this rate, I see this coming as the next logical step for Google. Bioharvesting of nerds for fun and profit. Especially nerdy gamer chix!
The dangers of knowledge trigger emotional distress in human beings.
Want a tip on when a stock is going to move? Monitor the number of times your users send email to one another containing the stock's symbol in the message. When the number goes up, activity is sure to follow.
But they wouldn't do that, right? Because...
Because...
Exactly.
--
You didn't know.
Hmm... prediction markets... Ahh: http://en.wikipedia.org/wiki/Prediction_market
Wonderful, Google realizes that many people think they're absolutely wonderful and finds a way to put those people to work for them. As for the possible MSN/AOL deal about to occur and "kill" Google, with ideas like this and a willing user base Google isn't threatened at all.
Support alternatives to Paypal: http://www.e-gold.com
GoogleAstrology
1. Predict Market
2. Act on Prediction
3. Cause Market Impact
4. Need New Prediction!
5. ????
6. Profit
exit();
"Group wisdom" is a misnomer for "herd mentality."
I predict that when you herd sheep from field A to field B, they will eat whatever is in field B rather than return to field A.
Read any good sonnets lately?
Reminds me of E.T. Jaynes's "Honest Weatherman" example in his book, Probability Theory: The Logic of Science (section 13.5). By making a weatherman's salary proportional to a certain function (logarithm of the probability he predicts, or entropy, or something, I forget), the weatherman has an incentive to make the best possible predictions: his salary will be directly proportional to the quality of his predictions, being maximized when his rain forecast probabilities match the actual probabilities of rain. You could set up a "free market betting system" this way, rewarding the quality of people's guesses regarding the likelihood of various outcomes.
I'm doing my best to ensure it does.
with terrorism? It's too bad people got all pissy since this method of prediction works fairly well.
Predicting involves extrapolating from a current ** sample** of data to predict the future based on one own interpretation /recognition of patterns.
The better your size and quality of your sample combined with a finely tuned pattern recognition the better your forecast (I won't go into exceptional events which by definition are exceptional).
So what do we have here? A larger sample of better quality for starters.
Also do not underestimate the power of the masses. If your sample=population size this is no longer forecasting (i.e. extrapolation) but the writing on the wall! (as long as people do as they say they are going to do).
So if your sample size increases dramatically, with better quality (smart employees) things will tend to happen as per the survey!
I know I am oversimplifying but but these are the basics of neural networks (or in this case a neural network of neural networks). Look it up.
Of course, predicting is not extrapolating if it is purely a random guess.
Have fun!
Artificial intelligence is no match for natural stupidity
I've started playing Yahoo! Buzz games for a few months now (I dropped a virtual packet on FireFox). It's called Yahoo! Buzz game. These folks seem to gather their data from Yahoo ! search queries and from the logs on who clicked what. Which is why it seems to really follow the non-geek popularity levels too well - Google is what the geeks use (which is why I almost always seem to lose there).
This is just Google calling Market surveys by another name. I'm sure I can dig up a couple of WalMart papers about the same thing in the real world. Sort of vote with your money approach vs tell us approach (demand vs desire).Quidquid latine dictum sit, altum videtur
This is really cool, but I hope they control for two of the less-than-ideal behaviors of markets.
1) There are two ways to "be right" in a market. First, I can make the right choice as to the actual ending outcome. This a buy-and-hold strategy. Or, second, I can make the right choice as to the direction of price movements in the market. This is the speculator's buy-and-sell strategy. The first strategy means the the market converges on the "true" expected value. The second strategy leads to bubbles and crashes that don't provide as much useful data on the actually variable being modeled by the market. Google wants to encourage the first type of trading, but not totally suppress the second type of trading because speculators provide liquidity in markets.
2) Manipulators can "cause" events to occur in the way that maximizes their return, but suboptimizes Googles performance. If I bet that a given project will be done in October and it looks like its getting done early, what stops me from causing a small delay? Of if the project is being delayed too much, what stops me from descoping the project or doing a fast, low-quality job to complete the project within my chosen time frame. In either case, I can manipulate the outcome to win in the market, but hurt Google.
Note that I don't include insider trading in the list problems. Google doesn't really care if the market is fair, only that it provides accurate predictions. In fact, Google might encourage insider trading as a way to encourage communication inside the company. The more people that share their "inside" information on upcoming strategic, the better.
Two wrongs don't make a right, but three lefts do.
... they do correctly predict:
- launch date of MS Vista;
- launch date of Ballmer's furniture;
- which 'feature/bug' will be dropped from the Vista release.
Disclosure: I'm stupid
Hey, I wonder what Google's doing today, I haven't heard anything about them lately.
The Pentagon was ahead of its time I think.
Except for ending slavery, the Nazis, communism, & securing American independence, war has never solved anything.
The classic example of "crowd wisdom" is the jellybeans-in-the-jar experiment, often used in introductory MBA classes to convince people that open markets value securities (basically) fairly. The experiment goes like this: the professor brings a jar of jellybeans and asks everyone to guess how many there are in the jar. The individual estimates may vary quite a lot, but the average of the estimates in the class is usually close, in fact often closer than the closest estimate of any of the students depending upon the size of the class. That is in a situation where the students have very little information about the jar and perhaps no experience with such estimates. If there were greater experience and/or they were allowed more information then presumably the individual and average estimates would be even closer. Basically, this can be described as the "Central Limit Theorem" in action- that the standard deviation of averages is smaller than the standard deviation of the individuals by a factor of the square root of the sample size, as illustrated in this applet or in this Mathworld description. The CLT actually says more- that as the sample size increases, the distibution of averages approaches that of the normal ("bell curve") distribution, so the distribution of avergaes is roughly normal, and then techniques designed to analyze the normal distribution can be applied with greater certainty.
It's psychosomatic. You need a lobotomy. I'll get a saw.
Its "well documented" that financial markets are not normal. They are "fat-tailed"--extreme events happen with regularity. This attempt has been tried thousands of times by people with far more at stake than google. They also had more money, more PhDs, etc. And they failed. They always fail at some time or another. There is no algorithm for the financial markets.
Those who doubt had best read the story of Long Term Capital Management. wikipedia link
What Google is doing is interesting, but it's no magic bullet. And I'm not even sure I'd use this system to place money on the markets. Fads come, fads go. Neural Networks. Genetic Algorithms. Fibonacci numbers. Eliiot Waves. what-have you. They're all nonsense. The big ol boys still make their money the old fashioned way: graft, theft, and deceipt.
Anyone honestly think Google knows something that the trillion dollar financial industry doesn't? They have far more at stake that Google. They have their fair share of MIT PhDs, economic nobel prize winners, sparc computers, access to nearly every price in the world at any time. And they still make the majority of their cash through commission.
Call me a cynic, but the math says this can't work. And I'll trust the math before I trust a company that doesn't even pay dividends any day.
http://zocalo.sourceforge.net/
Great realtime prediction market, written in python, uses ajax for updates, very slick. (Disclaimer: I was an intern with zLabs over the summer and chatted with the developers often, very smart people)
Yahoo has launched one of these to the public back at in March of 2005 at the O'Reilly Etech conference. They actually had a contest where the top performer got a Mac-mini.
sigs are a waste of space
I see all these dismissals, as if this is a joke. But might I suggest that the researchers at Google aren't trying to get an acceptable sample-size, as many people here are naively suggesting. It seems much more likely that Google is researching the thought-process of prediction itself. It also seems likely that a number of fairly intelligent industry-insiders also believes they're close to figured out how to create a prediction engine that is accurate (ahref=http://slashdot.org/article.pl?sid=05/09/22 /1229238&tid=109&tid=120&tid=217rel=url2html-6813h ttp://slashdot.org/article.pl?sid=05/09/22/1229238 &tid=109&tid=120&tid=217>)...
Think about it; If they have enough information (IE the web), they can hone it by feeding it past events to "predict" other past events and then comparing their prediction to what actually occurred. If it's able to predict events that have occurred given what came before those events, then there's a good chance that it will work with current data to predict future events.
Feel free to label me a wacko... Perhaps I've seen "Terminator" too many times. But I've also worked with a number of freakishly intelligent people; the kind that Google has been hiring right-and-left without any apparent reason. I've always said that technology could someday facilitate society's return to bondage... It may seem far fetched, until one considers the sheer breadth of people that hate "western" culture. Given the past as a predictor, I would be inclined to believe that few Middle Eastern leaders would hesitate to use technology to exterminate as much of western culture as they could... Folks, its not unrealistic to recognize the likelihood that technological-advancements will make sudden and drastic changes to our way of life... This ought to give us pause. What balances are essential to our culture? What imbalances will end it? Someday, somewhere, somebody is going to come up with a way to utilize technology to facilitate his/her agenda. Maybe we don't have to worry about Google, since the magnitude of the consequence is, at least in part, proportionate to the magnitude agenda of the individual(s). But things such as this should be cataloged in our brains as evidence that this mode of societal-failure is plausible.
Ok, flame away. But I hope I never have to say I told you so.
Thats not really what the main part of the book is about.
Its about distributed decision making. Its not a book about making money from the stockmarket, its about optimal decision making by the market. Indeed, the whole point is that markets are "smarter" than any individual who participates in them. Its actually an idea in opposition to the view that individuals can stand to "beat" the market by rejecting conventional wisdom. But furthermore, the fact the market makes "wise" decisions doesn't necessarily equate with anyone getting rich or otherwise getting what they want either.
I don't think you've actually read the book.
Plays violent online games as: Nerfherder76
This actually has nothing to do with the CLT. The CLT is a statement about the distribution of a sample average in relation to the population average.
OTOH, TFA is about information markets, which concerns comparisions between the average of people's forecasts of events, and the "true" (or natural) frequency of events. We are not comparing a sample average to a population average. (In other words, to apply the CLT here, we'd make a statement comparing the average forecast of 100 people to the average forecast that would be made by all the people in the population. We're absolutely not doing that.)
Instead, the concept to discuss here is what is sometimes called calibration. This is a concept used to measure the accuracy of forecasters (such as meterologists). It works like this: say we look at all the days a weatherman said "chance of rain is 30%". Then, we'd hope to see rain on around 30% of those days in order for that kind of prediction to be accurate. Calibration (roughly) asks: How accurate were all such statements that were made by the weatherman? (I.e., check "chance of rain 0%", 10%, 20%, etc... looking at each subsample of his forecasts.)
Here (pdf) is a randomly chosen article on this kind of calibration.
For those that read TFA, did you see the graph? Calibration asks for those two lines to overlap. Whether they do or not has nothing to do with the Central Limit Theorem.
CmdrTaco has been trying to do this for years with his comment moderation system. I think a key problem is that he dosen't use a big enough sample size. It only takes a few people to moderate a comment up to the highest level. The rules of crowd wisdom state that the larger your sample size, the more likely you are to arrive at the "correct" answer. Granted, with something like a story comment, there is no "correct" answer, only interesting and relevant responses. CmdrTaco's goal was not to tease out the interesting comments though, it was to filter out the irrelevant and wasteful spam.
In essence, CmdrTaco had no choice. Spam was starting to choke slashdot comments and making them less than useful. The moderation system saved the comment system, but didn't, as many people assume that it should have done, make the comments more interesting.
I believe that if the prevailing attitude among slashdot developers is to "weed out the spam", we'll see a slow decline of slashdot's popularity until it's made irrelevant by RSS feed aggregators.
IMHO, the attitude *SHOULD* be to exploit slashdot's major differentiator over simple aggregators, which is the community it has created. In other words, they should invert the "weed out the spam" attitude into a "make the comments more interesting" attitude. It's a subtle difference and, on the face of it, it would appear that one begets the other. I contend that weeding out spam does not make comments more interesting and conversely, making comments more interesting won't weed out the spam. Thus we come to the root of the problem, two crosswise goals.
CmdrTaco has to worry about the system from a performance standpoint. Weeding out the spam means less bandwidth and storage costs. That's immediate ROI, and a good thing on many levels. The community, however, needs more than 1,2,3,4 or 5 to determine what comments to read and which to ignore, to make them interesting. I can conjecture at a few ideas that would make it better, but I do not know the ultimate solution, and I doubt anyone else does either. I believe the problem requires more than just CmdrTaco playing whack-a-mole with ideas, meta-ideas and meta-meta-ideas etc. It requires serious PhD dissertation level study.
*Condense fact from the vapor of nuance*
Comment removed based on user account deletion