Tech Companies Draw on 'Wisdom of the Crowds'

An anonymous reader writes "News.com is carrying an article on a 'mini-conference' held at Yahoo's HQ this past Wednesday. The get-together put representatives from Google, Microsoft, Yahoo!, and HP together to talk about their experiments with predictive networks. The 'wisdom of the crowds' allows these companies to make use of the collective knowledge their employees hold to answer important questions for the company." From the article: "David Pennock, a principal research scientist at Yahoo Research, said the company has created a currency called a Yootle. It's described as a 'scorekeeping system for favors owed.' Pennock offered as an example a programmer offering to write a piece of code for a few Yootles. Or, when organizing a dinner outing, one employee could use an internal SMS tool to bid 2 Yootles for Italian and 4 Yootles for Mexican. 'If you don't get to go to the restaurant you want to, you get compensation' in Yootles, he said. Related to Yootles is Yahoo Research's experiment with a fantasy prediction market for technology called the Tech Buzz Game. It's a modified version of software licensed from NewsFutures in conjunction with O'Reilly Media and features topics like Atlantic hurricanes and portable media devices. Winners are those who predict how popular a topic will be on Yahoo Search. "

Comments

I think this has a name

[UbuntuDupe]

That sounds remarkably similar to the indifference vote that Paul Birch likes to promote. You bid for your preferred option until the money you allocated toward it would make you indifferent, and if you don't get your way, you are paid that much (instead of paying that much). If you do get your way, you pay your bid. I makes you, in essence, indifferent to the outcome. His example:

Let's try a simple example. ... You and your friends want to go to a restaurant. But which one? The Peking Duck or the Spaghetti Italiano? Charles prefers Chinese, but you're a bit strapped for cash and Italian's cheaper. You bid 50p. Charles goes 60p. The girls join in. Amy is on a diet and bids 50p for the Duck, but Beth is always hungry and bids 70p for the Spaghetti; the score is now £1.20 for Italian, £1.10 for Chinese. Amy looks at Charles, who goes up 11p to 71p. You decide to bid another 2p. Charles shakes his head. Amy reluctantly adds another 2p for Chinese herself. The final bids are £1.22 for Italian, £1.23 for Chinese. So off you all trot to the Peking Duck. Amy and Charles fork out 52p and 71p respectively; Beth gets 70p and you get 52p. Amy and Charles get the fancier but less fattening fare they wanted; you get enough money to cover the higher prices, and Beth gets enough for a larger helping. The waitress gets the penny left over.

Re:I think this has a name

[eln]

I prefer the system we use at my office: We stand around coming up with suggestions for 30 minutes. Invariably, every restaurant suggested is shot down by some other member of the group ("I'm not really in the mood for Chinese today..."). Eventually, we all sort of migrate out the door, get into someone's car, and magically end up at a restaurant no one can even remember suggesting, and we eat there.

Re:I think this has a name

[maomoondog]

The problem with Birch's scheme is that individuals are incentivized to lie about what they want in order to get more money from the group. In the example given where Charles is willing to subsidize a trip to Chinese food, you could extract extra money from him by pretending it's a huge disappointment to you.

For a strategyproof scheme, check out the Vickrey-Clarke-Groves mechanism. Basically, everyone gives weighted votes about something, and the winners of the tally pay a penalty equal to the imposition they caused the rest of the people. Only trouble is, this penalty has to be completely discarded to prevent hyjinx...

I am surprised it took this long...

[BWJones]

I am surprised it took them this long to implement as a similar project was implemented at select federal intelligence agencies through DARPA funding back in early 2002 to evaluate possible intelligence leads and threats to national security. Unfortunately the Total Information Awareness program developed out of this work and the true benefits of predictive networks using human intelligence have not really panned out due to an almost pathological reliance and worship of technology supplanting human intelligence rather then supplementing it. Only more recently have projects based on simple, yet tremendously technologies such as wikis been gaining more traction.

Re:I am surprised it took this long...

[ScentCone]

I am surprised it took them this long to implement as a similar project was implemented at select federal intelligence agencies through DARPA funding back in early 2002 to evaluate possible intelligence leads and threats to national security.

You may also recall that a particularly bad round of reporting on some related work (wherein people in the defense/intel world were "gambling" futures on which head of state, for example, would next come under attack from within, etc) resulted in headlines like "Government Officials Place Assassination Bets." They actually had to shut that one down because the media idiots got enough people to make congress creatures uncomfortable. I hope they just moved the research out of the bright lights and kept it up, but it just goes to show you that these slightly odd-seeming areas of research can be wildly misinterpreted by people who get all of their interpretation in 10-second sound bites. Um, or slashdot summaries.

Re:I am surprised it took this long...

[BWJones]

Have no fear as "betting pools" work *has* continued and they are proving themselves to be very effective. Also, you are quite correct about your assessment of "odd-seeming areas of research", as they often, particularly in a basic research sense turn out to be incredibly valuable. One classic case was a member of congress bitching about the NIH funding studies to examine bird songs without knowing or understanding some of the neuroscience implications of that work.

scorekeeping system for favors owed

[tverbeek]

My little sister always managed to keep score in our house without the use of technology. ("I took out the trash for you one day three weeks ago.") The rest of us (even Mom and Dad) found it really annoying... bordering on petty and selfish, and we're all glad she grew out of this (mostly). Doing favors for people shouldn't involve keeping score.

Re:scorekeeping system for favors owed

[tverbeek]

On the other hand, one should never perform labor for others for free or without expecting something in return.

What a cold and empty approach to life. Sounds like you don't have "friends" of even "family"... just "business associates".

However, there is nothing 'selfish' about expecting and/or demanding compensation.

No, that's pretty much the definition of it: what matters to you is yourself, not the needs or wants of others.

Brilliant

[MyLongNickName]

For an example of the wisdom of the masses, just look here. If that doesn't convince you, I do not know what will.

"Yootles?"

[Rob T Firefly]

Who came up with that name, My Cousin Vinny? "Two yoots..."

Gaming the system

[TheWoozle]

So you just always bid for a restuarant that nobody, under any circumstances, will ever actually want to go to. When you actually *want* to go somewhere in particular, you can outbid anybody else.

Anyway, my point is...this is great, except human nature will always win out. The system only works if people participate. To get maximum participation, you need some sort of incentive. As soon as there's incentive, people will figure out a way to game the system.

Yootle is a lame name

[badzilla]

I think Yootle is a stupid name for a unit of fake currency. How about... hmm... hey I know, "Flooz"! No wait, even better, "Beanz"!

More info on prediction markets

[FleaPlus]

Prediction markets are a major interest of mine. I'm in a bit of a rush at the moment, so I'll have to make some more extensive comments later, but in the meantime here's some neat links on prediction markets:

* Tradesports, a real-money prediction market on political and news events. The 2008 president market currently gives a Democratic a 50% probability of winning the White House in 2008, Hillary Clinton a 55% probability of getting the Democratic nomination, and John McCain a 49% probability of getting the Republican nomination.

* Futarchy, a system of government semi-seriously proposed by Robin Hanson which would use prediction markets as a means of government decision-making. People would vote on values, and use a prediction market to determine the optimal government policies to achieve those values, which would help get around some of the godawful stupid things democracies tend to do.

* Storage Markets, a real-money (but limited access?) market on the computer storage industry

* The Policy Analysis Market, a proposed prediction market for policies in the Middle East. It was IMHO a great idea, and could have potentially prevented some of the stupid decisions which have been made in the Middle East. Unfortunately, the government ended the project after it was the media (including slashdot) had a knee-jerk reaction to it and demonized it. The funny thing is, after the project was cancelled and the media learned more about it, coverage of the project became much more positive.

Wisdom of The Crowds

[mpapet]

Indeed...

The companies mentioned have some very smart people working for them. It's a shame the PHB's pretty much kill whatever innovation is happening in the belly of those beasts.

The wisdom of the crowds is frequently spoiled by individuals that game the system. Microsoft astroturfers on /. is a good example. So-called climate science coming from the U.S. Gov't that doubts global warming is another. As a former Tech Buzz Game player I can tell you from personal experience the game was stopped and restarted with new rules because of cheating.

The end result is the wisdom a crowd was supposed to provide essentially evaporates.

Yootles?

[SixDimensionalArray]

I'm sorry, but I fail to see the wisdom in a name as gawd-awful as "yootles"! I mean, you wouldn't expect to see some guru walking around and complaining that his/her existential theory of quantum physics and intracellular electromechanical equilibrium & interstellar space travel (IANAG - I am not a guru in those regards) was upset by not having enough "yootles" to buy a cup of coffee. Yootles are yet another substitute for good old hard cash?

Ok, maybe it's a little bit interesting, but seriously folks..

-6d

It's pretty clear...

[hikerhat]

that yahoo is going down the shitter at top speed. Get out now before you pay check comes in yootels (or whatever) rather than dollars!

Wisdom of the crowds = part of the long tail?

[dada21]

I've been playing with a collaborative filter engine called CRITEO that is completely blowing my mind in how it opens opportunities to gain that "wisdom of the crowds" bit for the average user -- not just huge companies like Amazon or these emergent venture capitalized corporations. Over the past 2 weeks I've been working on some Wordpress code to actually integrate this relevancy predictor (my results should be forthcoming by the first week of January) and it really seems like you NEED a predictive filtering engine to utilize the crowds to give each individual within the crowd relevant results as compared to just generic "ratings."

This Yootle system is interesting, but it doesn't go far enough. Just because the crowds skew towards a majority opinion doesn't mean that opinion is relevant to the majority (I know it sounds weird). Each individual will have certain likes and dislikes within that majority opinion. Without some sort of relevancy predictor, the "majority vote" is useless.

Hopefully we will see more people utilizing systems such as CRITEO's to actually take the input of the masses (thousands, millions, or even billions of decisions and ratings) and run them through a real-time engine to give everyone a unique view of what they might want/need/like/hate/etc. As I spent more time beating on trying to come up with my own quick/real-time solution, the more I realized that using someone else's services let me focus on what is best for my customer -- my content, generally.

The prediction system to rank Yahoo searches is very 2005 -- it really just capitalizes on the likes of the masses, which means it is hitting the top head of the long tail rather than the more important remaining 80%. I'd love to see a search engine that allows you to "rate" your search results or even individual search results in real time, maybe in collaboration with a system like CRITEO. Anyone interested in working on one? I'd be willing to bet that such an investment of time would give many of us a better search engine that actually returns results that are relevant to the individual's tastes rather than the masses' collective "favorites" which are usually way off base. It would also reduce the spam results greatly and open the door to the wisdom of the masses actually making a difference for each individual. What I like about collaborative filter is that 5 seconds per user can mean days or weeks saved for that user in the long run because of the 5 seconds "donated" by the million others.

Not a new concept, but loads of new applications

[Lord Puppet]

I've always been interested in ways of tapping into the crowd as a resource.

Slashdot and Digg got a lot of attention as news filters, but these things are now being used everywhere. Trusted Places for restaurant reviews, Crowdstorm for shopping recommendations, wine sites, health sites, etc., etc. I can't wait to see where this is all headed. What's the next logical step?

I bid $10 for "eat out of a dumpster".

[khasim]

Damn, it looks like I lost again. Well, looks like everyone else is paying for my lunch again.

Yes, it is an extreme example, but it shows how you can "game" that system. Not a good idea.

Color me a tad cynical

[rs79]

"Or, when organizing a dinner outing, one employee could use an internal SMS tool to bid 2 Yootles for Italian and 4 Yootles for Mexican. 'If you don't get to go to the restaurant you want to, you get compensation' in Yootles"

Alex I'll take "most retarded use of the Internet for $1000".

It's amazing poeple could go out to lunch harmoniously for decades prior to this stunning reveleation.

No wonder Yahoo is not relevant these days.

Whuffie

[DoorFrame]

Sort of like Whuffie from Down and Out in the Magic Kingdom although that was based on general goodwill and esteem, not specific favors per se. Wiki's definition is probably better:

Whuffie is the ephemeral, reputation-based currency of Cory Doctorow's sci-fi novel, Down and Out in the Magic Kingdom. This future history book describes a post-scarcity economy: All the necessities (and most of the luxuries) of life are free for the taking. A person's current Whuffie is instantly viewable to anyone, as everybody has a brain-implant giving them an interface with the Net.

Wisdom of Crowds

[Phoenix666]

Whenever I hear the phrase, "Wisdom of Crowds," I think of lemmings.