Slashdot Mirror

ESP Game:
http://www.espgame.org/

More info:
http://radar.oreilly.com/archives/2006/09/more_on_google_image_labeler.html

Very interesting video:
http://video.google.com/videoplay?docid=-8246463980976635143

Yes, some places like Slashdot have managed to build a public gathering spot and sell some ads around it, but it's quite another to get this crowd to do real, coordinated work.

One of the "Tricks" to the Crowdsourcing phenomenon is to provide a way for users to create value without feeling like they are working. Slashdot has done that to the extent that you have created content (your posting) questioning whether anyone has done this. Tom Sawyer indeed.

Check out the Carnegie Mellon projects, The ESP Game, Peekaboom, and Phetch for more examples where users are providing valuable services FOR FREE while playing a game. Similar to how you and I are creating value for free in this forum with our witty banter. It feels rewarding to post a comment. And it creates a valuable end product for Slashdot.

One "crowdsourced" concept that I find to be totally unethical is the archival of student papers. Force students to submit papers to your service, in the name of plagiarism-checking, and then hold them FOREVER, and build a database of content so that you can use other people's Intellectual Property. The McLean trial starts around January 23rd. Hopefully Slashdot will be covering it.

Sounds like what they're doing at Peekaboom and The ESP Game, harnessing humans to solve problems that are difficult for computers.
Here's an nice video on the subject.

Or better yet, just go ahead and play The ESP Game and Peekaboom.

The ESP game http://www.espgame.org/ was originally developed by Luis von Ahn http://www.cs.cmu.edu/~biglou/, and seems to be the basis of the Google product.

If you talk about brilliance, this has been done before http://www.espgame.org/

Beware, it may be addictive. ;)

This sounds like the ESP Game project of Carnegie Mellon University.

-1 Copycat...

http://www.espgame.org/

Props to Luis von Ahn's group at CMU.

Anti-props to Google for ripping it off without credit.

It looks like google just created a clone of the ESP Game.

I kind of feel bad for Cyc/OpenCyc... they've put so many years into this project, but using web-based games to collect and verify this common-sense data is much faster than using a few paid experts and can give much more data. For the curious, Luis von Ahn, a grad student (and now assistant professor) at Carnegie Mellon University gave a (rather entertaining) tech talk at Google about his work in this area.

He's recently been working on a project called Verbosity, which uses such games to collect the same sort of common-sense data that Cyc has been trying to collect all these years. Cyc's ontology apparently contains "hundreds of thousands of terms, along with millions of assertions relating the terms to each other." If Verbosity is as popular as von Ahn's ESP Game, the game could probably construct a better database in a matter of weeks.

Here's the abstract from a research paper on the topic:

Verbosity: a game for collecting common-sense facts

We address the problem of collecting a database of ""common-sense facts"" using a computer game. Informally, a common-sense fact is a true statement about the world that is known to most humans: ""milk is white,"" ""touching hot metal hurts,"" etc. Several efforts have been devoted to collecting common-sense knowledge for the purpose of making computer programs more intelligent. Such efforts, however, have not succeeded in amassing enough data because the manual process of entering these facts is tedious. We therefore introduce Verbosity, a novel interactive system in the form of an enjoyable game. People play Verbosity because it is fun, and as a side effect of them playing, we collect accurate common-sense knowledge. Verbosity is an example of a game that not only brings people together for leisure, but also collects useful data for computer science.

In order to use the p0rn site he ran, you had to either pay money or spend time identifying captchas.

The second approach was simply to set up captcha solving sweatshops somewhere in Asia with cheap labor, with people paid a few cents an hour to sit and solve captchas all day. This brought the cost of a new email address up to something like 1/3 cent, which for many spammers is still a viable price. The cost does limit this approach, though, so the captcha still helps.

The interesting thing about both of these strategies is that they use humans to solve a problem that is difficult for computers, which is von Ahn's research area - he's also one of those behind The ESP Game (caution - this can be shockingly addictive). There's essentially nothing that can be done to defeat either approach without also making a system a huge pain in the ass for legitimate users. From this point of view, spending time trying to come up with more advanced captchas is kind of pointless.

They're designed to get people to search google's other databases. Right now, if you use google search engines, you can click on those links above and get results relaetd to the search you just did. Adding relevance indicators makes this more obvious, but it simply wouldn't work horizonatally. Even if it did, as google adds more types of search it would eventually be infeasible. Strangely, the search categories aren't ordered by relevance to the current term.

Searching journal articles and the WWW for text makes some degree of sense, but searching for images via text is less useful, and video even less so. TheESPGame is an interesting site where you tried to guess how another person describes a picture. A picture might be worth a thousand words, but only perhaps five of them would be commonly used between people. Fortunately, we know what they are, so that game is winnable. This is what makes Google's ability to extract relevant words reguarding pictures on the web tenable, but these days I'm more interested in ways to improve search in ways Google isn't doing anymore.

I've seen a site that lets you draw images and it returns pictures that closely match. It was real neat, but not the sort of thing you can just leave on a single server on the internet; it got nailed by sites like slashdot. Donno if it's a performance problem or just scaling problems, but I'd like to see it come back some day.

Similarly, it'd be neat to have audio search based on audio samples. The music genome is interesting, but it represents a huge investment of the ESP game sort. They describe various attributes of music and their Pandora tool lets me ask for music similar to that. I'd imagine it'd be far more entertaining if people could hum or sing a few lines into a mic and the computer searches for likely candidates, like a Seinfeld episode or something.

All right, so Amazon have decided to handle manual labour which is necessary and it also rewards users in return for the service. They get a description of photos without much effort and re-pay back in some form of 'coupons'. More sophisticated things have been done for quite some time. There is a full object-labelling framework where surfers compete with one another and the Web site of the game is http://www.espgame.org/ . To quote an article from Post-Gazette: "Since the Post-Gazette first wrote about the ESP Game in October 2003, more than 80,000 people have played the game and in the process have generated more than 10 million descriptive words for 1 million images"