Slashdot Mirror
Teaching Computers to See with Games
An anonymous reader writes "The Pittsburgh Post-Gazette has a story on Peekaboom, a two-player on-line game in which one player tries to get the other player to guess a word associated with an image, by revealing parts of the image one click at a time. From the article, "The process of revealing objects, or highlighting images within the larger context of the photo, is the sort of thing that researchers in computer vision must do to teach computers to see.""
I always wanted a computer that could identify my predetermined pr0n fetishes and automatically download accordingly...then I could cut the browsing time in half and get right down to business.
Teaching, computers, games, yeah, fascinating... so, what's the deal with the moderation here? Why are there so few comments with scores over +3? My default is +5 and the whole front page right now shows *zero* comments at that level. Did they get real stingy with the mod points all of a sudden?
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
I'm no expert on this-- can anyone offer ways it could or couldn't work?
The human eye works in a similar way. The first layer of optic nerve after the retina recognise dots. The next layers recognise contrast and patterns in the previous layers, i.e. lines, edge recognition, etc. By the time it gets to the brain it's already broken down into basic shapes, at which point there are nerves that have been taught to look for certain combinations of shape and colour are triggered, causing the sensation of recognition.
I assumed some pattern recognition would already work like this. Could be wrong though.
The article only says that this technology has the *potential* to help computers to see objects, not that it *is*.
Quothal:
The process of revealing objects, or highlighting images within the larger context of the photo, is the sort of thing that researchers in computer vision must do to teach computers to see.
While the ESP Game was designed to generate descriptive labels for photographs and other images, Peekaboom is intended to help teach computers to see.
There's an old story from the early neural net image recognition days that seems germane to this. A group of researcher were trying to train an artificial neural net to recognize military tanks that were partially hidden in forested scenes (this was the bad old Cold War days and spotting Soviet tanks in West German forests was the problem du jour). Pictures of natural forested scenes with and without tanks were used to train and test the system. It seemed to work very well on all the training and test data.
But when they tried the system on more images, it failed miserably. Further investigation revealed that, by accident, all of the "tank" pictures had been taken on cloudy days and all of the non-tank pictures had been taken on sunny days. The system had learned, and learned beautifully, how to recognize cloudy vs. sunny days.
The point is that the software was good enough to learn to recognize the difference between the two populations of images but that that difference wasn't the one intended by the people working on the system. In the same vein, I'm sure that Peekaboom will learn to distinguish between objects in images but whether it learns the actual object or just some incidental characteristic of that pocture of the object will require a very very good diversity of training pictures to avoid accidental, non-meaningful patterns in the image data.
I do wish them luck. Perhaps Peekaboom could create a distributed version of the training process in which others can both submit and help train on new objects/images. Letting others submit images and train the system would help diversify the training & testing data sets. Because some people will, no doubt, submit porn, I'm sure the system might become quite adept at recognizing the nether regions of the human body.
Two wrongs don't make a right, but three lefts do.
The site works like a champ on my Mozilla (bangbang023). The game is a Java app and requires Sun's JRE 1.4 or later, so perhaps that's what's causing the problem for you.
Actually, this work is more related to his prior work on the ESP Game, which collected labels for images. The problem after that is that you know an image contains a boy and a dog, but you don't know what is the boy or what is the dog.
With Peekaboom, you give them the job of guessing "dog", and the parts of the image that are revealed are likely the parts of the image that contain the dog.
That said, the relationship to CAPTCHAs is still there. Simple image distortion CAPTCHAs don't really hold up, and the more difficult ones are based in the semantic understanding of an image.