Slashdot Mirror
Teaching Computers to See with Games
An anonymous reader writes "The Pittsburgh Post-Gazette has a story on Peekaboom, a two-player on-line game in which one player tries to get the other player to guess a word associated with an image, by revealing parts of the image one click at a time. From the article, "The process of revealing objects, or highlighting images within the larger context of the photo, is the sort of thing that researchers in computer vision must do to teach computers to see.""
Teaching, computers, games, yeah, fascinating... so, what's the deal with the moderation here? Why are there so few comments with scores over +3? My default is +5 and the whole front page right now shows *zero* comments at that level. Did they get real stingy with the mod points all of a sudden?
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
There's an old story from the early neural net image recognition days that seems germane to this. A group of researcher were trying to train an artificial neural net to recognize military tanks that were partially hidden in forested scenes (this was the bad old Cold War days and spotting Soviet tanks in West German forests was the problem du jour). Pictures of natural forested scenes with and without tanks were used to train and test the system. It seemed to work very well on all the training and test data.
But when they tried the system on more images, it failed miserably. Further investigation revealed that, by accident, all of the "tank" pictures had been taken on cloudy days and all of the non-tank pictures had been taken on sunny days. The system had learned, and learned beautifully, how to recognize cloudy vs. sunny days.
The point is that the software was good enough to learn to recognize the difference between the two populations of images but that that difference wasn't the one intended by the people working on the system. In the same vein, I'm sure that Peekaboom will learn to distinguish between objects in images but whether it learns the actual object or just some incidental characteristic of that pocture of the object will require a very very good diversity of training pictures to avoid accidental, non-meaningful patterns in the image data.
I do wish them luck. Perhaps Peekaboom could create a distributed version of the training process in which others can both submit and help train on new objects/images. Letting others submit images and train the system would help diversify the training & testing data sets. Because some people will, no doubt, submit porn, I'm sure the system might become quite adept at recognizing the nether regions of the human body.
Two wrongs don't make a right, but three lefts do.
Actually, this work is more related to his prior work on the ESP Game, which collected labels for images. The problem after that is that you know an image contains a boy and a dog, but you don't know what is the boy or what is the dog.
With Peekaboom, you give them the job of guessing "dog", and the parts of the image that are revealed are likely the parts of the image that contain the dog.
That said, the relationship to CAPTCHAs is still there. Simple image distortion CAPTCHAs don't really hold up, and the more difficult ones are based in the semantic understanding of an image.