Slashdot Mirror
Teaching Computers to See with Games
An anonymous reader writes "The Pittsburgh Post-Gazette has a story on Peekaboom, a two-player on-line game in which one player tries to get the other player to guess a word associated with an image, by revealing parts of the image one click at a time. From the article, "The process of revealing objects, or highlighting images within the larger context of the photo, is the sort of thing that researchers in computer vision must do to teach computers to see.""
How apt, first five or six attempts to view this got:
:)
Nothing for you to see here. Please move along.
Game dev and music blog
Also known as Pictionary..
If you're tired, sleep! Wenn Sie muede sind, schlafen!
unless you get to reveal the picture using crayola crayons...
potato and paint is my favorite method
Could they use /. to teach a webserver how to stay up...?
... tell MS about this! Can you imagine?
Free Firefox news reader.
So what?
I will possible to use that in biometrics
Luis Freire
http://webdicas.com/
http://www.numberbit.com/
I always wanted a computer that could identify my predetermined pr0n fetishes and automatically download accordingly...then I could cut the browsing time in half and get right down to business.
Keep going with the science, but I'm revoking your right to name anything. "Peekaboom." Good God.
A few weeks ago there was an article about teaching a computer to play chess using a bayesian spam filter. While it was kludge-y, it was a pretty good idea, and had some interesting results.
Why not try that with vision? Ditch the spam filter and use high-end bayesian analysis, feed the bayesian learning-program all of the data about different objects in a video game who edges are already defined, usually by colors and texture borders, see what you get.
I'm no expert on this-- can anyone offer ways it could or couldn't work?
Yep, most boring game in the world.
My wife's sketchblog Blob[p]: Gastrono-me
Always pick a hint.
That adds 25 points to your score.
During the bonus round you get points for clicking the same spot as your teammate. Once numbers start appearing, keep clicking right there for maximum points.
Pass if the word looks difficult. Don't hesitate.
Pass if your partner passes, too. He probably has a good reason to.
Nowwww. Let's see. Boob!!!
"I used to have that really cool,funny sig
Teaching, computers, games, yeah, fascinating... so, what's the deal with the moderation here? Why are there so few comments with scores over +3? My default is +5 and the whole front page right now shows *zero* comments at that level. Did they get real stingy with the mod points all of a sudden?
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
Let's say that we can get the robot to recognize something, perhaps an egg. How can we get the robot to understand that an egg can also be cracked and that the insides are edible? At some point the robot may drop the egg on the floor and it will break. How can the robot know that eggs are not meant to be broken on the floor? Will it understand that the floor is not necessary when breaking an egg, neither is extra claw/hand pressure. We need to be able to trust that the robot can learn how to crack an egg so that it doesn't make a mess when cooking an omelette.
The whole problem is much bigger than simply recognizing one thing or another. Any object, whether a face, an egg, or anything, has a set of things that can be done to it. Just recognizing an object for what it is does not lead to any great understanding of what that object is and what can be done with it.
If we were to have to provide a database of actions for each and every object that a robot could ever encounter, I would hardly call that "teaching". In humans, we teach children certain heuristics so that they have a general understanding of the world around them. But hard-wiring in our brains also allows us to think outside of those heuristics and in some cases to think past the boundaries of current thought (geniuses and savants).
How can we get a robot to 1) understand simple heuristics, and 2) to think beyond its initial programming? Can we provide a robot with such programming that it can grow in "intellect" on its own?
Bypassing captchas?
Each day Research takes us one step closer to A. I
Chris ,
Php Programmers.
The article only says that this technology has the *potential* to help computers to see objects, not that it *is*.
Quothal:
The process of revealing objects, or highlighting images within the larger context of the photo, is the sort of thing that researchers in computer vision must do to teach computers to see.
While the ESP Game was designed to generate descriptive labels for photographs and other images, Peekaboom is intended to help teach computers to see.
There's an old story from the early neural net image recognition days that seems germane to this. A group of researcher were trying to train an artificial neural net to recognize military tanks that were partially hidden in forested scenes (this was the bad old Cold War days and spotting Soviet tanks in West German forests was the problem du jour). Pictures of natural forested scenes with and without tanks were used to train and test the system. It seemed to work very well on all the training and test data.
But when they tried the system on more images, it failed miserably. Further investigation revealed that, by accident, all of the "tank" pictures had been taken on cloudy days and all of the non-tank pictures had been taken on sunny days. The system had learned, and learned beautifully, how to recognize cloudy vs. sunny days.
The point is that the software was good enough to learn to recognize the difference between the two populations of images but that that difference wasn't the one intended by the people working on the system. In the same vein, I'm sure that Peekaboom will learn to distinguish between objects in images but whether it learns the actual object or just some incidental characteristic of that pocture of the object will require a very very good diversity of training pictures to avoid accidental, non-meaningful patterns in the image data.
I do wish them luck. Perhaps Peekaboom could create a distributed version of the training process in which others can both submit and help train on new objects/images. Letting others submit images and train the system would help diversify the training & testing data sets. Because some people will, no doubt, submit porn, I'm sure the system might become quite adept at recognizing the nether regions of the human body.
Two wrongs don't make a right, but three lefts do.
Fix the damn moderation system! /. without mod points at low thresholds.
No posts at 4+ for a long long time.
I am not gonna read
If you want to vote for it - reply below.
In some sense, Slashdot is the ultimate MMORPG. The comments system provides almost immediate feedback in the form of replies and moderation. Most posters can be categorized into some sort of stereotype.
Some posters like giving lots of information and opinion and getting lots of replies in return. They typically have a little background in what they are discussing, or have a very strong opinion on the subject. When they post and get modded up and have lots of replies, they have achieved a personal victory.
Other posters enjoy causing mayhem. They will typically post a comment taking a very odd stance towards a topic that many people feel strongly about, or they may post blatantly incorrect information on a topic that everyone is well-versed in. Their goal is not direct replies specifically, but rather that a heated debate follows from that first troll post. The best troll posters are those who can get both a slew of replies and start a flamewar. Moderation is a peripheral concern to these players, but they obviously prefer to be modded upwards rather than downwards.
Another player is the newb. This player simply doesn't get that the forum is a game populated by players much better than he. He posts replies in earnest to troll posts and karma whore posts, and may try to make on-topic jokes. This type of user is frequently seen making Star Wars references, posts about "42", and other stupid things that garner him neither karma nor respect from his peers. He is also frequently seen repeating Benjamin Franklin's worn out "those who would blah blah blah" catchphrase.
Finally there are the vermin of the forum. These will typically post off-topic comments about all sorts of strange fetish behavior. Whether it be the innocuous first posters or the ASCII art purveyors, these posters are not welcomed by most of the community. That they are able to stick around despite constant down-modding is a testament to their cockroach-like existence.
However without moderation, no one is interested in posting. The last few stories have only a handful of comments and most of them are posted by the vermin. The karma whores don't stick around because there is no payoff, the newbs are all gone because they follow the whores like flies on dog crap, and the trolls have no one to troll with the newbs and whores gone.
The healthy Slashdot ecosystem is significantly disturbed by this sudden lack of moderation.
This would be great if people on the internet weren't retarded. I bet when I click over to Fark, this is on there. Maybe Bill Gates bringing computers to the masses wasn't such a bright idea.
http://www.aladdin.cs.cmu.edu/workshops/lamps05/Sl ides/Peekaboom.ppt
the one in google's index now seems to be broken
If you get a word that's possibly inappropriate for children (boobs is the mot common one, but also tits, gay, sex, ass, etc), pass immediately. There is a filter that will prevent your partner from guessing these words, but it will still give you pictures labeled with them.
Seconding the use of labels. They're worth 300 points over the course of a full game. If none of them apply, because, for example, the word is an adjective, select "text". It's the least likely to be misleading to your partner.
Common labels:
man, men, woman, women, people, building, buildings, sky, water, grass, tree, trees, airplane, leopard. If you see something that could be one of these, guess it quickly.
Try various levels of specificity. Many of the pictures have very generic labels, others are very specific. If you see a duck, try "bird", "animal", and "mallard" as well.
Some pictures have labels that describe the picture itself, rather than it's content, such as: picture, photo, drawing, page, rectangle(!), etc. If your partner seems to be trying to uncover the whole image, try some of these kinds of words.
Finally, this was mentioned in the parent, but cannot get enough emphasis. DO NOT BE AFRAID TO PASS. If you get a clue that you don't think you can show, pass. If your parter passes, pass. If you can't think of any more synonyms for the words your partner has marked as hot, pass. Passing costs you nothing more than the few seconds it takes to start a new image. Do not waste time on an image you're not going to get.
My research involved developing a software system to "learn" a protolanguage of nouns/verbs based on visual perception. Part of the vision system involved having the computer detect "significant" objects & relationships in video frames and tracking similar object/relationships across both different frames & different videos. Here's a short paper.
This way to the egress...
As a computer vision researcher, I thought I'd share a little insight as to why this is helpful for the computer vision community.
Whenever one wants to train an algorithm to detect or recognize an object in a data set, one needs both the data set and the ground truth. The data set is usually a large set of images and the ground truth is some semantic information associated with each image, such as the locations of people and cars, or perhaps a representative word or category. The data set is usually easy to obtain, however the ground truth usually involves manual input. Considering that data sets regularly have more than 10,000 images, this can be quite a challenge since it can't be automated (if it could, your research would be pointless eh?).
This is where the peekaboom application comes into play. Now, the task of annotating the images with semantic information is distributed among thousands of slashdot readers and other assorted nerdy individuals. Not only does this program provide a single ground truth for researchers to analyze, but a statistical description of the ground truth, that is validated by another user guessing the semantic information.
More information about this project can be found at: http://www.cs.cmu.edu/~biglou/research.html
As an aside, a friend of mine is working on a project to turn planetary transit model fitting into a web-based game. Keep a lookout for it in the next few months.
With the current crop of games (like Doom 3 and GTA) I would have expected the article to be titled:
Teaching Computers to Kill with Games
The premise was that no algorithm existed for computers to be able to find where within a picture a certain object existed. But who is good at doing these things? People. Normally, a group of people would be paid to sort through hundreds of thousands of different images and find where a certain object was. But this was slow, and consumed unnecessary resources (like money). However, the ingenious people at CMU developed a clever way to make it fun, so much so that people would actually WANT to do it. By making a game.
The creater, a person whose name is Roy, spent a year working on the game. Please don't diss it too much. How would you like it if someone dissed your program that you spent your life on, making it as good as possible?
On a side note, the ESP Game was sold to Google for 1 million dollars. Not kidding.
Maybe 'face' recognition systems could use the same strategy for free research.