Slashdot Mirror
Facebook Thinks Occlusion Is the Next Great Frontier For Image Recognition
An anonymous reader writes: Researchers at Facebook AI Research (FAIR) have published a paper contending that image recognition research is now advanced enough to consider the problem of occlusion, wherein the objects AI must identify are either partially cropped or partially hidden. Their solution is the predictably labor-expensive route of human annotation of existing image-set databases, in this case 'finishing off' occluded objects with vector outlines and assigning them a z-order. This article looks at the practical and even philosophical problems of getting IR algorithms to 'guess' objects usefully, and asks whether practical IR research might not be currently limited both by the use of over-specific image datasets and — in the field of neural networks — by problems of theory and limited 'local' processing power in critical real-time situations.
It's a slashdot record! It's blatantly wrong, within the first two words of the title .
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
you
Especially black faces. Facebook is so racist. So racist. Their Republican rulers are racists. If they can't even find a face, how are they going to identify other objects, especially by looking for only parts of other objects.
If a series of images is available and observer or target or intermediate objects are moving, occlusion will vary image to image and the nature of the delta portions should be highly informative for recognition. This requires an object/region re-identification subsystem.
Also, scene context statistics should be used, much as preceding utterances are used in speech recognition. Given that we've already recognized a situation type with this that and the other object-type in it in this (possibly dynamic) relation, what are the a priori probabilities for these other types of objects to occur in scene, and assess occluded objects against highest probability objects in situation type. Much more constrained/determined recognition problem in which pieces of objects might suffice to identify them.
Where are we going and why are we in a handbasket?
So if regular object recognition is such a solved problem, why to they need people to manually prepare the images? I'd just take a normal image, recognize the objects, and then partially cover some of them to train their algorithm.
The use of vector completion and all is a good idea, but it seems systems like that would work better in conjunction with other techniques, like trying to consider context of the area where you are in. What is behind a tall narrow object varies a lot depending on if you are in a jungle vs. a parking garage...
"There is more worth loving than we have strength to love." - Brian Jay Stanley
That sure makes sense. Don't tell them, though. The inability of image recognition software to handle cropped pictures is one thing which my better replacement for CAPTHCA depends on. CAPTHCA sucks because humans aren't much better at computers at recognizing squiggly letters. We are, however, MUCH better at recognizing certain specifc types of images when they are cropped and rotated.
Pull back. Wait a minute. Go right. Stop.
Enhance 57 to 19. Track 45 left. Stop.
Enhance 15 to 23.
Gimme a hard copy right there.
The main application I see of this is in detecting pornographic images containing a penis that's partially embedded in, and obscured by, a vagina or an anus.
While thanks to technological advances it may be trivial to detect a fully erect and unobscured penis, detecting a penis that is only partially visible could very well be an extremely difficult problem.
It's difficult to censor such images when it's difficult to determine if they depict penetration to begin with.
What about a kind of genetic algorithm to evolve candidate 3D models, and the model that best matches observations and context "wins". However, that is computationally intensive. But, it is highly parallelizable.
Table-ized A.I.
Porn!
In fact, this won't stop at merely recognising faces that are partially obscured - in the not so distant future, they will be able to recognise faces that are completely absent!
Things that are (partially) hidden behind others are more difficult to recognise?
News at eleven!
This is one scene in that film where every geek freaks out in annoyance before the explanation is given on how the system works. (admittedly there was also some artistic freedoms used in some areas because FILMS LOL)
The scene in question was the ability for the spooks to rotate around a scene virtually from one or more cameras (which we can do), but they took it to an extreme and could visualize areas that were impossible for any camera to see.
The system created a rough estimate of what a scene would probably look like just for the sake of completeness rather than accuracy.
Then, if they are lucky, a camera caught a view where there was a change in the layout of the scene, so it could be updated globally through the whole animation to create a scene of events that are reasonably accurate, given limited information.
So, that guy dropped a gameboy or something in to Will Smiths pocket at some point, one camera might have seen his bag at one angle, and another camera at another, and there was a difference in shape.
Even in the film though, they say it could be anything, even nothing. It might just be the bag was dislodged from a tense state, or the lights.
Predictive image recognition will be an interesting field for sure.