Slashdot Mirror
Predator Outdoes Kinect At Object Recognition
mikejuk writes "A real breakthough in AI allows a simple video camera and almost any machine to track objects in its view. All you have to do is draw a box around the object you want to track and the software learns what it looks like at different angles and under different lighting conditions as it tracks it. This means no training phase — you show it the object and it tracks it. And it seems to work really well! The really good news is that the software has been released as open source so we can all try it out! This is how AI should work."
The kinect doesn't have stereo cameras. It has one color camera which isn't really used for much, a IR projector (that projects IR dots all over the scene) and a IR camera. The IR camera uses the pixel distance between the dots to find the distance. The depth image you then get is used as input to the algorithm that detects the body parts and their orientation etc.
Any sufficiently advanced technology is indistinguishable from magic.
I'm looking forward to looking at the GPL'd source code. There are a lot of ways to do object tracking, and they've all generally got problems, but I was rather impressed with this presentation. It was able to track the moving vehicle while it passed into and out of shadows (non-uniform saturation), as well as track that panda while it turned around (changing its shape), and it was able to distinguish a black-and-white version of the presenter's face (not based on color). It was able to recognize objects that moved off screen, which seems to indicate that it's not just drawing a snake around the moving object. Furthermore, it doesn't seem to need to be specifically programmed to track each object (as we saw the presenter just drag-and-drop a box around his hand/face.)
This isn't a breakthrough. Much of the technology for tracking objects in this way has been out for about a decade. See this Wikipedia article for one technique for doing this:
http://en.wikipedia.org/wiki/Scale-invariant_feature_transform
who can't spell
I guess you're fluent in Czech?
The source code was already released. https://github.com/zk00006/OpenTLD
There are a few more repos here.. http://www.google.co.th/#q=site:github.com+%22TLD+is+an+algorithm+for+tracking+of+unknown+objects%22&hl=en&filter=0
it should be no problem to track individual limbs to generate a skeleton of the user
I'm not so sure about that. He is using a tracking algorithm paired to a template matching algorithm. His claim is that, although both methods have high error rates, their errors are mostly orthogonal to each other. In other words, one method works better sometimes, the other method works better sometimes, and combined, they do a pretty good job. In his videos he's left out scenes where there is a large area of near constant intensity. I'm curious how his method deals with this as there aren't enough details to track, nor are there enough features to template match. Also, with arms and legs, if the texture is generally the same between the two (say you are wearing sweatpants and a sweatshirt of the same color), then there really isn't enough information for the tracker to work with in order to distinguish a leg from an arm. Straight arms and straight legs will both match the template, the tracker will likely struggle with the relatively large area of constant intensity.
That's not to detract from Kalal's research - this is really good work - I just want to point out that it very likely suffers from a few achilles heals not mentioned in his video.
Now pair this method with the kinect, and you might see a real improvement.
Government is not reason; it is not eloquent; it is force. Like fire, it is a dangerous servant and a fearful master.
No - wrong on all counts.
- Kinect doesn't have stereo cameras (it has an IR camera for depth perception and a visible light camera for other usage)
- Kinect doesn't use the visible light camera for body recognition. Recognition is based on the depth map provided by the IR grid projector and IR camera.
- Kinect doesn't operate like a laser rangefinder (it operates via structured light displacements, not via light pulse reflection times)
- Kinect doesn't track a wireframe (it tracks independent body parts)
How you got modded as "4 - informative" is beyond me. The blind leading the blind.
The way Kinect works is by projecting a dense evenly-spaced grid of IR dots (i.e. structured light) on the scene, then using it's IR camera (horizontally offset from the grid projector) to pick up the reflected dot pattern.
Due to depth differences in the scene, and the offset of the IR camera from the IR projector, the reflected dot pattern is not evenly spaced - the dots are horizontally displaced based on depth. To understand this, consider shining two parallel beams of light at a) a flat surface, and b) a surface angled at 45 degrees away from the light source. If you took a step sideways away from the light beams and looked at their reflections of the two objects, the dot (beam) separation on the flat surface would be the same as the true beam separation, but the dot separation on the angled surface would be increased. by an amount you could calculate using simple trig.
In order to operate in real-time with low cost, a dedicated chip processes the IR camera image and converts the dot displacements into the corresponding depth map.
The clever, and somewhat counter-intuitive, part is how Kinect then turns this depth map into a body part map. The basic idea is that it probabilistically maps local clusters of depths to body parts (via having been trained on a huge manually body-part-labelled image set), then converts these local probabilities into larger scale body part labels (i.e. if 60% of the local clusters in a region say "hand" , then the region is labelled as a hand). This way it doesn't track overall body postion or a wireframe, but rather independently tracks body parts (which is why it has no trouble correctly tracking muliple partially occluded people in frame).