Slashdot Mirror
Making 3D Models from Video Clips
BoingBoing is covering an interesting piece of software called VideoTrace that allows you to easily create 3D models from the images in video clips. "The user interacts with VideoTrace by tracing the shape of the object to be modeled over one or more frames of the video. By interpreting the sketch drawn by the user in light of 3D information obtained from computer vision techniques, a small number of simple 2D interactions can be used to generate a realistic 3D model."
wow, what a terrible link.
A quick search turns up the project homepage http://www.acvt.com.au/research/videotrace/
Software like Canoma from the now-defunct Metacreations would let you create 3D models from 2D images in the mid-to-late 90s. I also remember reading about people using Viz ImageModeler to convert images from video to models even though the software is also designed for still images - the users would just capture those frames they needed to create the 3D model.
The only thing "new" about this is using video as the input without having to grab the individual frames yourself.
Never let reality temper imagination
Never let reality temper imagination
Have you heard of the Scale Invariant Feature Transform? Well you have now. There are libraries written in C# (no less) which are publicly available to do this stuff. You can recognize a large collection of objects.
How we know is more important than what we know.
I'm a Ph.D. student at UC Santa Cruz. I finished my masters a few years ago working on enhancements to a project with similar goals. My advisor, Jane Wilhelms (who unfortunately died shortly after I finished my masters) was working on computer vision techniques for several years. Her work focused on extracting motion for animals (often children or horses) out of videos. My Masters contribution was to look at how the accuracy and usability of the software could be improved if we assume that the general motion of a walk is the same for all instances of a particular species (the knees all bend the same way, and the legs move in the same order, etc). I didn't have a high quality capture to start with, so the results were a bit fuzzy in terms of accuracy, but it did make the process easier for the user. The user had only to make the "original" motion match the video at key frames (maybe 4 per "walk cycle"), and the computer could easily interpret the rest; I don't recall off the top of my head, but I think the number of key frames the user had to specify was reduced by half or more over the former process (without the canonical motion as a starting point). I didn't publish any papers based on my work, but my masters thesis (with example filmstrips) is available.
SIFT is ok even for occluded objects, but is horrid in 3-d because SIFT features cannot match up for a significantly rotated scene. There are better algorithms that can recover both the shape of the scene as in the article and even produce the location of the camera as a by-product.
In terms of object recognition, there has been great work done by treating an "nxn" pixel image as a point in n^2 space, and then reducing the computation space and projecting a given image onto that new, lower-dimensional approximation of the original object, and finding a match via a nearest-neighbor search through recognized objects.
There is also good work being done in terms of getting a detailed 3-d model using structured light methods: http://www.prip.tuwien.ac.at/research/research-areas/3d-vision/structured-light
There is good literature out there, but sometimes the math gets over my head =P
http://www.youtube.com/watch?v=vda2RAEuW_g
The same is true for image recognition. You can get a computer to recognize movement pretty easily. Heck, the ability for software to detect the 3d form of an object has been around for ages. However, getting a computer to watch Star Wars and say "I see Dennis Lawson sitting inside an X-Wing fighter." is, as I said before, difficult to do without a concept of 'experience'.
We'll get there one of these days, but right now the sorts of cool-sounding advancements we've been seeing really only work in very specific circumstances.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I get that a lot. Blind people still have a 3d imagination. They need to know where the doors are, where the stairs are, and where objects they use are. You need a 3d imagination space to have AI and that is the primary reason that past attempts at making AI have failed. I love to watch the advances in video card technology and the competition between NVIDIA and ATI because the more they work, the easier it will be to do AI, and all computer advances for that matter. I think I could start some basic AI with this 3d recognition software with the hardware of an average modern desktop. I think it is just a software problem and not necessarily a hardware one. We'll see. I'm going to keep in touch with this group and see if they let me use their software because I'm an unemployed coder and I might as well work on AI because some group has to do it. I'll make it an open source project in Source Forge and maybe extra coders will jump on.
God spoke to me.
...and no one is going to make a porn joke?