Slashdot Mirror
Automatic 3D Reconstruction of Scenes
Neil Halelamien writes "New Scientist reports on a piece of software by MDRobotics called instant Scene modeler (iSM), which automatically generates 3D reconstructions of scenes, using a few hundred frames from a pair of ordinary video cameras. The software uses David Lowe's SIFT vision algorithm to quickly locate common features between sequential images, for use in the reconstruction; SIFT has also been useful for generating panoramas and object recognition. MDRobotics has a demo page showing the software being used for crime scene reconstruction, along with animated GIFs of input video and the resulting 3D model."
CSI: Crime Scene Reconstruction
In C++, friends can touch each others private parts.
Finally we can do what Harrison Ford can do... :-)
Wow I bet it's just like in CSI they will be able to zoom up 10000% on the digital image, 'sharpen' it and read all sorts of interesting things off the back of things after rotating them virtually.
Since slashdot will probably burn out the web server hosting images: http://www.mdrobotics.ca.nyud.net:8090/ism/behind. htm
--
Free iPod? Try a free Mac Mini
Or a free Nintendo DS, GC, PS2, Xbox
Wired article as proof
Educationally, people could truly "walk around" in a virtual museum. This is lightyears ahead of QuickTime VR(?) where one simply can rotate about one point and zoom in or out.
It's only going to work on stationary scenes, as that sleeping fellow showed us. Basically, anything from the Real World you want to "import" into VR will be much easier to do.
If anyone likes FPS, you could model a map based on real scenery.
Most inventions and technology came into being before people found a use for it. It just seems pretty darn cool if nothing else.
"The first rule of intelligent tinkering is to save all the pieces." --Aldo Leopold (Paraphrased)
This reminds me of the bit in Enemy of the State where the government operatives take the lingerie store security footage, and then use their computer to "rotate the camer 90 degrees." And on top of that, they then see something in the bag that wouldn't have been at all detectable from the actual camera's angle.
It's pretty silly to suppose that this thing will be able to generate a 3-D representation of a scene without without getting highly-detailed footage of everything from every angle. Otherwise, it would just be a completely bogus modelling system to pull a fast one on people who don't know any better. "Ladies and gentlemen of the jury, although the original crime scene evidence photos don't show it, when you look rotate the angle and look at the far side of the desk, the defendent's fingerprints are clearly visible"
until they start using this technology for porn
The problem with slashdot is that most of its users were bullied and stuffed into lockers as kids!
Here, and here. Lin/Win/Mac versions.
From one of the links: The SIFT algorithm is restricted by patents in the United States and hence this software is not completely free to use. For details see the LICENSE file included in the distribution, before you start to use this software.
Hopefully, they're liberal about the patent and will let noncommercial nonresearch applications use the algorithm. Otherwise, we would have to wait for the really interesting software to come out.
A C# implementation with support for Mono is available to play with for anyone who is interested: http://user.cs.tu-berlin.de/~nowozin/libsift/
--
Free iPod? Try a free Mac Mini
Or a free Nintendo DS, GC, PS2, Xbox
Wired article as proof
I have been waiting for the results for quite some time and they surely look impressive. I might add that the underlying concept is not very hard to understand and one could even make a simple 3-D model of distant objects (like e.g. buildings in your city) using only two eyes, paper, pencil and some basic trigonometry.
/|
Look at this model:
A---B
|\
| C |
|/ \|
D---E
Where D and E are your two eyes, two cameras, or two positions from which you look at the object C that appears to be eclipsing A and B respectively. The distance between any of those points and their relative 3-D positions can be calculated when you know some of the distances (e.g. DE and AD) with very high precision.
Recommended Wikipædia reading for anyone interested: Parallax, Triangulation, Stationary point, Pythagorean theorem, Euclidean geometry, Astrometry, Binocular vision, Stereoscopy. Have fun.
Sincerely,
Pan Tarhei Hosé, PhD.
"Homo sum et cogito ergo odi profanum vulgus et libido."
Olympus has had this functionality in their Camedia cameras for a few years now. The panorama stitching is actually pretty easy. You put it into pano mode, and the camera then uses the initial exposure/shutter setting for the entire series, and puts little bars on the sides of the LCD to tell you were to do the overlap. The (otherwise crappy) Camedia Master software recognizes that it is a series, and stitches pretty quickly and seamlessly. That function only works with original Olympus SM cards, though.
Autopano and Autopano-sift. I have't had good experiences with the SIFT-based software. They always tend to pick the most inappropriate points, like trees/leaves (that move between shots) and the middle of objects (where there aren't many features). I almost always have to go through and remove the bad points, adding in my own reliable ones (corners, unique features, etc). I just don't use them anymore because I actually spend less time if I do all the points myself manually. The GUI of Hugin usually saves me plenty of time already. It does a good job of picking the matching point when I click on one photo. That's all I need anymore.
Though the technology would need some additional improvements, it might be interesting to apply it to tracking shots in old movies (like Casablanca) and in addition to reconstructing the sets one could also replay a scene from a slighly different angle.
The other slight modification would be to combine the possible modification (getting a slightly different angle from an existing tracking shot) and build a stereo 3D image of the shot or film segment.
Buy Steampunk Clothing Online!
For goodness sake man, that's your first though?!
Don't you see? This could be used for porn!
For the betterment of all mankind, it must be done!
Curiosity was framed. Ignorance killed the cat.
This is not limited to static scenes as one comment says. It could be used to reconstruct moving objects just as well, with a bit more software.
You could very accurately construct physical models of criminals from security tapes.
You could also construct an accurate model of how they walk. Since every person has a unique walk, that would be more difficult to disguise than physical appearance alone.
You could discover identifying details of the cars they drive, like a small dent in the fender.
This would be perfect for eBay, you could send them a short film of the object you're selling and they would post a 3D model of the item.
This heralds the end of both motion capture and the existing hours long '3D scanning' of clay models used in films like LoTR. Instead of requiring a mechanical stylus to touch every point of a model, you just film it.
Once the software has the ability to turn multiple 2D viewpoints into a single 3D image, this will be the perfect replacement for VR gloves as well. You could have a cameras on either shoulder and your hands would be your 3D mice. That sounds like a nicely intuitive interface.
Moving companies could find this useful. They could film the objects, the moving truck, and in return get an optimal packing order. You could also film the stairway up to an apartment and the software could figure out how to get through any of the particularly tight spots, if it's possible.
This would be good for the sort of augmented reality that washington.edu has researched. When the software can regognize the separate parts in a machine it can display directions for disassembly on a heads up display.
Oh, I can think of lots more uses, but better to get hold of the code and try to implement some of the random ideas above.
Shae Erisson - ScannedInAvian.com
I just hope they patent it before anyone else can use it. I'd hate to see them give up their ability to innovate.
Can't wait til i can stroll thru my old high school with a pair of cameras, take that info, load it into the computer, algorithmize it, and load that data into Splinter Cell / Halo / Unreal Tournament. Ah man, think of all the cool new maps that'd be popping up every day if this system ever gets widespread!