Slashdot Mirror
Automatic 3D Reconstruction of Scenes
Neil Halelamien writes "New Scientist reports on a piece of software by MDRobotics called instant Scene modeler (iSM), which automatically generates 3D reconstructions of scenes, using a few hundred frames from a pair of ordinary video cameras. The software uses David Lowe's SIFT vision algorithm to quickly locate common features between sequential images, for use in the reconstruction; SIFT has also been useful for generating panoramas and object recognition. MDRobotics has a demo page showing the software being used for crime scene reconstruction, along with animated GIFs of input video and the resulting 3D model."
CSI: Crime Scene Reconstruction
In C++, friends can touch each others private parts.
Finally we can do what Harrison Ford can do... :-)
Wow I bet it's just like in CSI they will be able to zoom up 10000% on the digital image, 'sharpen' it and read all sorts of interesting things off the back of things after rotating them virtually.
I love taking panoramic shots with my camera, but the stinkiest part has always been stitching all the images back together. I haven't seen any package like this before...too bad its not open source :)
Since slashdot will probably burn out the web server hosting images: http://www.mdrobotics.ca.nyud.net:8090/ism/behind. htm
--
Free iPod? Try a free Mac Mini
Or a free Nintendo DS, GC, PS2, Xbox
Wired article as proof
Yes, I would love to see an open source implementation of this program. Does anyone know of anything like this, or similar software?
The sending of this message pretty much inconveniences everyone involved.
Educationally, people could truly "walk around" in a virtual museum. This is lightyears ahead of QuickTime VR(?) where one simply can rotate about one point and zoom in or out.
It's only going to work on stationary scenes, as that sleeping fellow showed us. Basically, anything from the Real World you want to "import" into VR will be much easier to do.
If anyone likes FPS, you could model a map based on real scenery.
Most inventions and technology came into being before people found a use for it. It just seems pretty darn cool if nothing else.
"The first rule of intelligent tinkering is to save all the pieces." --Aldo Leopold (Paraphrased)
This reminds me of the bit in Enemy of the State where the government operatives take the lingerie store security footage, and then use their computer to "rotate the camer 90 degrees." And on top of that, they then see something in the bag that wouldn't have been at all detectable from the actual camera's angle.
It's pretty silly to suppose that this thing will be able to generate a 3-D representation of a scene without without getting highly-detailed footage of everything from every angle. Otherwise, it would just be a completely bogus modelling system to pull a fast one on people who don't know any better. "Ladies and gentlemen of the jury, although the original crime scene evidence photos don't show it, when you look rotate the angle and look at the far side of the desk, the defendent's fingerprints are clearly visible"
until they start using this technology for porn
The problem with slashdot is that most of its users were bullied and stuffed into lockers as kids!
From one of the links: The SIFT algorithm is restricted by patents in the United States and hence this software is not completely free to use. For details see the LICENSE file included in the distribution, before you start to use this software.
Hopefully, they're liberal about the patent and will let noncommercial nonresearch applications use the algorithm. Otherwise, we would have to wait for the really interesting software to come out.
A C# implementation with support for Mono is available to play with for anyone who is interested: http://user.cs.tu-berlin.de/~nowozin/libsift/
--
Free iPod? Try a free Mac Mini
Or a free Nintendo DS, GC, PS2, Xbox
Wired article as proof
I have been waiting for the results for quite some time and they surely look impressive. I might add that the underlying concept is not very hard to understand and one could even make a simple 3-D model of distant objects (like e.g. buildings in your city) using only two eyes, paper, pencil and some basic trigonometry.
/|
Look at this model:
A---B
|\
| C |
|/ \|
D---E
Where D and E are your two eyes, two cameras, or two positions from which you look at the object C that appears to be eclipsing A and B respectively. The distance between any of those points and their relative 3-D positions can be calculated when you know some of the distances (e.g. DE and AD) with very high precision.
Recommended Wikipædia reading for anyone interested: Parallax, Triangulation, Stationary point, Pythagorean theorem, Euclidean geometry, Astrometry, Binocular vision, Stereoscopy. Have fun.
Sincerely,
Pan Tarhei Hosé, PhD.
"Homo sum et cogito ergo odi profanum vulgus et libido."
I can see this sort of thing being useful for space exploration.
I might know what I'm talkin' about, but then again, this is Slashdot...
Though the technology would need some additional improvements, it might be interesting to apply it to tracking shots in old movies (like Casablanca) and in addition to reconstructing the sets one could also replay a scene from a slighly different angle.
The other slight modification would be to combine the possible modification (getting a slightly different angle from an existing tracking shot) and build a stereo 3D image of the shot or film segment.
Buy Steampunk Clothing Online!
For goodness sake man, that's your first though?!
Don't you see? This could be used for porn!
For the betterment of all mankind, it must be done!
Curiosity was framed. Ignorance killed the cat.
This is not limited to static scenes as one comment says. It could be used to reconstruct moving objects just as well, with a bit more software.
You could very accurately construct physical models of criminals from security tapes.
You could also construct an accurate model of how they walk. Since every person has a unique walk, that would be more difficult to disguise than physical appearance alone.
You could discover identifying details of the cars they drive, like a small dent in the fender.
This would be perfect for eBay, you could send them a short film of the object you're selling and they would post a 3D model of the item.
This heralds the end of both motion capture and the existing hours long '3D scanning' of clay models used in films like LoTR. Instead of requiring a mechanical stylus to touch every point of a model, you just film it.
Once the software has the ability to turn multiple 2D viewpoints into a single 3D image, this will be the perfect replacement for VR gloves as well. You could have a cameras on either shoulder and your hands would be your 3D mice. That sounds like a nicely intuitive interface.
Moving companies could find this useful. They could film the objects, the moving truck, and in return get an optimal packing order. You could also film the stairway up to an apartment and the software could figure out how to get through any of the particularly tight spots, if it's possible.
This would be good for the sort of augmented reality that washington.edu has researched. When the software can regognize the separate parts in a machine it can display directions for disassembly on a heads up display.
Oh, I can think of lots more uses, but better to get hold of the code and try to implement some of the random ideas above.
Shae Erisson - ScannedInAvian.com
I just hope they patent it before anyone else can use it. I'd hate to see them give up their ability to innovate.
Two time-separated frames in a video and sufficiently intelligent software is a stereoscopic camera.
Shae Erisson - ScannedInAvian.com
Can't wait til i can stroll thru my old high school with a pair of cameras, take that info, load it into the computer, algorithmize it, and load that data into Splinter Cell / Halo / Unreal Tournament. Ah man, think of all the cool new maps that'd be popping up every day if this system ever gets widespread!
That is the third time an article on Slashdot has described something I had hoped I could do my masters thesis on.
First was the decentralized bittorrent network.
Then was the method by which angle changes in camera shots can be deduced by comparing the images (the jigsaw puzzle solver). That was probably an infantile persuit anyway.
And now this.
I will go ahead and spill the next idea along the same lines as the last two projects:
What if you were to put a light collector that could detect angle and intensity of light on top of a camera. Then, when your camera is filming your scene you are also recording the manner in which ambient light is reaching your scene. Later, using you 3D scene reconstruction, you can throw in new objects such as creatures and whatnot and use the data from you light collector to apply correct lighting to the new objects you introduce.
I would imagine someone is about to release this technology and we will see an article about it on Slashdot in a couple weeks. Look forward to it.
Direct away from face when opening.
So they are not as much editors as they are approvers.
Wow, I actually tried out that autostitch program. It works extremely well. For carefully shot pictures, it will stitch more or less perfectly. For wrecklessly shot, less than perfect but much better than what I can do with Canon photostitch. Considering it was automatically stitching stuff better than what I was doing with Canon photostitch with a lot of manual tweaking, it's impressive. I hope this guy's development work becomes a commercial product.