Automatic 3D Reconstruction of Scenes

Neil Halelamien writes "New Scientist reports on a piece of software by MDRobotics called instant Scene modeler (iSM), which automatically generates 3D reconstructions of scenes, using a few hundred frames from a pair of ordinary video cameras. The software uses David Lowe's SIFT vision algorithm to quickly locate common features between sequential images, for use in the reconstruction; SIFT has also been useful for generating panoramas and object recognition. MDRobotics has a demo page showing the software being used for crime scene reconstruction, along with animated GIFs of input video and the resulting 3D model."

Comments

Awesome!

[ylsul]

I love taking panoramic shots with my camera, but the stinkiest part has always been stitching all the images back together. I haven't seen any package like this before...too bad its not open source :)

Good Use for Importing Stationary Objects

[fcolari]

Educationally, people could truly "walk around" in a virtual museum. This is lightyears ahead of QuickTime VR(?) where one simply can rotate about one point and zoom in or out.

It's only going to work on stationary scenes, as that sleeping fellow showed us. Basically, anything from the Real World you want to "import" into VR will be much easier to do.

If anyone likes FPS, you could model a map based on real scenery.

Most inventions and technology came into being before people found a use for it. It just seems pretty darn cool if nothing else.

Re:Good Use for Importing Stationary Objects

[PxM]

Virtual museums are overrated. When VRML became The Next Big thing about 10 years ago, I looked into creating virtual museums, and we're still not at the tech level that we can pull them off. 2D objects look bad when they're distorted into a 3d projection on such a small screen. The best way to view virtual paintings is just as a normal bitmap on a large enough screen. 3D objects like vases and small sculptures do work well in VRML since you can rotate them and view them from any angle. Large (with respect to the virtual user) sculptures and architecture tend to look boring on a screen compared to the real thing and aren't worth it unless viewed in 3D googles to get a sense of the depth involved.

And the whole walking part in a virtual museum is completely worthless unless the museum's architecture is artistic by itself since walking is slower than the "teleportation" available with normal hyperlinks.

BTW, this technology doesn't really compare with QTVR since QTVR is an output system and this technology is an input system to generate 3D models. You would still need a 3D renderer with texture support to view the data produced from this system.

--
Free iPod? Try a free Mac Mini
Or a free Nintendo DS, GC, PS2, Xbox
Wired article as proof

Re:Good Use for Importing Stationary Objects

[fcolari]

I agree nothing beats the Real Thing. "Museums" was a poor choice of word. Any space a person cannot normally get to, whether it be because of security or fragility or locality, would be an ideal canidate to be "shared". The realtor idea was a good one. Of course, Once the 3D model is generated, and once it becomes something I view, then it becomes an output system to me. My space is not interesting enough to generate a 3d model...

Cool applications...

[catdevnull]

I can see this sort of thing being useful for space exploration.

Re:Blade Runner Scene Machine

[Anonymous Coward]

I thought the same thing. But on reflection, Decker only used 'reflection' from the flat, still image bouncing off of the mirror. It was a great scene, however. And in the imortal Bob Hope.... Thanks for the memories.

Set reconstruction from old movies

[Anonymous Coward]

Though the technology would need some additional improvements, it might be interesting to apply it to tracking shots in old movies (like Casablanca) and in addition to reconstructing the sets one could also replay a scene from a slighly different angle.

The other slight modification would be to combine the possible modification (getting a slightly different angle from an existing tracking shot) and build a stereo 3D image of the shot or film segment.

Re:Set reconstruction from old movies

[robbo]

The MDR product is a stereo camera (a PGR Bumblebee), making the reconstruction significantly easier (but not necessarily easy). Also, all the intrinsic camera parameters are constant, whereas they're dynamic in most movie footage (changing focal length, etc). So, the reconstruction from movies problem remains pretty hard. That said, lots of people are working on it...

Re:Set reconstruction from old movies

[adpowers]

Also, the sets probably changed quite a bit between shots. When they move the camera, they don't necessarily keep lighting the same, so you would get a whole different set of problems compounding each other. You are right about the focal length. Plus, a lot of times backgrounds are blurred (to put focus on the actors), so it would be hard reconstruct that.

fyi MD Robotics

[Anonymous Coward]

MD Robotics are the makers of the Canada Arm I & II.

For those that didn't know.

Special camera not needed.

[shapr]

Two time-separated frames in a video and sufficiently intelligent software is a stereoscopic camera.

Re:Coming this season...

[oscartheduck]

Actually, my first thought was "Let's run the Zapruder film through this thing!" Or we can hook it up with this and have some fun results flying across the torrents in no time.

Re:Damn.

[imroy]

Paul Debevec has probably already done something like that. He's done a lot of work with "image based rendering", including reconstructing scenes from photos and extracting "light probes" from photos of shiny spheres. He's got a lot of papers and demos on his home page there.
Sorry bud. I know how you feel! I've had similar experiences myself.

Autostich impressive

[foxalopex]

Wow, I actually tried out that autostitch program. It works extremely well. For carefully shot pictures, it will stitch more or less perfectly. For wrecklessly shot, less than perfect but much better than what I can do with Canon photostitch. Considering it was automatically stitching stuff better than what I was doing with Canon photostitch with a lot of manual tweaking, it's impressive. I hope this guy's development work becomes a commercial product.