Recreating Cities Using Online Photos

Roland Piquepaille writes "The billion of images available from a site like Flickr has stimulated the imagination of many researchers. After designing tools using Flickr to edit your photos, another team at the University of Washington (UW) is using our vacation photos to create 3D models of world landmarks. But recreating original scenes is challenging because all the photos we put on Flickr and similar sites don't exhibit the same quality. With such a large number of pictures available, the researchers have been able to reconstruct with great accuracy virtual 3D model of landmarks, including Notre Dame Cathedral in Paris and the Statue of Liberty in New York City."

Comments

Re:seen this

[Sectrish]

Yep, it was Photosynth, a cool looking project from a company bought by microsoft. It's site is: http://labs.live.com/Photosynth.aspx

How about telling us what it's called?

[BertieBaggio]

This story is a bit old (well, it's from Roland after all). There was a demo of this tech by Blaise Aguera y Arcas at TED earlier this year. the two underlying components are Seadragon and Photosynth, both of which are mighty impressive. Also, despite the Mozilla-esque name 'Seadragon', both of these technologies are actually owned by Microsoft. There is a tech preview of Photosynth up for download, but I don't think Seadragon is available yet.

There is a video of the TED demo, which shows off some of the things Seadragon and Photosynth can do, the including Notre Dame example mentioned in T(second)FA. The talk is also on YouTube.

Re:How about telling us what it's called?

[Aaron England]

Is Photosynth "owned" by Microsoft? I thought Microsoft only acquired Seadragon and that Photosynth was a joint project between UW and Microsoft.

Re:How about telling us what it's called?

[BertieBaggio]

Mod parent up. I thought MS had bought both, but Photosynth is collaborative research by Noah Snavely (undergrad at UW), Steve Seitz (also UW), and Richard Szeliski (from Microsoft Research). Maybe in six months time I'll be retroactively correct when they do buy it...

(from http://labs.live.com/photosynth/ and http://labs.live.com/photosynth/aboutus.html)

Re:How about telling us what it's called?

[nomis80]

You are confusing two things. This new paper adds 3D modeling. Now you don't just know the relative pose of pictures, you can also use them to compute the 3-D shape of things in those pictures.

Great accuracy?

[DrXym]

It looks like an incredible idea with a lot of promise, but the shots they've shown so far look like some lumpy rejects from a plastic vacuum forming machine. There is some great potential here to involve people to generate better models by asking them to contribute pics of certain monuments with certain characteristics such as resolution, position and so on.

Re:Great accuracy?

[Anonymous Coward]

Imagine a 3D recreation of Goatse. The traditional Goatse photograph is one in a set, of course. Maybe they could even locate the hidden turds that aren't visible in the traditional photo.

Terrorists

[nurb432]

Only a terrorist would want a 3d map of a city. This must be stopped.

Re:Terrorists

[nurb432]

I was actually trying to be sarcastic, but i got modded insightful??

This just in...

[It doesn't come easy]

Online photos of any physical location on any planet in the local galactic group are now forbidden due to the possibility that the photos might be used by terrorists or those who may be helping terrorists to plan terrorist attacks on said locations.

By order of Ultra Super Secret Chief Intelligence Officer, Department of Homeland Security

Home use

[Cillian]

I wonder how many photos it needs of an object to produce a decent model and if the software will ever be released to the public - it would be amazing to be able to produce a 3d model of an object just by snapping a bunch of photos from different angles and bunging them into a piece of software. (I know there are things to create a model using a video camera and a turntable, but that's not quite as easy as being able to grab your camera anywhere and make a model.)

Re:Home use

[Chris Pimlott]

The SIFT algorithm created by David Lowe, might work for that; it's commonly used in panorama stitching software to automatically determine the common points between two overlapping photos. Provided you had only a small amount of rotation between photos, it could probably give you some good results.

Nope, not photosynth...

[tommyhj]

This has nothing to do with the Photosynth/seadragon project, other than the fact that both teams use a whole bunch of different photos of the same object/setting.

This approach tries to make the actual 3D objects from a bunch of 2D photos of varying quality. Photosynth just tried to MAP the photos in a rough 3D space. Making the actual 3D model to any degree of accuracy is really a challenge when you can't control the input images.

The goal is different in the two cases, but they should definitely get together and exchange technologies and algorithms, because I SO much want this tech built into Google Earth!!

Does it work with people?

[TheVelvetFlamebait]

If so, I think it's time to rent a few Jessica Alba movies.

Re:Does it work with people?

[BlueParrot]

If so, I think it's time to rent a few Jessica Alba movies.

You may also want some of the CSI image analysis software which support zooming beyond the image resolution. ; )

Re:Does it work with people?

[Squalish]

We certainly have software that can take a video of static objects and turn it into a 3d scene.

We have, in TFA, software that can take frames of static objects, remove the dynamic objects among them, and leave us with a 3d scene.

We probably have software that can interpolate a static object which is bounding a nonstatic/elastic layer (the shape of a statue under a swaying tarp, the dimensions of a box inside a grocery bag someone is swinging).

We probably do not, however, have software that can efficiently calculate the at-rest dimensions of an elastic, mobile object(Jessica Alba) beneath a nonstatic/elastic layer (clothes). We've just barely reached the point where we can depict the behavior of the squishy, bony, muscular, hairy human body accurately, much less interpolate a hidden body.

One wonders what it would cost to develop such software to the satisfaction of a pervert, compared to what it would cost to simply fund a movie where the pervert gets to do this.

Re:Does it work with people?

[Chris Pimlott]

We probably do not, however, have software that can efficiently calculate the at-rest dimensions of an elastic, mobile object(Jessica Alba) beneath a nonstatic/elastic layer (clothes). We've just barely reached the point where we can depict the behavior of the squishy, bony, muscular, hairy human body accurately, much less interpolate a hidden body.

So what you're saying is we need naked pictures of Jessica Alba... for science.

Warsaw

[flyingfsck]

After the 2nd war, Warsaw was rebuilt from photos. If you visit Warsaw today, you'd think that it is an old city. In fact, it is all new, the Poles just like it that way.

Re:seen this

[WalterGR]

Yep, it was Photosynth, a cool looking project from a company bought by microsoft.

No. Photosynth is a collaboration between the University of Washington and Microsoft Research. See here.

I'll see your 3D city and raise you a D.

[gt384u]

There's a professor in my building who works on constructing spatio-temporal representations of information from 2D images that allow you to see a city evolve through time. Not sure if all their demos are on the website, but the ones I've seen are pretty ridiculous. http://www.cc.gatech.edu/4d-cities/dhtml/index.html

How many photos of the rear and sides?

[nick_davison]

You'll notice the Notre Dame model was of the endlessly photographed front aspect.

I wonder how much detail such a technique can pull on the rear of the building? Or the back of Rodin's Kiss? How about the top surfaces of the Statue of Liberty?

Of course that won't stop CSI, next season, from pulling a bunch of pictures from Flickr of the front exterior of a building, reconstructing a complete 3D model, open accurately hinging doors, travelling inside, going down in to the basement and looking at the reflection in someone's eye to identify the killer.

Differences with Photosynth

[snavely]

I was a co-author on this work -- it's great to see it on Slashdot =)

I also worked on the Photo Tourism project (which is related to Photosynth). There's a big difference between Photosynth and this new 3D reconstruction work, in that Photosynth takes a photo collection and reconstructs camera positions and a sparse point cloud (a set of disconnected 3D points floating in space), while in this new work we build *dense* 3D models of scenes (in the form of polygon meshes). Dense models are usually much better for use in applications like computer graphics, since they can be used to render scenes with much more photo-realism.

These two problems require different algorithms to solve---for this dense problem we use a technique called multi-view stereo, but we made it work with images taken by many different people under different conditions.

- Noah