Photosynth Team Does It Again

STFS found an update to the Photosynth stories that we already ran. You might remember the amazing photo tourism demos. Well, this new version kicks things up several notches with paths and color correction to more smoothly transition between photos taken in different lighting conditions. As before, this stuff is worth your time. Check it out.

Comments

fascinating

[thedonger]

Science fiction and VR have primed me to believe someday we would all be walking around some imaginary digital world (oh wait, WoW anyone?), but this is "virtualization" of the real world. Like Google street view on crack. I am simultaneously in awe of the technological achievement and embarrassed that my life in computers hasn't yet created anything so cool.

I, for one, welcome our new PhotoSynth overlords.

Re:Huh

[BitterOldGUy]

Shut The Fuckup Sonny - it's what the old guys on Photo.net say when you tell them that photography as we know it is dead - especially if you mention film in the same sentence. It'd be like saying that BSD is dead here on /..

Video

[c_g_hills]

Obligatory link to the youtube video (not a rickroll, I promise!)

Thanks, Network Mirror!

Re:I'm confused by all this

[Anonymous Coward]

It needs neither input of coordinates or input of a rough 3d layout. It generates its own 3d model by analyzing the photographs programatically, you do not even need to tell the program they were taking in the same area. The photographs are then automatically applied to the generated 3d model and finally it lets you move freely in the generated 3d world selecting the best photo matching your current viewpoint while applying perspective remapping, color correction and lens correction.

Re:I'm confused by all this

[hkz]

From what I took away from the original demo, they were doing everything algorithmically. The original demo showed a wireframe of the Notre Dame generated completely from amateur pictures, then overlaid with those same pictures to give it texture. So yes, it is quite impressive. I'd be surprised if Google wasn't doing anything similar for Google Maps though.

Re:I'm confused by all this

[dave420]

Because you can, say, search Flickr for a landmark, get the images, run them through this, then you can navigate through the space in 3D, looking at high-res imagery of the subject, from all available angles, without having to previously know anything about the subject. Even the system doesn't need to know anything about the subject, it only needs the photos. It is ENTIRELY automatic, using only images. If you look at the old Notre Dame demo, you can see that it even correctly inserted a photo of a poster of Notre Dame into the 3D model, in the exactly-correct position. 100% automatic. That's the breakthrough.

Re:No sense to limit how many photos you take...

[fastest fascist]

Geotagging is not that hard nowadays, assuming you have a device capable of creating a gps log and are using a digital camera (timestamps). Take the log, load it into gpicsync and let the program tag your photos for you. Just make sure the gps device and your camera have their clocks synchronized. I'm still waiting for a decent way to browse photos on a map, though - pretty much what you're looking forward to, I guess. Picasa lets you view geotagged photos in albums in google earth, but it's not much more than a gimmick. Something that would let you draw a circle on a map to limit your search to that area, then use a time slider to further refine the search and then allow keywords, face recognition once we get that, would be useful.

Re:No sense to limit how many photos you take...

[ka9dgx]

Because a video camera is nowhere near the quality of a still image, still cameras will win for a number of reasons:

• Still Camera - less motion blur, if any
• Bigger sensor - less noise
• Focus mechanism - an SLR has a much better focusing mechanism
• Image Compression - almost all video codecs record a stream of images, and do not optimize the quality of an individual frame
• Exposure time - A still camera can take from 1/8000 second to 5 minute exposures for a single frame, as opposed to a fixed time of about 1/30 second for NTSC
• Aperture - A still camera can control the aperture to get desired depth of field

So, those are the ones I can think of off the top of my head.

Re: Exposure time

[DamnStupidElf]

NTSC is worse than you described; you have two 1/60th second exposures interlaced together. Utterly worthless for still frames.

Once progressive HD video cameras become cheap, then video will suck slightly less for the average family archive.

Re:No sense to limit how many photos you take...

[QuestionsNotAnswers]

If a poor photographer uses a cheap video camera - what can be acheived using image processing? In theory signal processing of a long enough video could:

• Remove blur using deconvolution - multiple frames can be used to reduce deconvolution noise sensitivity and multiple frames help calculate point spread functions. Of couse motion blur is also avoided in video cameras by using Image Stabilisation (adaptive optics are becoming common even in low end cameras - eg. TX-1).
• Multiple frames can be used to remove most forms of noise (and sensor noise in particular), remove specular reflections, remove obscuring objects (if can see around obstruction using different POVs), perhaps even help with removal of some shadows (maybe by moving point lighting while videoing).
• With enough frames and not far out of focus, DOF can be increased. Also normal focus hunting, or by using say CHDK to continually vary focus, can synthetically increase DOF in post processing.
• With varying F-stop post processing of a video stream could recover a higher dynamic range.
• Aperture - I don't understand your point - video cameras have aperture control too.
• Long exposure times - can be simulated via integration of multiple images - don't digital cameras do this internally anyway? Certainly the CHDK firmware does this to simulate exposure times over 65 seconds. Short exposure times can obviously be done with a suitable video camera.
• Compression artifacts - yeah - a real problem.

Image processing is getting better - and I am definitely keen on using software to process videos to synthetically generate high definition images. My time and dollar resources are limited for getting the perfect photo.

Re:Wow

[complete loony]

There are other features that I don't see how they're getting, such as the zones where photos were shot from. That takes an awful lot of extrapolation. What's the difference between a photographer 10 feet away, and a photographer 200 feet away with a good zoom lens? Almost nothing, except maybe a little focal distortion at the edge of the photo. That varies with the quality of the camera and lens anyways.

Perspective changes a lot based on where the camera is, a big zoom lense does nothing to change the perspective it just makes the image larger.

Their process finds machine recognisable points in each photo, then looks for matching points between photos. Once you know that 2 photos are of the same subject you can use the separation between these known points to work out the relative viewing position of each camera. It only takes about 4-5 common points on different planes to pinpoint where each camera is relative to other camera's. I can visualise how this process could be completely automated.

At the end of the process they have a 3D model of where all these identifiable points are relative to each other, and they know where to project the plane of each photo within that model.