Slashdot Mirror
Seeing Around Corners With Dual Photography
An anonymous reader writes "This project (which is part of this year's SIGGRAPH) has absolutely blown my mind. Basically they photograph an object with the photosensor at one point, and the light projector at another, and use the Helmholtz reciprocity algorithm to virtually switch the locations of the camera and projector, showing exactly what the light source "sees"! If that doesn't make sense to you, check out the research page and make sure to watch the 60MB video at the bottom. The playing card trick will leave you speechless!"
Dual Photography
Abstract
We present a novel photographic technique called dual photography, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene. With a video projector providing structured illumination, reciprocity permits us to generate pictures from the viewpoint of the projector, even though no camera was present at that location. The technique is completely image-based, requiring no knowledge of scene geometry or surface properties, and by its nature automatically includes all transport paths, including shadows, interreflections and caustics. In its simplest form, the technique can be used to take photographs without a camera; we demonstrate this by capturing a photograph using a projector and a photo-resistor. If the photo-resistor is replaced by a camera, we can produce a 4D dataset that allows for relighting with 2D incident illumination. Using an array of cameras we can produce a 6D slice of the 8D reflectance field that allows for relighting with arbitrary light fields. Since an array of cameras can operate in parallel without interference, whereas an array of light sources cannot, dual photography is fundamentally a more efficient way to capture such a 6D dataset than a system based on multiple projectors and one camera. As an example, we show how dual photography can be used to capture and relight scenes.
(a) Conventional photograph of a scene, illuminated by a projector with all its pixels turned on. (b) After measuring the light transport between the projector and the camera using structured illumination, our technique is able to synthesize a photorealistic image from the point of view of the projector. This image has the resolution of the projector and is illuminated by a light source at the position of the camera. The technique can capture subtle illumination effects such as caustics and self-shadowing. Note, for example, how the glass bottle in the primal image (a) appears as the caustic in the dual image (b) and vice-versa. Because we have determined the complete light transport between the projector and camera, it is easy to relight the dual image using a synthetic light source (c) or a light modified by a matte captured later by the same camera (d).
Seeing around corners is really stretching it. You switch positions with the light source, so you can technically look at the scene from a point which is "around a corner". What they so casually mention as "structured lighting" is really the key to the whole algorithm and means that the light source shines a pattern on the scene which then allows the camera to retrace where every bit of light it sees is coming from. This means that the light source needs to be part of the scheme. You won't be able to switch yourself into the position of arbitrary lights on the street.
Only the first part for now :
a rt1.mp4.torrent
http://dload.digitalriviera.com/DualPhotography-p
Second part in 30 minutes !
First torrent I host, I hope it's ok.
Another mirror here. No guarantees as to how long it will stay up; if it pushes me close to my monthly bandwidth limit I'll kill it...
How far could you get with all the information escaping the window in your direction?
It's called optical tempest. With a high enough sampling rate you can reconstruct what is being shown on the monitor/TV. Each pixel as it illuminates causes a brief spike in the ambient brightness; by measuring this spike one can reconstruct the pixels being shown. After that, it's pretty simple to find the horizontal and vertical retraces.
more info
I am disrespectful to dirt! Can you see that I am serious?!