Slashdot Mirror
Creating 3D Computer Graphics From 2D HDTV Camera
photon jockey writes: "Everyone knows that holograms are cool! But these three dimensional images are hard to make and need special conditions to view. A group from The University of Tokyo have taken a step toward 3D displays with this paper in Optics Express.
Using a HDTV camera they effectively capture the light rays passing through a plane from a lit scene and then reconstruct the three dimensional geometry of the scene. Some pretty movies are available from the same page to show this. The paralax is limited by the size of the CCD and the distance to the object.
From the paper: In the field of computer graphics (CG),the use of real images has been attracting
attention as a method of attaining image synthesis that results in a more photo-realistic
quality. This field of technology is called image-based rendering. The authors have
attempted to solve this problem, by applying the approach of 3-D display technology,
in which light rays are the most primitive elements of visual cues.
"
I am only dimly remembering this from an optics course at uni, but one way of understanding the process conceptually is that each of the point light sources of which the obect is composed generates its own Fresnel zone plate.
I can't remember the maths of the Fresnel zone plate, but basically, if you interfere a point light source with a parallel light source, take the resulting pattern of light and dark rings and etch it onto a surface, light reflected off that surface is diffracted in such a way that it is focused, as if it has passed through a lens with its focal point at the same distance behind the plate as the point source was in front. This creates a virtual image of the original point source. You can view the object as being composed of such light sources each generating their own zone plate.
Like I said, I can't remember the maths, and the extrapolation from one point source to a whole object isn't exactly mathematically trivial, but I remember that the reason the Fresnel zone plate worked was very straightforward to grasp conceptually and helped the holography concept to fall into place. I always preferred to have a way to visualise something like that even when the maths clearly works out fine.
The property of light you exploit when making of a hologram is called coherency, not polarization. That's why lasers are necessary, since polarized light could be also be generated with a bulb and a filter.
But it probably doesn't matter that much really, since the rest of your explanation also sounds a bit weird to me. I'm not an expert on holography, but AFAIK the trick is to capture not only the light intensity (as a photograph would) but also the phase information of the light (perhaps this is where you got that notion of polarization being necessary from). This is typically done by splitting a laser beam, and recording the interference patterns between the ray that hits the object and its undisturbed counterpart (something like that is only going to work with coherent light). On this interference pattern - which is captured on a special kind of film - a coherent lightsource can be used to reconstruct the entire phase information upon projection, which explains why a laser is also necessary to illuminate classical holograms.
This might not be a 100% correct description of the process, but it is probably more on the mark than your explanation. Then again, it might be not. Go dig up a physics book and check! :-)
Just my $0.2E-32
A.W.
Must be all that rock solid open source software.
This group is using the technique to extract depth using a single HDTV camera. That makes sense, although the approach is somewhat low-res. Depth extraction from stereo images is commercially available, and is an alternative to this approach.
This is image-based rendering. In image-based rendering, there is no 3D geometry, in the sense of vertices and polygons. What you have is an image of a subject from a number of angles, and you interpolate to get the angles in between. Remember Quicktime VR? (My graphics wonk credentials include unadvisably early adoption of QTVR.) It's the first reference in this article. Regular SIGGRAPH attendees will be used to having their jaws dropped by new image-based rendering advancements every year. This is definitely a field to watch, but what's presented here looks like an incremental advance more than a breakthrough. The article is a bit unclear, but it sounds like the new wrinkle here is that you use a set of microlenses to capture all the light coming through a plane...as if you had a solid wall of cameras. Then you have an output device which can take advantage of all the information in each of those source images (actually source video in this case). The point is a flat device which will give the illusion of 3D (including parallax) without an intervening plane or glasses. If I'm reading this correctly, it sounds like the effect will be very similar to a plane holograph, where you can walk around in front of the display and look at it from different angles, but if you go to far the image breaks, and of course you can't go behind it or rotate the object. So saying you can look at it from "arbitrary angles" is a bit thick. Anyway, this is a very separate thing from those systems that take a number of photographs and reconstruct the geometry, or those other systems that use range-finding lasers or the like to actually measure the geometry. In short, no application for Quake here, so it's surprising it got funded in the first place. :)
(Um...now watch them come out with image-rendered Quake...)
I can't agree with that. while the Campanile film is stunning, it seems to require a handcrafted 3d model onto which to wrap the multiple images.
This article seems to be a technique to recover 3d information from a large number of slightly offset pictures.
The paper shows an illustration in which the camera lens is covered by a large number of circular/fish-eyeish lenses in a grid pattern.
By integrating the slightly different view points of each one, the 3d information of the scene can be recovered. The Japanese researchers (and this is where I started skimming...) seem to have several heuristics for performing this transformation in realtime. The application is that HDTV sets equipped with appropriate decoding circutry could allow the user to rotate the scene very slightly at will. Of course, this comes at a significant resolution cost... almost so much so that I wonder if it wouldn't be more efficient to send 9 or so exponentially placed pictures in one (reducing resolution by 3) and using simple morphing to simulate free movement between them.
Just a thought.
After developing the film, if you illuminate the film with the laser light from the same angle and of the same wavelength as before, then a 3-dimensional image of the object is created within the plane.
The cool thing about the reconstructed image is that it is 'true' 3-d. That means, as you move your head from side to side as you are looking at it, you can actually see behind the objects in the image.
To understand how this works, I struggled through the mathematics of how the light beams pass through the developed film as well as how the original image exposes itself on the film. I don't understand it conceptually, but the mathematics involved nothing more than trig.
My personal disclaimer: I didn't read the article that the abstract refers to (no PDF viewer), but based upon the abstract, it sounds like they are using a techniques similar to what I described.
Keeping
Does this mean I can create an Unreal Tournament map of my house just by filming it?
----------------------------
-----------------------
Moderator's essentials
You might have seen it already, but a forerunner of this technique more suited to getting a 3D model of architecture is described at Paul Debevec's Home Page, with the famous Campanile film. It's pretty amazing what can be done - perhaps one day digitised actors can stroll around a digitised building, with various other additions made to it...
Ford Prefect
Tedious Bloggy Stuff - hooray?
It basically depends on polarization. All light is polarized, meaning that the electric wave and the magnetic wave that make up a photon are orthogonal (at right-angles within a plane) to each other. Most light is randomly polarized... that is, it bounces around at random with no structure to it. That's why lasers are commonly used in holography; it provides a polarized constant.
A traditional hologram is made by bouncing polarized light off an object (possibly from several different angles) and then exposing a piece of film to both the original, highly polarized light, and the light that is reflected off the object. When light is reflected, you change it's polarization to be (typically) parallel to the incident of reflection.
This makes miniature "grooves" in the image... they're virtual grooves, meaning they have no height, but all the same they selectively reflect only light of certain polarizations. Then, by shining the same type of polarized light on the exposed image, different angles of viewing select different polarizations, meaning different angles of viewing on the target object.
As for how this technology works, from what I can tell they're capturing the color and polarization of all the photons. This, combined with the width of the CCD, allows you to capture 3D information about the subject matter. If you were to add a source of polarized light to this thing, you could probably through the use of mirrors capture EVERY angle, just like a traditional hologram.
As a matter of fact, it doesn't even have to be visible light. Infrared will work fine, though you'll only get a rough gray-scale. But then, you don't need to be shining red/green/blue laser light around everywhere...
Won't the matrix people be mad at this! They spent Some Great Value Of Hard Earned Cash (SGVOHEC) to develop bullet-time, and now they can just use what turns out to be existing technology, making that expenditure of SGVOHEC a moot point.
Oh well, maybe the superbowl people will get with it next time so that my super-zooming rotating image of the QB won't jerk around like a 10-year-old computer trying to run quake...
I am disrespectful to dirt! Can you see that I am serious?!
Hrm.
I can see the 3d placement along the 2d camera path -- basically, you are deriving how the virtual camera should move and then blue screening that onto the real footage.
But: are you able to deduce accurate environment maps? Occulusion of the 3d virutal elements by real world items? This would seem to require true scene intepretation, while just deducing a virtual camera path from the real world footage sounds much more doable. Do you also deal with zooming and depth of field?
See an example of some raw footage of a canyon.
Then see the 3D model of the canyon he recovered from it.
Seastead this.
reconstruct the three dimensional geometry of the scene
meant what it said.
Seastead this.
First we're squishing cats into jars, and now dogs between panes of glass, my god people, what are we doing to our pets?
Sneakemail is to spam filters what an ounce of prevention is to a pound of cure.
I dont understand how they can make a 3d model out of a 2d picture. It is hard to understand their page. I'm thinking that they catch the light rays, and can tell which light rays arrive at which time, letting it perceive depth. But how do they tell the difference from old light rays? And obviously, i dont know how you'd be able to see how something looks from behind when it doesnt show up on the on the 2d picture.
something tells me this will bring a revolution to the kinds of RSI that can be obtained from playing Quake. I wish thee all happy hunting.
People replying to my sig annoy me. That's why I change it all the time.
www.opticsexpress.com, huh? Sure, maybe you can tell the boss they are "Optics Express" but we all know it's reall "Optic Sex Press". Motto: "Where girls push themselves against your eyeballs"
--
Non-meta-modded "Overrated" mods are killing Slashdot
(Hey Ryan! Here's your proof!)
...pornographic videos were never the same.
Bryan R.
Bryan R.
The price of freedom is eternal vigilance, or $12.50 as seen on eBay.....