Light Field Photography Is the New Path To 3-D

waderoush writes "In November, Lytro, the maker of the first light field camera for consumers, upgraded its viewer software to enable a feature called 'Perspective Shift.' In addition to refocusing pictures after they've been taken, Lytro audiences can now pivot between different virtual points of view, within a narrow baseline. This 3-D capability was baked into Lytro's technology from the start: 'The light field itself is inherently multidimensional [and] the 2-D refocusable picture that we launched with was just one way to represent that,' says Eric Cheng, Lytro's director of photography. But while Perspective Shift is currently little more than a novelty, the possibilities for future 3-D imaging are startling, especially as Lytro develops future devices with larger sensors — and therefore larger baselines, allowing more dramatic 3-D effects. Cheng says the company is already exploring future versions of its viewer software that would work on 3-D televisions. 'We are moving the power of photography from optics to computation,' he says. 'So when the public really demands 3-D content, we will be ready for it.'"

Comments

Bought a Lytro

[Anonymous Coward]

Returned it.

It was awful, and the resolution wasn't hot

Re:Bought a Lytro

[dfghjk]

An AC calling someone a piece of shit. What a surprise.

I don't recall claiming I was interested in technology, but the fact that I am means I have actually studied what Lytro does. That's why I understand that it will be a failure. I suggest you funnel your anger toward a big, fat investment in Lytro if you feel otherwise.

My first thought

[Anonymous Coward]

Enhance 224 to 176. Enhance, stop. Move in, stop. Pull out, track right, stop. Center in, pull back. Stop. Track 45 right. Stop. Center and stop. Enhance 34 to 36. Pan right and pull back. Stop. Enhance 34 to 46. Pull back. Wait a minute, go right, stop. Enhance 57 to 19. Track 45 left. Stop. Enhance 15 to 23. Give me a hard copy right there.

Lytro's 3-D is inherently limited

[levork]

Wow, TFA is really glossing over an inherent limitation:

the "shiftability" of a Lytro image is a function of the width of the image sensor

If the goal of this is to produce useful stereo content that replicates the parallax seen by humans, then the image sensor needs to be at least as big as the average distance between two human pupils. That's roughly six centimeters. The Lytro's sensor is around six millimeters. Somehow I doubt they're going to increase their form factor by ten times in each dimension, and since the point of a Lytro is to avoid fancy lenses they can't bend the light path to compensate.

Re:Lytro's 3-D is inherently limited

[dgatwood]

No, but they might be able to avoid the lens entirely and do microlensing on a flat surface. For example, I could totally see the entire back of a cell phone be a light field camera, automatically throwing out data from where your fingers overlap the edges from holding it by using capacitive sensors in some way. I mean, we're probably talking twenty or thirty years out here, but that's the direction I see things heading eventually. And that would give you a believable stereo spread, not to mention much more usable resolution.

Re:Lytro's 3-D is inherently limited

[mr_exit]

As long as you've got enough parallax to work out the depth information from your scene, you can push the effect to recreate viewpoints that are wider then you have real data for.

You will end up with tiny slivers of image that you don't have pixel data for when there's a foreground element that diverges more then it did before, but that's easy to recreate. All post converted 3d films have this problem to an even greater extent, there's algorithms out there to clone the surrounding pixels or even use pixels from other frames if the object is moving through the scene

There are lightfield cameras out there that instead of using a single chip, they use an array of small cameras (think cell phone cameras) The adobe one is 500 Megapixels

See the research by Todor Georgiev http://tgeorgiev.net/ The Lytro camera is a nice cheap toy, but there's some stunning results form researchers.

Re:Lytro's 3-D is inherently limited

[Guspaz]

Typical IPD is 54-68mm. IMAX film stock is 70mm wide. This is not an insurmountable problem, at least for professional use.

Imagine a 70mm lightfield motion picture camera. Other than the fact that the data throughput would be positively insane, the requirements for physical size would be substantially less than a current IMAX camera.

I suspect that you can actually get away with less than the typical IPD and still produce a convincing effect. In which case, you can buy the required sensor today; you can get 48mm wide medium format digital sensors, and there's nothing special about the sensor in the Lytro. It's the array of microlenses and software that make it special. So it would be possible today to build a Lytro motion picture camera with a 48mm digital sensor, and I suspect that 48mm is close enough to the typical IPD to produce a convincing effect. Such sensors also have the resolution to make lightfield work for a motion picture (50 MP models turned in up the first page of results on B&H), and the cameras themselves are smaller than most motion picture cameras (or even ENG cameras)...

I suspect that the primary problem would be, again, the data throughput. Uncompressed 24fps 50 megapixel 36-bit images, those would pump out 41 gigabits per second... Compression would be pretty much required. If we use redcode as a benchmark (because apparently motion picture productions are happy with the quality of the compression enough to use it), where the minimum camera-supported compression ratio (on the RED ONE) is 8:1 and the highest is 12:1... This gives us about 5.1 Gbps and 3.4 Gbps... Heck, that's easy to handle. Existing communications tech can handle that, you could have a single 10 Gbps ethernet cable running out the back of your camera to an on-site storage box, and storing that sort of data rates isn't hard. Even a 4TB on-camera SSD module could store 156 minutes of footage... and handle those kinds of write speeds.

when the public really demands 3-D content

[Sarusa]

'when the public really demands 3-D content'

When it doesn't require glasses and doesn't give you headaches.

Re:when the public really demands 3-D content

[gmueckl]

A lenticular lens array in front of LCD screens are a nice do-it-yourself solution that almost does the trick. It makes an autostereoscopic display that can display more than 2 images in different directions, making it possible to move around in front of the screen and see a stereo image without glasses. However, there are a couple of limitations. The LCD resolution suffers tremendously and the number of zones that you can create still isn't very high. Maybe it gets better with retina displays, but I'm not sure. Even paper printouts of 20 to 30 images at 600dpi are barely good enough.

Another interesting idea is this proposal: http://gl.ict.usc.edu/Research/LFD/ - replace each pixel on a *huge* screen with a microprojector acting as a directional light source. It is insane in its own special way, but this research group has successfully thrown massive amounts of hardware at problems in the past.

Re:when the public really demands 3-D content

[Dwedit]

There was an article earlier about Tensor Displays (slashdot link), (MIT link), which used a sandwich of three high-refresh-rate LCD screens to simulate a light field by using the screens to selectively block light in multiple directions.

Re:The 3D thing is kind of fake

[Dynedain]

But look at the background through the bubbles. The background behind the bubbles doesn't change when you change the psuedo-POV

Look again. Pay attention to the bubble that masks the rear stair column. As you shift the image, the placement of the column within the bubble shifts (the amount of plant displayed varies).

This is beyond stereoscopy, and is a direct result of how they are capturing images. Stereoscopy only works on a single axis of view. This works on multiple axis.

The reason why it looks "fake" to you, is because none of this is done by the camera itself. Everything is done by running computations on the captured image. And as a result, the only way to display it is via an interactive container like Flash. The computations aren't done in realtime, so you only get whatever focus planes the algorithm (or artist) picked out when running the processing. Presumably, you could reprocess for different focal depth points.

Re:3D imagine that may leads to 3D printing !!

[ceoyoyo]

"Currently, to gather data on 3D imagery we use technologies such as MRI, which in itself not really portable."

We use things like MRI to gather tomographic data. It's for seeing inside. Lytro doesn't do that, and never will.

Currently if you want to do the kind of 3D we're talking about you can buy a Lytro and get low resolution with a lot of data and processing, or you can buy one of the commonly available compact cameras that include two lenses (like this one) and get instant, high res results.