Slashdot Mirror
Camera Lets You Shift Focus After Shooting
Zothecula writes "For those of us who grew up with film cameras, even the most basic digital cameras can still seem a little bit magical. The ability to instantly see how your shots turned out, then delete the ones you don't want and manipulate the ones you like, is something we would have killed for. Well, light field cameras could be to today's digital cameras, what digital was to film. Among other things, they allow users to selectively shift focus between various objects in a picture, after it's been taken. While the technology has so far been inaccessible to most of us, that is set to change, with the upcoming release of Lytro's consumer light field camera."
if you refocus that comment it reads as "first".
Enhance.
how is babby formed?
Have you ever tried to reduce depth of field (DOF) to a photo that has too much (for artistic purposes) DOF? It's not easy at all. If you had a pair or more of images, of the same subject, from a slightly different viewpoint (i.e. of the kind you'd take for "stereoscopic" photography) it might be easier because at least then you had some additional cues as to distance from the imaging plane of various objects within the scene, and using that it should be possible to create software to uses those cues to refine the DOF, however I don't think it would be perfect. Doing it by hand (e.g. in Photoshop) is possible but time consuming and very difficult to do right (there are a lot of approximations you have to make). The easiest, and best IMO, way of achieving your desired DOF is to do it with the camera at the time you're taking the photo.
Back to the article, I actually don't understand how the process reported could work. To record light the recording medium (e.g. CCD or CMOS sensor) has to have the light fall on it and this implies focus. Possibly it somehow also records the direction of light to allow focus manipulation post-capture. Or possibly it takes multiple shallow DOF images at once. I wish the "article" had more details.
... then why not just have the whole thing in focus at once. Infinite depth of field.
I watched the video and I believe the guy being interviewed said you can do just that.
For making movies, this would be very useful. Because when taking a movie, it is generally quite difficult to keep focus.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
From the sound of it, it basically sounds like it captures a picture with a Z-buffer -- that is, they capture spatial information and angular information, and the angular information is then matched up to find corresponding objects to assess depth for refocusing.
One nifty thing about pictures and videos with built-in Z-buffers would be that it'd be really easy to render into them. Heck, you could have a camera with a built-in GPU that could do it in realtime as you're recording. :)
One step beyond the Z-buffer would be to then do a reverse perspective transformation and extract polygonal information from the scene. This would be of particular use in video recording, where people moving allows the camera to see what's behind them, hidden sides of their bodies, etc. Then you could not only refocus your image, but outright move the camera around in the scene. Of course, if we get to that point, then we'll start seeing increasing demand for cameras that always capture 360-degree panoramas. Combine this with built-in GPS and timestamping and auto-networking of images (within whatever privacy constraints are specified by the camera's owners), and the meshes captured from different angles by people who don't even know each other could be merged into a more complete scene. In busy areas, you could have a full 3d recreation of said area at any point in time. :) "Let's do a flyover along this path in Times Square on this date at this time..."
"99 dead duelists of Dios on the wall. 99 dead duelists of Dios! Take one's ring, pass it around..."
No. This is known as plenoptic imaging, and the basic idea behind it is to use an array of microlenses positioned at the image plane, which causes the underlying group of pixels for a given microlens to "see" a different portion of the scene, much in the way that an insect's compound eyes work. Using some mathematics, you can then reconstruct the full scene over a range of focusing distances.
The problem with this approach, which many astute photographers pointed out when we read the original research paper on the topic (authored by the same guy running this company), is that it requires an imaging sensor with extremely high pixel density, yet the resulting images have relatively low resolution. This is because you are essentially splitting up the light coming through the main lens into many, many smaller images which tile the sensor. So you might need, say, a 500-megapixel sensor to capture a 5-megapixel plenoptic image.
Although Canon last year announced the development of a prototype 120-megapixel APS-H image sensor (with a pixel density rivaling that of recent digital compact point-and-shoot cameras, just on a wafer about 20x the area), it is clear that we are nowhere near the densities required to achieve satisfactory results with light field imaging. Furthermore, you cannot increase pixel density indefinitely, because the pixels obviously cannot be made smaller than the wavelength of the light it is intended to capture. And even if you could approach this theoretical limit, you would have significant obstacles to overcome, such as maintaining acceptable noise and dynamic range performance, as well as the processing power needed to record and store that much data. On top of that, there are optical constraints--the system would be limited to relatively slow f-numbers. It would not work for, say, f/2 or faster, due to the structure of the microlenses.
In summary, this is more or less some clever marketing and selective advertisement to increase the hype over the idea. In practice, any such camera would have extremely low resolution by today's standards. The prototype that the paper's author made had a resolution that was a fraction of that of a typical webcam; a production model is extremely unlikely to achieve better than 1-2 megapixel resolution.
The website about the camera doesn't have enough details, either, but this paper does give a reasonable idea of what's going on.
... demonstated to be a working principle.
The paper includes graphics and formulas... a fuck load more detail than the story link given to us...
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
It's called a Plenoptic Camera. You put a bunch of microlenses on top of a regular sensor. Each lens is the equivalent of a single 2D image pixel, but the many sensor pixels under it capture several variations of that pixel in the light field. Then you can apply different mapping algorithms to go from that sub-array to the final pixel, refocusing the image, changing the perspective slightly, etc. So color-wise it's just a regular camera. What you get is an extra two spatial dimensions (the image contains 4 dimensions of information instead of 2).
Of course, the drawback is that you lose a lot of spatial resolution since you're dividing down the sensor resolution by a constant. I doubt they can do anything interesting with less than 6x5 pixels per lens, so a 25 megapixel camera suddenly takes 1 megapixel images at best. The Wiki article does mention a new trick that overcomes this to some extent though, so I'm not sure what the final product will be capable of.
Thank you! And I was just going to post a reply to my own message wondering aloud if they manipulated the light using at the microlens level. Seem that this is exactly what they're doing
[quote]This is achieved by inserting a microlens array between the sensor and main lens, creating a plenoptic camera.[/quote]
That would still only give several (two, maybe three depending on the array) planes of focus, though, and at a sacrifice of resolution. Still, pretty cool idea.
There has been a fair amount of computer science research over the last decade over what you could do if you took a picture with a plane of cameras instead of just one or two. The resulting dataset is called a "light field". You can re-composite the pixels to change depth of focus, look around or through occluding obstacles, dynamically change point of view, etc. As digital webcams became dirt cheap people started building these hyper-cameras and experimenting with them. people learned you could relatively interesting things with small arrays of 4 or 5 squared cameras. Later on they discovered you do this with one camera, with a multi-part lense, then reconfigure the output pixels in the computer in real time. I've seen all these systems demo'ed at SIGGRAPH over the years. Now someone appears to be commercializing one.
I think the infamous bullet-dodging scene in the first Matrix movie was a type of hyper-stereo camera, a row of them albeit. The output lightfield was reconfigured expand point-of-view into time.
Have you ever tried to reduce depth of field (DOF) to a photo that has too much (for artistic purposes) DOF? It's not easy at all
Bonus! Artiste types love to brag/complain about how difficult/expensive their work was to make.
The non-artsy types don't really care about technical quality or anything other than getting a tolerably viewable "subject standing next to cultural item"
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
Read this paper (or at least skim it) - these are called plenoptic cameras.
It doesn't do any particular voodoo. I suppose you could distill it down to the point where the camera is (in function) a compound eye.
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
All the information is about the implications but not about how it actually works or the trade offs required to get there. They also seems to going directly to the consumer. There are only two reasons to bypass big spending pros and prosumers when introducing new technology:
My guess is #2. Exploding the pixel count of the sensor would make the product outrageously expensive. Clearly they are not doing that. So that means the quality suffers as finely adjustable optical focus is replaced by coarse digital focus achievable from the available sensors. We are probably getting camera phone level results. Good enough for Facebook but not something you want to print.
Because a big sensor with a microlens array could be calibrated, you could use Richardson-Lucy deconvolution to recover almost all of the raw resolution of the original sensor if the computing resources are available, in a given plane of focus.
When you focus a camera and change the aperture, you filter out some of the information coming into it. (roughly) This captures all that info. Somehow. The idea isn't to be a good photographer, but to capture more information about a scene.
Assuming the microarray isn't part of the lens, you could seemingly reduce the cost and complexity of big telephoto lenses by a lot, which are the most expensive part of any good setup.
I don't believe that it is. From the cursory second reading of the paper- it's a new type of sensor.
The paper says that the sensor was a Kodak KAF-16802CE. http://www.datasheetarchive.com/KAF-16802CE-datasheet.html#datasheets is the datasheet for this chip, and it appears to be a stock Kodak CCD sensor. Nothing particularly new about it at all. The CE part implies it is a color filtered version.
The new part is the microlens array bolted on the front.
... the prototype required a 16mp sensor array to produce a 90kp image. Some similar relationship is expected for a production camera.
Less than a 1 megapixel image. That's pretty small - would be OK for web viewing but not for printing. However, unless you 'stack' the images together to get a very large depth of field (which would often look very unreal), printing the image would not get you much aside from deciding what the focal plane would be.
A web gallery, however, would allow you to move the focus in and out at will (as shown in the examples) and might be more commercially viable. Hogan's main complaint is that they will have to sell a metric butload of them to make a profit and that would be hard to do as a one trick, low resolution pony. I'd love a higher resolution version for macrophotography but I guess I will just do plain old focus stacking for a while longer.
Faster! Faster! Faster would be better!
The same way it does in a regular camera. If a fence is close to the lens, keeping it out of focus lets you see through it just fine.
It works because the lens has finite size, and from some parts of the lens you see past the wires in the fence, while from others you do not.
The sacrifice of resolution isn't really that big a concern. Consumer cameras have far more resolution than they need these days, as the almighty megapixel has been used as a marketing ploy even though increasing pixel density on the CCDs has led to lower image quality overall. My 10 year old 2Mpx Canon still takes better pictures than any of my wife's last 3 compact cameras (4, 5 and 8Mpx Nikon and Canons), especially in low light. I would go so far as to say it doesn't make sense to have go beyond much more than 4Mpx with lenses the size of compact cameras, as details will be lost due to lens quality long before the pixel count causes loss of detail.
You might want to look up "light field"; apparently it's a well-defined term within the field (which has some connections to what I'm familiar with and hence was talking about but is formulated differently for different application). In particular, "full light field" is different from "all the light rays".
Measuring the full light field means measuring all the available information (in a different context you may be familiar with, if you work with light in any way, an analog would be a full Stokes polarimeter that measures all available polarization parameters of the light, not just one or two, as most cameras and filters do) in the light that the sensor receives, whereas measuring all the light rays would mean what you were talking about---capturing the light emitted into all solid angles.
I'll admit first that I myself wasn't familiar with the term "light field" and the term itself did sound a little sci-fi-y, but it is a valid, technical jargon, not just marketing speak or jibberish you see in sci-fi.
It happens with autofocus: on the tall grass in the foreground, not on the interesting animal a bit further.
Light can use a lensing to travel around obstructions. On earth for example in fata morganas.
"I'm not much interested in interoperability. I want substitutability. I want to be able to throw your software out."
This was already done years ago in a different manner by Ren Ng (I believe was his name) who used an extra digital sensor and it's not a holocam it's a standard digital camera modified to enable changing of focus after the fact.