Slashdot Mirror
Camera Lets You Shift Focus After Shooting
Zothecula writes "For those of us who grew up with film cameras, even the most basic digital cameras can still seem a little bit magical. The ability to instantly see how your shots turned out, then delete the ones you don't want and manipulate the ones you like, is something we would have killed for. Well, light field cameras could be to today's digital cameras, what digital was to film. Among other things, they allow users to selectively shift focus between various objects in a picture, after it's been taken. While the technology has so far been inaccessible to most of us, that is set to change, with the upcoming release of Lytro's consumer light field camera."
if you refocus that comment it reads as "first".
A holocam.
vos nescitis quicquam, nec cogitatis quia expedit nobis ut unus moriatur homo pro populo et non tota gens pereat.
Enhance.
how is babby formed?
For all the data it collects, does it do full spectrum or just 3 colors of light? Polarization after the fact? I wonder how long this will be "at least a year away." If it is the real, I can think of lots of scientific applications more useful than a consumer camera.
What I want to know is, if they can focus at any point in the picture - and it looks as though they can, the interactive graphic is amazing - then why not just have the whole thing in focus at once. Infinite depth of field. If you wanted a shallow depth of field for artistic purposes, you could presumably add that later too. Neat.
I have a small point and shoot camera, and I rarely ever have the problem that my photos are out of focus. Blurry photos on evening and night shots is the most common problem I have. Not to say this technology sucks, but I doubt that you can get the average consumer to pay double the price for this feature. However, there are probably tons of other uses that this technology might have (in more profitable areas). Maybe for security cameras, or unmanned vehicles.
Old News! this is just another plenoptic camera... booring.. so 2006.
goatse.
How can I believe you when you tell me what I don't want to hear?
Don't click his link. It's the Goatse.cx image.
Conceptually, its a little like focus stacking http://prometheus.med.utah.edu/~bwjones/2009/03/focus-stacking/ only with a compound lens that does all the exposures at once. More examples of focus stacking here: http://prometheus.med.utah.edu/~bwjones/tag/focus-stacking/
Visit Jonesblog and say hello.
The first product will probably be a DSLR-sized sensor with mobile phone-type image sensor density. They are trading in a lot of pixels for this feature. You'll need 100 megapixel sensors to end up with usable image sizes as one microlens covers many sensor cells. It will be interesting to see how low light noise artifacts will look as there is bound to be a lot of them with such high sensor density.
A witty
The only downfall of this is that it reduces the resolution considerably of the camera sensor.
The camera works by placing an array of micro lenses in front of the image sensor. This allows you to record the direction of the light was traveling in addition to just its luminance. In effect you can mathematically change the focus of the image by selecting what directional light rays are incorporated into the photo.
You can read the CEO's thesis here
http://www.lytro.com/renng-thesis.pdf
Camera lets you suck 50 000 000$ from VCs and disappear.
If you examine their demo file, you'll find the 5 static JPEGs inside.
http://cdn01.lytro.com/media/lytro/lyt-37/lyt-37.lfp
What exactly was the point of that "demo"?
Don't click his link. It's the Goatse.cx image.
Also known as the poor man's basilisk.
For making movies, this would be very useful. Because when taking a movie, it is generally quite difficult to keep focus.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
No. This is known as plenoptic imaging, and the basic idea behind it is to use an array of microlenses positioned at the image plane, which causes the underlying group of pixels for a given microlens to "see" a different portion of the scene, much in the way that an insect's compound eyes work. Using some mathematics, you can then reconstruct the full scene over a range of focusing distances.
The problem with this approach, which many astute photographers pointed out when we read the original research paper on the topic (authored by the same guy running this company), is that it requires an imaging sensor with extremely high pixel density, yet the resulting images have relatively low resolution. This is because you are essentially splitting up the light coming through the main lens into many, many smaller images which tile the sensor. So you might need, say, a 500-megapixel sensor to capture a 5-megapixel plenoptic image.
Although Canon last year announced the development of a prototype 120-megapixel APS-H image sensor (with a pixel density rivaling that of recent digital compact point-and-shoot cameras, just on a wafer about 20x the area), it is clear that we are nowhere near the densities required to achieve satisfactory results with light field imaging. Furthermore, you cannot increase pixel density indefinitely, because the pixels obviously cannot be made smaller than the wavelength of the light it is intended to capture. And even if you could approach this theoretical limit, you would have significant obstacles to overcome, such as maintaining acceptable noise and dynamic range performance, as well as the processing power needed to record and store that much data. On top of that, there are optical constraints--the system would be limited to relatively slow f-numbers. It would not work for, say, f/2 or faster, due to the structure of the microlenses.
In summary, this is more or less some clever marketing and selective advertisement to increase the hype over the idea. In practice, any such camera would have extremely low resolution by today's standards. The prototype that the paper's author made had a resolution that was a fraction of that of a typical webcam; a production model is extremely unlikely to achieve better than 1-2 megapixel resolution.
In their video demo, they let you pick the focal depth or you can hit the button that makes everything focused. The software can pick different portions of the image and apply the necessary focal depth.
I've been doing this for a few years, with one camera taking many views, since I first found out about the research they were doing at Stanford. Here are some scenes around Chicago which are composites of many photos to generate a synthetic focus. The idea is to capture the scene from many slightly different points of view, and to capture all of the parallax information, which then yields depth.
I haven't be able to make it happen, but it should be possible to combine N pictures to get a bit less than N times the normal resolution. If you had 100 photos that were 8 megapixels each, you should be able to composite them into a 100 megapixel image with the right alignment and extrapolation algorithms.
There has been a fair amount of computer science research over the last decade over what you could do if you took a picture with a plane of cameras instead of just one or two. The resulting dataset is called a "light field". You can re-composite the pixels to change depth of focus, look around or through occluding obstacles, dynamically change point of view, etc. As digital webcams became dirt cheap people started building these hyper-cameras and experimenting with them. people learned you could relatively interesting things with small arrays of 4 or 5 squared cameras. Later on they discovered you do this with one camera, with a multi-part lense, then reconfigure the output pixels in the computer in real time. I've seen all these systems demo'ed at SIGGRAPH over the years. Now someone appears to be commercializing one.
I think the infamous bullet-dodging scene in the first Matrix movie was a type of hyper-stereo camera, a row of them albeit. The output lightfield was reconfigured expand point-of-view into time.
I have had a rather enjoyable experience playing with a light field camera ( www.raytrix.de/index.php/id-4d-cameras.html ) and this article really undersells their capabilities! Not only can you dynamically refocus the image you can also alter perspective, slightly change the angle of view and change the depth of field... This video shows some of this pretty well: http://www.youtube.com/watch?v=9H7yx31yslM
All the information is about the implications but not about how it actually works or the trade offs required to get there. They also seems to going directly to the consumer. There are only two reasons to bypass big spending pros and prosumers when introducing new technology:
My guess is #2. Exploding the pixel count of the sensor would make the product outrageously expensive. Clearly they are not doing that. So that means the quality suffers as finely adjustable optical focus is replaced by coarse digital focus achievable from the available sensors. We are probably getting camera phone level results. Good enough for Facebook but not something you want to print.
Transparent sensors. Every picture is a 3d array instead of a 2d array.
this sounds like something that can be done cheaply with a kinect mod
You just need to open up your lens and put in a coded aperture, and the rest is all software. See the paper Image and Depth from a Conventional Camera with a Coded Aperture, and especially check out the last several slides from the supplementary file where they take a single picture and refocus it at a new focal length.
So I can buy a camera that is likely to be very expensive and take a picture that I can selectively focus on my computer after the fact.
Ok, but will the image always look as bad as the one on the page that was linked? Because I have a webcam that takes better pictures than that.
If my Canon point-and-shoot took pictures that bad I would send it in for repair.... or buy a new one since it was only a hundred bucks.
Guaranteed that I would send in either of my dSLRs in for repair. One of them is OLD (6mp) and still takes much better pictures. Of course it has the disadvantage of me having to know what I am taking a picture of when I take a picture.....
Yeah, generally speaking I see this as "tech for tech's sake" rather than anything actually useful.
Take a shot with a large focal length then blar around surrounding focus point!!! Simple stuff
"I haven't be able to make it happen, but it should be possible to combine N pictures to get a bit less than N times the normal resolution. If you had 100 photos that were 8 megapixels each, you should be able to composite them into a 100 megapixel image with the right alignment and extrapolation algorithms."
No, you can't. Using super-resolution and an expensive mount that can shift the picture by EXACTLY half a pixel (or a quarter, or an eighth), you can get better resolution out of multiple shots, but the technique is severely limited in practice. If you get a factor of two you're doing well.
Unless you're talking about using a longer lens, taking multiple pictures and stitching them together. That's trivial.
Lookout!
...which explains the tech and its application in wonderful detail. http://www.lytro.com/renng-thesis.pdf
Transparent sensors. Every picture is a 3d array instead of a 2d array.
Foveon
It would be great if these two technologies can dovetail in a way that I can get a high resolution (6-8 megapixel equivalent in current terms) picture with the ability to pick both my depth of field and focal point post processing.
The sun is the same in a relative way, but you are shorter of breath and one day closer to death
... the prototype required a 16mp sensor array to produce a 90kp image. Some similar relationship is expected for a production camera.
Less than a 1 megapixel image. That's pretty small - would be OK for web viewing but not for printing. However, unless you 'stack' the images together to get a very large depth of field (which would often look very unreal), printing the image would not get you much aside from deciding what the focal plane would be.
A web gallery, however, would allow you to move the focus in and out at will (as shown in the examples) and might be more commercially viable. Hogan's main complaint is that they will have to sell a metric butload of them to make a profit and that would be hard to do as a one trick, low resolution pony. I'd love a higher resolution version for macrophotography but I guess I will just do plain old focus stacking for a while longer.
Faster! Faster! Faster would be better!
To a degree. Kinect has limited range on depth sensor. Camera image is from exactly one POV, so you can't get the micro-movements you can with their camera.
a matrix of sensors allow you really point and shoot and re-focus , re-adjust lights whenever you want to
The real question is, are we already living in a dystopia?
Yeah, Satan is the ruler of this system of things. But a new king is coming; be awake and be ready.
when most people do not need the 12MP photos they take now; cam makers can offer this or similar features based upon the micro camera feature to sell greater MP sensors for images that are no larger than 12MP. Initially, I'd imagine they'd want a work around for when they do not want to use this feature so they can sell you a 18MP camera but the new mode outputs "small" images which are still plenty large for sharing online.
Democracy Now! - uncensored, anti-establishment news
Actually, with Richardson-Lucy deconvolution it's possible to recover the information, as long as you have the positions of the pixels, and the diffusion function to enough precision.
No, it's not. You can do a little bit, provided you have enough precision moving the camera, but super resolution (no matter what deconvolution algorithm you use, and RL isn't exactly cutting edge) doesn't really buy you much. It's useful in a few niche areas, like microscopy, but other than that it's impractical.
Yes, I did super resolution research a few years ago.
To do a focus sweep (near to far or whatnot) in a fast shutter speed storing each shot. So 1 4000 of a second cranked to lets day 6400 ISO +. Or in high def video mode. DONE.
To do a focus sweep (near to far or whatnot) in a fast shutter speed storing each shot. So 1 4000 of a second cranked to lets day 6400 ISO +. Or in high def video mode. DONE.
Your solution isn't simple and it doesn't solve the problem. Getting pictures in 6400 ISO gives you horrible noise in your CCD unless you have really high quality sensors. Likewise, unless it's really bright, your images will probably be dark because the sensor isn't getting enough light to resolve things. What happens if it's dark and 6400 ISO gives you black images, also how do you deal with a fast moving object going through the frame during your focus sweep. Each frame gets a different scene and your image processing algorithm will get very confused.
Imagine this technology paired with eye tracking so that where you look in a picture becomes where the focus is. Now run it at 24+ frames per second. Cool stuff.
http://news.bbc.co.uk/1/hi/technology/3643964.stm
quote:
Using wavefront coding, the system encodes the image so it always looks the same without losing any information.
Wavefront coding was originally proposed by Dr Edward Dowski, of the University of Colorado, for microscopy.
BugBear
it is clear that we are nowhere near the densities required to achieve satisfactory results with light field imaging.
Density would just be one way to do it. Slice it up over time, add more sensors and split the light, use some of those 3D sensors, etc. Each of those has its own set of trade-offs, but we're just talking about time here. The VC's likely know that the sensor tech is poised to be right to eliminate those trade-offs, making now the right time to start the company and put out a 1.0 camera.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Now I hope when I watch a 3D movie, the focus of the picture follows what my eyes are focusing! That would makes 3D movie much more enjoyable.
I'd like to see this combined with 3D tech to allow me to focus on something on the screen, and not just on what the director focused on.