Slashdot Mirror
Camera Lets You Shift Focus After Shooting
Zothecula writes "For those of us who grew up with film cameras, even the most basic digital cameras can still seem a little bit magical. The ability to instantly see how your shots turned out, then delete the ones you don't want and manipulate the ones you like, is something we would have killed for. Well, light field cameras could be to today's digital cameras, what digital was to film. Among other things, they allow users to selectively shift focus between various objects in a picture, after it's been taken. While the technology has so far been inaccessible to most of us, that is set to change, with the upcoming release of Lytro's consumer light field camera."
if you refocus that comment it reads as "first".
A holocam.
vos nescitis quicquam, nec cogitatis quia expedit nobis ut unus moriatur homo pro populo et non tota gens pereat.
Enhance.
how is babby formed?
For all the data it collects, does it do full spectrum or just 3 colors of light? Polarization after the fact? I wonder how long this will be "at least a year away." If it is the real, I can think of lots of scientific applications more useful than a consumer camera.
What I want to know is, if they can focus at any point in the picture - and it looks as though they can, the interactive graphic is amazing - then why not just have the whole thing in focus at once. Infinite depth of field. If you wanted a shallow depth of field for artistic purposes, you could presumably add that later too. Neat.
I have a small point and shoot camera, and I rarely ever have the problem that my photos are out of focus. Blurry photos on evening and night shots is the most common problem I have. Not to say this technology sucks, but I doubt that you can get the average consumer to pay double the price for this feature. However, there are probably tons of other uses that this technology might have (in more profitable areas). Maybe for security cameras, or unmanned vehicles.
goatse.
How can I believe you when you tell me what I don't want to hear?
Conceptually, its a little like focus stacking http://prometheus.med.utah.edu/~bwjones/2009/03/focus-stacking/ only with a compound lens that does all the exposures at once. More examples of focus stacking here: http://prometheus.med.utah.edu/~bwjones/tag/focus-stacking/
Visit Jonesblog and say hello.
The first product will probably be a DSLR-sized sensor with mobile phone-type image sensor density. They are trading in a lot of pixels for this feature. You'll need 100 megapixel sensors to end up with usable image sizes as one microlens covers many sensor cells. It will be interesting to see how low light noise artifacts will look as there is bound to be a lot of them with such high sensor density.
A witty
Don't click his link. It's the Goatse.cx image.
Also known as the poor man's basilisk.
From what I recall of reading about this a year+ ago, the same tech would allow for mathematically changing focus, zoom, pan, tilt in real time on the signal from the camera or post-fact on a recording of the same signal. So, basically, a flat non-moving sensor can now emulate a PZT camera. I can imagine that with a spherical lens or one of those weird mirrors that lets a regular camera catch a 360 image, it should be possible to make a near holocam. Imagine a movie shot like this and glasses that allow you to focus on different depths at will (maybe by watching your eyes?) it would be a "real" 3d experience, none of that fuzzy background you get on stereo 3d movies.
For making movies, this would be very useful. Because when taking a movie, it is generally quite difficult to keep focus.
If Pandora's box is destined to be opened, *I* want to be the one to open it.
No. This is known as plenoptic imaging, and the basic idea behind it is to use an array of microlenses positioned at the image plane, which causes the underlying group of pixels for a given microlens to "see" a different portion of the scene, much in the way that an insect's compound eyes work. Using some mathematics, you can then reconstruct the full scene over a range of focusing distances.
The problem with this approach, which many astute photographers pointed out when we read the original research paper on the topic (authored by the same guy running this company), is that it requires an imaging sensor with extremely high pixel density, yet the resulting images have relatively low resolution. This is because you are essentially splitting up the light coming through the main lens into many, many smaller images which tile the sensor. So you might need, say, a 500-megapixel sensor to capture a 5-megapixel plenoptic image.
Although Canon last year announced the development of a prototype 120-megapixel APS-H image sensor (with a pixel density rivaling that of recent digital compact point-and-shoot cameras, just on a wafer about 20x the area), it is clear that we are nowhere near the densities required to achieve satisfactory results with light field imaging. Furthermore, you cannot increase pixel density indefinitely, because the pixels obviously cannot be made smaller than the wavelength of the light it is intended to capture. And even if you could approach this theoretical limit, you would have significant obstacles to overcome, such as maintaining acceptable noise and dynamic range performance, as well as the processing power needed to record and store that much data. On top of that, there are optical constraints--the system would be limited to relatively slow f-numbers. It would not work for, say, f/2 or faster, due to the structure of the microlenses.
In summary, this is more or less some clever marketing and selective advertisement to increase the hype over the idea. In practice, any such camera would have extremely low resolution by today's standards. The prototype that the paper's author made had a resolution that was a fraction of that of a typical webcam; a production model is extremely unlikely to achieve better than 1-2 megapixel resolution.
I think you're confusing presentation with the source data. If those five JPEGs are generated from a single light-field exposure, then the demo isn't faked. (Or at least, is faked to the same extent that a typical camera shot is faked because the sensor data needs to be processed before it's displayed to you.)
I've been doing this for a few years, with one camera taking many views, since I first found out about the research they were doing at Stanford. Here are some scenes around Chicago which are composites of many photos to generate a synthetic focus. The idea is to capture the scene from many slightly different points of view, and to capture all of the parallax information, which then yields depth.
I haven't be able to make it happen, but it should be possible to combine N pictures to get a bit less than N times the normal resolution. If you had 100 photos that were 8 megapixels each, you should be able to composite them into a 100 megapixel image with the right alignment and extrapolation algorithms.
There has been a fair amount of computer science research over the last decade over what you could do if you took a picture with a plane of cameras instead of just one or two. The resulting dataset is called a "light field". You can re-composite the pixels to change depth of focus, look around or through occluding obstacles, dynamically change point of view, etc. As digital webcams became dirt cheap people started building these hyper-cameras and experimenting with them. people learned you could relatively interesting things with small arrays of 4 or 5 squared cameras. Later on they discovered you do this with one camera, with a multi-part lense, then reconfigure the output pixels in the computer in real time. I've seen all these systems demo'ed at SIGGRAPH over the years. Now someone appears to be commercializing one.
I think the infamous bullet-dodging scene in the first Matrix movie was a type of hyper-stereo camera, a row of them albeit. The output lightfield was reconfigured expand point-of-view into time.
All the information is about the implications but not about how it actually works or the trade offs required to get there. They also seems to going directly to the consumer. There are only two reasons to bypass big spending pros and prosumers when introducing new technology:
My guess is #2. Exploding the pixel count of the sensor would make the product outrageously expensive. Clearly they are not doing that. So that means the quality suffers as finely adjustable optical focus is replaced by coarse digital focus achievable from the available sensors. We are probably getting camera phone level results. Good enough for Facebook but not something you want to print.
The problem is this: this is not how movies work. It'll cost a fortune to make a movie where you can "look around". Shots are usually planned in detail, so if something is out of focus and is a prop/set, it's way cheaper that way. Even in CG movies, there are still digital props, sets, etc, and they are planned according to the needs of script and director's ideas. The level of detail varies and usually is only enough to do the job, doing otherwise would be a waste of money -- if it doesn't end on film, it's a waste.
Here, suddenly, you'll need sets with way more detail, all props will be subject to potential scrutiny even if artistically they're not very important, and it'll be way more work for continuity and to make sure there's no guffaws in the recorded material.
I don't see it happening, not without a major mental shift in the moviemaking business.
A successful API design takes a mixture of software design and pedagogy.
So what, you expected them to recode their algorithms in flash, send you a source image that's dozens of megabytes in size, and have you wait tens of seconds, possibly minutes, while the whole thing is recalculated after each click? Haha.
A successful API design takes a mixture of software design and pedagogy.
And if it weren't a "faked demo," how exactly would the images be presented to you?
Are you really that dense? Did you think they were going to create a full fledged web viewer for files of god knows what size so you could get the genuine experience over what 5 demo images could show you?
Transparent sensors. Every picture is a 3d array instead of a 2d array.
"I haven't be able to make it happen, but it should be possible to combine N pictures to get a bit less than N times the normal resolution. If you had 100 photos that were 8 megapixels each, you should be able to composite them into a 100 megapixel image with the right alignment and extrapolation algorithms."
No, you can't. Using super-resolution and an expensive mount that can shift the picture by EXACTLY half a pixel (or a quarter, or an eighth), you can get better resolution out of multiple shots, but the technique is severely limited in practice. If you get a factor of two you're doing well.
Unless you're talking about using a longer lens, taking multiple pictures and stitching them together. That's trivial.
...which explains the tech and its application in wonderful detail. http://www.lytro.com/renng-thesis.pdf
Transparent sensors. Every picture is a 3d array instead of a 2d array.
Foveon
It would be great if these two technologies can dovetail in a way that I can get a high resolution (6-8 megapixel equivalent in current terms) picture with the ability to pick both my depth of field and focal point post processing.
The sun is the same in a relative way, but you are shorter of breath and one day closer to death
... the prototype required a 16mp sensor array to produce a 90kp image. Some similar relationship is expected for a production camera.
Less than a 1 megapixel image. That's pretty small - would be OK for web viewing but not for printing. However, unless you 'stack' the images together to get a very large depth of field (which would often look very unreal), printing the image would not get you much aside from deciding what the focal plane would be.
A web gallery, however, would allow you to move the focus in and out at will (as shown in the examples) and might be more commercially viable. Hogan's main complaint is that they will have to sell a metric butload of them to make a profit and that would be hard to do as a one trick, low resolution pony. I'd love a higher resolution version for macrophotography but I guess I will just do plain old focus stacking for a while longer.
Faster! Faster! Faster would be better!
To a degree. Kinect has limited range on depth sensor. Camera image is from exactly one POV, so you can't get the micro-movements you can with their camera.
The real question is, are we already living in a dystopia?
Yeah, Satan is the ruler of this system of things. But a new king is coming; be awake and be ready.
when most people do not need the 12MP photos they take now; cam makers can offer this or similar features based upon the micro camera feature to sell greater MP sensors for images that are no larger than 12MP. Initially, I'd imagine they'd want a work around for when they do not want to use this feature so they can sell you a 18MP camera but the new mode outputs "small" images which are still plenty large for sharing online.
Democracy Now! - uncensored, anti-establishment news
Actually, with Richardson-Lucy deconvolution it's possible to recover the information, as long as you have the positions of the pixels, and the diffusion function to enough precision.
No, it's not. You can do a little bit, provided you have enough precision moving the camera, but super resolution (no matter what deconvolution algorithm you use, and RL isn't exactly cutting edge) doesn't really buy you much. It's useful in a few niche areas, like microscopy, but other than that it's impractical.
Yes, I did super resolution research a few years ago.
To do a focus sweep (near to far or whatnot) in a fast shutter speed storing each shot. So 1 4000 of a second cranked to lets day 6400 ISO +. Or in high def video mode. DONE.
Imagine this technology paired with eye tracking so that where you look in a picture becomes where the focus is. Now run it at 24+ frames per second. Cool stuff.
it is clear that we are nowhere near the densities required to achieve satisfactory results with light field imaging.
Density would just be one way to do it. Slice it up over time, add more sensors and split the light, use some of those 3D sensors, etc. Each of those has its own set of trade-offs, but we're just talking about time here. The VC's likely know that the sensor tech is poised to be right to eliminate those trade-offs, making now the right time to start the company and put out a 1.0 camera.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
Now I hope when I watch a 3D movie, the focus of the picture follows what my eyes are focusing! That would makes 3D movie much more enjoyable.