Refocusable Plenoptic Light-Field Photography

virgil_disgr4ce writes "Wired is reporting that a Stanford student using about 90,000 microlenses has developed a plenoptic camera whose images can be refocused, via software, after they are exposed." From the article: "'We just think it'll lead to better cameras that make it easier to take pictures that are in focus and look good,' said Ng's adviser, Stanford computer science professor Pat Hanrahan."

Comments

You know what this means...

[doxology]

Better Porn!

Re:You know what this means...

[Anonymous Coward]

Focus me up...

Mythbusters

[SirWraith]

Fuzzy monsters like Bigfoot shalt no longer wreak havoc on unsuspecting rednecks who would have means of affording such a 16-megapixel camera. It'll be like Oliver Wood meets Bo Duke.

Make it easier to take pictures...

[Trigun]

Automatic focus available only on next years model.

Re:Make it easier to take pictures...

[Jozer99]

Very true. The problem with this camera is that in order to get a good image out of it, you have to have a supercomputer crunch numbers for awhile.

innovation

[Lord Ender]

As soon as I heard of this, I immediately realized how to do it. But I would not have thought to do it on my own. This kind of smart thinking is why we have a patent system. The patent system was not designed to protect business methods, such as completing a sale using n clicks instead of n+1.

Re:innovation

[ScentCone]

This kind of smart thinking is why we have a patent system. The patent system was not designed to protect business methods, such as completing a sale using n clicks instead of n+1.

But you are going to run into the inevitable trolling twit who will complain that this is just a way of "collecting and recording light with a new twist, and no one can own that, man!" Any second now, just wait.

Re:innovation

[Anonymous Coward]

To the contrary, this kind of smart thinking is exactly why we don't need a patent system. Did this guy get a patent? No! Patent rights are not what is motivating him at all. Furthermore, this guy didn't invent this idea. Practically no worthwhile invention is invented out of the blue by a single person, at least not any more. People have been researching this for years. Building on the accomplishments of previous experiments, publishing their results in peer-reviewed journals, that sort of thing. The only thing patents could have done to the development of this technology was obstruct it. Luckily that doesn't seem to have happened.

Re:innovation

[RedWizzard]

As soon as I heard of this, I immediately realized how to do it. But I would not have thought to do it on my own. This kind of smart thinking is why we have a patent system. The patent system was not designed to protect business methods, such as completing a sale using n clicks instead of n+1.

The patent system is not meant to protect an idea either. It's meant to protect a non-obvious implementation of an idea.

Re:innovation

[Directrix1]

If you look at this site: Stanford Lightfield Project. You will see that the basic premise behind defining a light field and mathematically manipulating it has been around sine the 30's. Whats cool here is the camera. In fact being in the photography business myself, I was just telling my father a couple months ago about how it would be easy to refocus an image if there was a lense that just captured a grid of images with slightly different perspectives from each cell. Refocusing the light field is a pretty obvious benefit to this system, I would deem not worthy of a patent, as it is just a way to mathematically manipulate a light field.

Re:innovation

[NoTheory]

Heh, this is how the eye/brain processes information (vaguely speaking). I wonder if you could argue that that constituted prior art. Guess it'd depends on if you're an Intelligent Design nut.

Re:innovation

[EternityInterface]

The only thing patents could have done to the development of this technology was obstruct it. Luckily that doesn't seem to have happened.

Well if the average percieved income of patenting a device was say $100 million, you could of spent a lot more in research. (How it is with the medical industry, I think?)

Re:innovation

[servognome]

To the contrary, this kind of smart thinking is exactly why we don't need a patent system. Did this guy get a patent? No! Patent rights are not what is motivating him at all.

True but patent rights might motivate somebody to actually make a product we could use in a shorter time frame.

"It stems from early-20th-century work on integral photography, which experimented with using lens arrays in front of film, and an early-1990s plenoptic camera developed at MIT and used for range finding... Turning Ng's invention into a commercial product poses a few challenges. First, it works best with expensive high-resolution cameras, and when you add the price of Ng's device, the cost could be prohibitive (Ng declined to estimate a cost)"

It's great to have ideas on paper, but until the public can actually get a product it's academic. Think Canon will spend tens of millions of dollars to develop a consumer product if it can be reverse engineered and exactly copied? Yes, innovation happens without patents, but patents can make it profitable to speed the process up through investment.
I do think the 17 year protection may be outdated given the rate of change has increased and time to market has decreased over the last 200 years.

Re:innovation

[fossa]

I've often felt the same way about various things I've heard about. I wonder, is there anything scientific to this? If someone tells me X has been done (even if it hasn't), am I more likely to come up with a way to accomplish X? Or is it just my imagination?

Re:innovation

[mOdQuArK!]

patent rights might motivate somebody to actually make a product we could use in a shorter time frame.

Patents are more likely to discourage somebody from making a product because they wouldn't be able to make anything without violating a half a dozen stupid patents.

Re:innovation

[Anonymous Coward]

For someone "skilled in the art" I suppose that given a problem there's a good chance that a brainstorming session will come up with an answer that somebody else has tried to patent. Not always... but many ideas from brainstorming end up being filtered out for one reason or another despite their patentability.

Re:innovation

[minorproblem]

Isn't this just the same as de-convoluting an image except its easier to work out the mathamatical function of the lens?

Re:innovation

[Anonymous Coward]

As soon as I heard of this, I immediately realized how to do it. But I would not have thought to do it on my own. This kind of smart thinking is why we have a patent system. The patent system was not designed to protect business methods, such as completing a sale using n clicks instead of n+1.

The patent system is not meant to protect an idea either. It's meant to protect a non-obvious implementation of an idea.

incorrent. The patent system is meant to protect the profits that can be made from an idea.

Re:innovation

[ceoyoyo]

In what way? The human eye only has one lens, which focuses. Now an insect eye... maybe sort of.

Re:innovation

[Ed Avis]

With an ordinary photograph you can play with the Refocus GIMP plugin. But you'd have to manually choose the radius of the circle of confusion. I don't think it will be possible to automatically get the whole range of focus ranges shown in their example images.

Re:innovation

[Anonymous Coward]

The patent system is not meant to protect an idea either. It's meant to protect a non-obvious implementation of an idea. incorrent. The patent system is meant to protect the profits that can be made from an idea.

incorrect. The patent system is meant to protect mature industries from innovation.

Re:innovation

[Rakishi]

Patents don't do that, a bad implementation of patents does that. I believe the phrase is "throwing out the baby with the bath water."

Re:innovation

[BillyBlaze]

Do you really think this guy was the first to think of it? Surely you've had like 10 great ideas this week that you don't have the time, resources, or expertise to develop. How would you feel if next year, you finally get around to it, but then get sued by some jerk (or incorporated group of jerks) who beat you to the patent office?

To put it another way, maybe you "wouldn't have thought of it," but surely it doesn't follow that nobody would have or did.

Re:innovation

[DZign]

humans have 2 eyes and the brain overlaps the images of both eyes into one sharp view we think we see

Re:innovation

[Anonymous Coward]

What you don't understand is that the human eyes have to focus on a single point, this camera does not focus at all.
It uses the CCD to store the direction of the light coming in to it and then works backwards from that to create the final image.
The human eye is a simple camera not this device.

Re:innovation

[yfkar]

I'd say the technical part should be patentable, the mathematical software part not.

Re:innovation

[bit01]

True but patent rights might motivate somebody to actually make a product we could use in a shorter time frame.

And it might not. Nobody knows. This is the usual PTO wishful thinking. The PTO has no scientific, objective evidence that patents in large areas of technology do anything more than hinder progress and interfere in the citizen's business. There is plenty of anecdotal evidence that patents in wide areas cause more harm than good and cost billions of dollars. This should be regarded as criminal negligence by public servants.

In this case it might stop many Asian companies creating cheap versions of the product and creating a big market instead of a niche product.

I find it very noticeable that most patented ideas end up as expensive, niche products instead of a high impact mass market items with realistic free market competition. By the time the patent has expired the market has moved on. It's the non-patented stuff (e.g. the web or the idea of opening a simple/modern furniture chain like Ikea) that has the big impact.

And please, no nonsense about how people will not think up new things with no patent protection. In large areas of technology and industry this is simply not true.

---

You communist! Breathing shared air!

Re:innovation

[MooseTick]

"A slashdotter who did not build his own computer is like a jedi who did not build his own lightsaber."

If I'm not mistaken, Luke didn't build his own lightsaber. He got it as a hand-me-down from his dad.

Re:innovation

[h4ck7h3p14n37]

The patent system is not meant to protect an idea either. It's meant to protect a non-obvious implementation of an idea.

True, but the USPTO has been granting quite a few patents for "inventions" that don't exist. For example, Sony's brain controller.

Re:innovation

[mOdQuArK!]

I don't agree.

The fundamental mechanism of the patent system is to allow a patentholder to selectively suppress the constructive activity of others. _Any_ system implemented using this kind of mechanism is going to end up suppressing the overall economic activity of society.

Re:innovation

[CodeShark]

Hello?

Patents are for implementations of ideas, not the ideas themselves. So while the idea of a drug that "cures cancer" is decades old, a new drug's implementation that comes closer is patentable.

Similarly, this plenoptic camera seems to represent a new (i.e.) digital method of analyzing light along a path and using the information to digitally restore an image to focus, and as such it seems to clearly qualify for patent protection.

P.S. In case you wonder, I sometimes pick up extra work occasionally doing professional photography (commercial and some portrait work) when a project strikes my interest and/or the $$ are interesting enough, so I'm not just a casual reader on camera related technologies. Heaven knows, if I'd invented an after-the-fact refocusing system, I'd want a patent on it as well.

Re:innovation

[Anonymous Coward]

It isn't MEANT to do that...
but it still does it.

Re:innovation

[Directrix1]

The camera itself? Sure, patent it. The mathematical manipulations to calculate ray convergence onto a plain is not a novel idea at all, and is in fact the very definition of focusing an image. This should NOT be patented.

Re:innovation

[ceoyoyo]

Each view is a standard picture, with one distance (the SAME distance for both eyes) in focus. If your eyes focus on different things at the same time see a neurologist immediately -- you've probably had a stroke. Your brain doesn't construct an image the way this camera does. If it did your eyes wouldn't need lenses and near or far sightedness would be a neurological problem, not an issue with misfocusing in your eyes (correctable with eyeglasses).

In fact, your eyes not only have to focus, but they really only care about having a small area of the image that falls on the fovea in focus. Try it -- look directly at an object. You can't distinguish fine details in your visual field to either side of that object without shifting your eyes.

Re:innovation

[servognome]

The fundamental mechanism of the patent system is to allow a patentholder to selectively suppress the constructive activity of others. _Any_ system implemented using this kind of mechanism is going to end up suppressing the overall economic activity of society.

You left out "for a limited time and only under the condition of full disclosure". Until recently it has been relatively easy to design around patents. Essentially what would happen is the publication of a non-trivial implementation would reveal the fundamental ideas so that others can have their own non-trivial implementation. For example Westinghouse creating a two-piece lightbulb to get around the patent for a one-piece lightbulb.
The problem now is that many patents are overly-broad; they protect ideas, not implementations as patents originally were meant to protect.

Re:innovation

[servognome]

There is plenty of anecdotal evidence that patents in wide areas cause more harm than good and cost billions of dollars. This should be regarded as criminal negligence by public servants.

Yes, overly broad patents do cause more harm than good, but its a problem with the patent reviewers, not the overall system.

I find it very noticeable that most patented ideas end up as expensive, niche products instead of a high impact mass market items with realistic free market competition

Not really, computer chips have dozens of patents, cars have patents, even shampoo bottles and showerhead designs have patents. I wouldn't call these things niche products.
Many lead-free solder alloys are patented, but that hasn't stopped the electronics industry conversion. First each patent that is published gives insight into novel solutions (eg. introduction of dopants), rather than blocking off development, they encourage sharing ideas. If I see a patent with a cobalt dopant, I can quickly understand how it works and look at some other alloy with similar properties.
Where the patent process has broken down especially in the area of software, is that patents are being given for ideas, rather than implementations. Patents used to be given for a specific solution to a specific problem, with vague patents the system favors profit over innovation. The patent office needs to adhere to the rules, rather than throw out the system completely.

And please, no nonsense about how people will not think up new things with no patent protection. In large areas of technology and industry this is simply not true.

People think up new things all the time. The problem is getting that knowledge shared, and actually realizing those ideas as a product for public consumption. Often ideas will get to a point, then they require investment to be fully realized as an actual product.

Re:innovation

[mOdQuArK!]

The problem now is that many patents are overly-broad; they protect ideas, not implementations as patents originally were meant to protect.

Any system based on suppressing the activity of others will almost inevitably end up in a similar state, as the users of the system keep pushing the envelope as to what can be brought under their control.

If society wants a mechanism to encourage innovation, then it should encourage innovation directly (for example, hiring people to do the R&D that will benefit society but which is not cost-effective for private entities to finance), not use a backwards, half-assed mechanism which is designed to suppress the kind of activity that society wants to encourage.

Re:innovation

[servognome]

Any system based on suppressing the activity of others will almost inevitably end up in a similar state, as the users of the system keep pushing the envelope as to what can be brought under their control.

You mean just like the restrictions of government itself. It's obvious that corporations have too much influence, does that mean we abandon our republic and switch to anarchy? No we seek to ensure the system works as originally outlined (eg campaign finance reform).

If society wants a mechanism to encourage innovation, then it should encourage innovation directly (for example, hiring people to do the R&D that will benefit society but which is not cost-effective for private entities to finance), not use a backwards, half-assed mechanism which is designed to suppress the kind of activity that society wants to encourage.

There is a gap between academia and the market, which is difficult to bridge. Personally I don't want government micro-managing the direction of science.
For example fuel cells have been around a long time, why don't we have them in our cars? Because a significant resource investment needs to be made to create something that can be mass produced for the public.
If goverment was in control would we invest in fuel cell energy research or improving the efficiency of oil based energy? Well we know what way the current administration would go. On the flip side a private companies can justify an investment of millions of dollars a year in such research if there because there is an expectation of profitable return.

Re:innovation

[CodeShark]

Agreed.

Re:innovation

[bit01]

You've got religion; repeating the feel-good PTO gospel without evidence.

I repeat: There is no scientific, objective evidence that patents overall do more good than harm. Given the billion dollar impact that patents have on industry that is criminal.

Stop thinking in terms of patents and not-patents, a false dichotomy. There is a universe of possibilities that we've barely scratched the surface of. IP law is a product of the mind and can be anything we want it to be. Let's get it right. Huge numbers of technology practitioners, not lawyers and PTO employees who have a vested interest, are saying that patents are broken. I think that's pretty good evidence for at least a fundamental rethink.

Yes, overly broad patents do cause more harm than good, but its a problem with the patent reviewers, not the overall system.

Nonsense. That is merely a small part of the overall problem. Even if it were the whole problem it is humanly impossible for a small government department to assess all human knowledge for prior art. Anybody who claims otherwise is lying. Only a scientist working in a very narrow area for a lifetime can do that and even then they make mistakes all the time.

Other fundamental problems with the patent system include the complete non-recognition of independent re-invention by the PTO, inventions whose time has come, the PTO's messed up understanding of what an idea is, their complete non-recognition of how much investment is required in an idea and their legal, category, all-or-nothing reasoning that maps poorly to real life.

Not really, computer chips have dozens of patents, cars have patents, even shampoo bottles and showerhead designs have patents. I wouldn't call these things niche products.

Yep, and those products progressed well long before the patent mafia got in on the act. A classic example being the Kilby patent where the industry was progressing massively before it was resolved. Patents do nothing now except act as road blocks to progress.

People think up new things all the time. The problem is getting that knowledge shared, and actually realizing those ideas as a product for public consumption. Often ideas will get to a point, then they require investment to be fully realized as an actual product.

Yep, and patents more often than not block the promulgation and spread of knowledge and investment. Most ideas have no patent protection and yet are invested in and spread just fine.

e.g. I have the idea of opening a hardware store in a particular small town. Nobody's had that idea before after some investment it turns out to be successful. I think I should get a patent on that idea so nobody can open a competing hardware store in that town or towns of that size. Why not?

Fact is, the mere act of investing in a product is often enough to deter copy cats. The originator is first in the market and can have dominant market share as a result. No government intervention in the citizen's business needed.

---

Scientific, evidence based IP law. Now there's a thought.

Re:innovation

[servognome]

repeat: There is no scientific, objective evidence that patents overall do more good than harm.

And there is none to the contrary.

Huge numbers of technology practitioners, not lawyers and PTO employees who have a vested interest, are saying that patents are broken. I think that's pretty good evidence for at least a fundamental rethink.

They say the process is broken, not the fundamental idea

Only a scientist working in a very narrow area for a lifetime can do that and even then they make mistakes all the time.

That's if you continue with a single person reviewing, rather than an open source type approach. Let those skilled in the art review and approve.

Yep, and those products progressed well long before the patent mafia got in on the act. A classic example being the Kilby patent where the industry was progressing massively before it was resolved. Patents do nothing now except act as road blocks to progress.

And they are progressing at an exponential rate even with patents in place.

Yep, and patents more often than not block the promulgation and spread of knowledge and investment. Most ideas have no patent protection and yet are invested in and spread just fine.

General ideas which don't have protection, but individual implementations do.

e.g. I have the idea of opening a hardware store in a particular small town. Nobody's had that idea before after some investment it turns out to be successful. I think I should get a patent on that idea so nobody can open a competing hardware store in that town or towns of that size. Why not?

Because its an idea not an implementation. If you have a specific store design or approach to shipping, then that can be patented.

The originator is first in the market and can have dominant market share as a result. No government intervention in the citizen's business needed.

Except when the first to market is a small business, so rather than a billion dollar company investing in their buisness to acquire the technology (which happens all the time). The billion dollar company can copy with no investment and out manufacture/promote the smaller company out of business.

Re:innovation

[Muchsake]

And Look how Luke turned out. The prosecution rests.

oh so 1996

[griffster]

http://graphics.stanford.edu/projects/lightfield/ If you've attended siggraph for the last 8 or 9 years you yawn with me.

Re:oh so 1996

[Anonymous Coward]

...and the other 99.9% of us, who haven't, can be very interested by this article.

However, I'm sorry that slashdot hasn't been perfectly tailored to your needs. I'm sure Rob & co will get right on to that!

Re:oh so 1996

[Anonymous Coward]

By the definition of news, this isn't.

Re:oh so 1996

[griffster]

>However, I'm sorry that slashdot hasn't been perfectly tailored to your needs. I'm sure Rob & co will get right on to that! You sound like an expert - I'd suggest you learn to login.

Re:oh so 1996

[Anonymous Coward]

I'm sorry that slashdot hasn't been perfectly tailored to your needs.

Indeed, Slashdot seems to be lately tailored to overused jokes rather than informative comments, which, as we have seen, are now labeled "Offtopic". Perhaps there should be just one topic: "Soviet Russia".

Re:oh so 1996

[Anonymous Coward]

Quote from the abstract of the paper: "This paper contributes to the theory of photograph formation from light fields." Apparently he knows he didn't invent it. He's contribution, yet again from the abstract, "The main result is a theorem that, in the Fourier domain, a photograph formed by a full lens aperture is a 2D slice in the 4D light field." which has nothing to do with the paper from 1996. Plus, the guy came up with a camera that could capture the light field in one shot instead of requiring an array of photographs. If you've been to siggraph for the last 8 or 9 years, you should know better.

From the stanford websire, you can see the image of the array of photograph needed in 1996 (taken either by an array or cameras or a single camera manipulated by a robot arm): http://graphics.stanford.edu/papers/light/coherenc e_preview.jpg

It's a pitty people are modding you informative, your post is ignorant at best.

Re:oh so 1996

[griffster]

Actually no, I've been following light field rendering for sometime... I direct you a Stanford paper on the light field camera from 2002 http://graphics.stanford.edu/papers/lfvc/wilburn-l fcamera-spie02.pdf So, is the press release a "break through" or incremental?

Re:oh so 1996

[convolvatron]

you're right. lightfields are cool. and they are very old news.

but its nice to actually build a compact instantaneous lightfield
capturing physical artifact, dont you think?

i worry though about the impacts on resolution. its a bit more
information than a 2d image, and the sample i saw shows it

Re:oh so 1996

[k98sven]

Oh wow, you heard about it in 1996. Good for you. But why does that deserve a high moderation?

It's still a recent result (page says april 2005) and in case you missed it, it's the same researcher (Ren Ng) that's mentioned on that Siggraph page.

They've presumably made progress in 9 years. That isn't worth reporting on?

Re:oh so 1996

[Anonymous Coward]

In Soviet Russia, the camera focuses YOU!

Re:oh so 1996

[Anonymous Coward]

Actually this is not the same thing as the research done way back when.
The algorithm used to generate the image is what is interesting: it allows much faster refocusing of the image.
Before the process to generate the image from the camera data was extremely slow: O(N^4).
But these guys used a different algorithm which has a slow preprocessing phase and a fast refocusing phase: O(N^4 * logN) and O(N^2 * logN) respectively.
If you want more info there is a link to the fourier slice photography page from the stanford site.

Re:oh so 1996

[ichigo 2.0]

Hey, I hate Soviet Russia jokes you insensitive clod!

Re:oh so 1996

[Anonymous Coward]

it's not anything new, so no it's not worth reporting on. it's just more of the same. very yawn inducing.

therefore, parent poster's high moderation is very deserved and will remain.

What kind of focusing?

[Dekortage]

I'm curious... how adjustable is the post-processing focusing? E.g. depth of field, f/stop, etc. Do you basically get to adjust ANY of that after the image is recorded?

Re:What kind of focusing?

[GoRK]

If you think about how this type of camera records an image you will see that depth of field is actually just a byproduct of the current photographic process and f-stop is a method of controlling that byproduct. So in answer to your question, yes, you can adjust depth of field in post processing by changing the focus curves.

Re:What kind of focusing?

[EternityInterface]

Except the image will just be goo since it's a compressed JPG at 70 quality and 4:4 subsampling. (No, this is just a whine that all cheap cameras don't have .png option, and I wouldn't see why with new tech this would change, and yes, I'm just whining)

Re:What kind of focusing?

[Zakabog]

Did you even read the article? It's not meant to work on low resolution poorly compressed consumer grade pictures. It's more for raw images, and it says 8MP is too low and 16 is good, so that's just a handful of pro cameras and they all take raw images.

Re:What kind of focusing?

[GoRK]

WTF are you even talking about? The camera captures a 4D light field. I would imagine that the image data is absolutely immense which is why the resolution of the prototype is so small. If you need to generate a 3MP 2D result you'd be capturing something on the order of 25 terabytes of raw data according to some simple back of the napkin calculations. It has no comparison to JPG or any other image format, though I suspect that if this technique wants to becomes commonplace in consumer cameras someone will develop a lossy compression format that severely reduces the storage requirments. The data, after all, ought to be highly suited to compression.

3d Images

[Anonymous Coward]

I wonder if the image data gathered by such a camera could somehow be transformed into basic 3d depth information. If so, this could be the beginning of 3D imaging for the consumer.

Re:3d Images

[griffster]

There was a demo by Sony at GDC 2005 where they had a next generation "eye-toy" that could (essentially) extract a Z buffer with the captured image. they had some very cool demos... the most memorable was a virtual butterfly that flew around the head of the demonstrator and then landed on his arm :)

Re:3d Images

[Wilson_6500]

Why not?

Take each focal plane (let's say one plane of "best" focus). Throw the pixel RGB values there into a 2x2 matrix. Now take the next plane and lay it behind in the same way--2x2x1. Repeat for "n" focal planes giving a 2x2xn image. Now the challenge is displaying it.

"Say Sayonara to Blurry Pics"???

[5, Troll]

This is great technology but the author of the article used an incorrect title. Blurry photos are almost always caused by camera shake, not focusing on the wrong subject in a depth of field situation.

This technology doesn't do anything to prevent camera shake. Most modern cameras are extremely good at autofocusing on the correct subject in a short depth of field situation. The camera designed by the Stanford guys is an amazing invention and will revolutionize action, sport, and scientific photography (especially at the macro level) but it will do nothing special for the consumer who simply doesn't understand that the longer the exposure the more likely the blur from camera shake.

Re:"Say Sayonara to Blurry Pics"???

[Anonymous Coward]

And especially spy and security work. Take a single picture down a crowded hallway. Postprocess it so get a clear image of every single individual regardless of distance from the camera. Same thing over an area, just go ahead and shoot the picture. We'll post process clear shots of everyone in the camera's field of view.

Taking a picture of some foreign government's area, just go ahead and shoot a single shot and get your ass outta there. When you get home, we'll refocus it over a broad range and examine the entire area.

Re:"Say Sayonara to Blurry Pics"???

[dgatwood]

There are already plenty of medium- to high-end lenses with optical image stabilization. That's a solved problem, at least if you're willing to pay the price. One would hope that such technology will become more commonplace....

Re:"Say Sayonara to Blurry Pics"???

[winkydink]

You can already do this. It's called stopping down the lens.

Re:"Say Sayonara to Blurry Pics"???

[fireman sam]

It article states "This property allows us to extend the depth of field of the camera without reducing the aperture, enabling shorter exposures and lower image noise". This in itself will reduce the effects of shaking.

Re:"Say Sayonara to Blurry Pics"???

[Digital11]

Which would normally cause undesirable camera-shake in such a situation due to the decreased shutter speed

Re:"Say Sayonara to Blurry Pics"???

[altstadt]

The shake will be particularly bad when you hold the camera upside down (when shooting vertical format) as shown in the article. Your hands are supposed to be under the camera, not over it.

Conventional wisdom says that the slowest speed you can shoot 35 mm film hand held is at the reciprocal of the length of the lens. If you are using a 50 mm lens, that's 1/50 sec. If you are using a 200 mm lens, that's 1/200 sec. I have found that if I hold the camera properly (both hands under it) I can usually shoot at about 4 times that exposure time (e.g. 200 mm at 1/50 sec) and get a good shot. I have taken lots of tight indoors sports shots in dim light where the people are very blurred due to their motion, but the background is sharp.

Re:"Say Sayonara to Blurry Pics"???

[HuguesT]

Read The Fine Article. The proposed solution is almost the exact opposite to the one you are talking about.

The microlens approach doesn't require any moving part, it allows not only to refocus but also to extend focus as if one were using a very high F-stop for large depth of field, without the associated noise due to low light.

The downside is that it requires many pixels to produce a good image, but as the pixel count grows exponentially with time as per Moore law, it will soon be a winning proposition, even with cameras in mobile phones.

On the other hand optical stabilization is as expensive as ever, requires many moving parts and does not allow focus extention.

Re:"Say Sayonara to Blurry Pics"???

[ceoyoyo]

Optical image stabilisation only gains you a bit. Not sufficient to clean up the poorly-lit-rooms-with-an-itty-bitty-little-lens shots.

Re:"Say Sayonara to Blurry Pics"???

[ceoyoyo]

But if you make your pixels bigger then you get more sensitivity, reducing camera shake. ;)

Re:"Say Sayonara to Blurry Pics"???

[hhawk]

If you look at the sample images, you can see the type of image that will be "helped" by their camera...

Auto focus cameras have to focus on something... and many times I've had them focus on the wrong thing. There isn't really anything you can do at that point except reshoot.. or use the system such as they describe.

This would be of great value to me, I have many photos where the image is otherwise perfect except the focus point is off.

Re:"Say Sayonara to Blurry Pics"???

[mattthomas]

What about altering the ISO speed, which most digital cameras can do instantly? It's not that hard to buy faster film.

Re:"Say Sayonara to Blurry Pics"???

[Ceriel Nosforit]

Konica Minolta has got an anti-shake system that moves the CCD instead of the optics. They even have cheap cameras with this feature.
An experienced photographer can shoot at 1/4 second shutter speed using their AS. Normal flash photography is maybe 1/150 second, for comparison.

Unfortunately Konica Minlota hasn't got any SLRs above six megapixels, but if they did this new lens system could allow for some very nice night-time photography when in conjunction with their anti-shake system. Something to keep one's eyes open for, for sure.

Let's just hope Sony doesn't screw KM over, since they seem to have some business going together. DRMed RAW files would just spoil this all.

Re:"Say Sayonara to Blurry Pics"???

[PigleT]

> Blurry photos are almost always caused by camera shake

Erm... where do you get that idea from? Have you ever used something other than a point-'n'-shoot camera in automatic mode where it tries to maximize DoF all the time?

Me, I'm wondering if this will facilitate software that implements tilt after the event, too.

Re:"Say Sayonara to Blurry Pics"???

[yulek]

thanks troll for plagarizing my wired comment directly...

~yulek aka the popmonkey

Re:"Say Sayonara to Blurry Pics"???

[Carnildo]

This is great technology but the author of the article used an incorrect title. Blurry photos are almost always caused by camera shake, not focusing on the wrong subject in a depth of field situation.

I doubt that. About 90% of the blurry photos I've got are due to the subject moving. The other 10% are caused by the camera autofocus picking something towards the edge of the frame rather than the subject, or the camera focussing on the background instead of the subject in a macro shot, or the autofocus failing entirely because the scene isn't well-lit enough.

(As a side note, do you know how hard it is to convince a bird in flight to hold still while you take its picture?)

Re:"Say Sayonara to Blurry Pics"???

[Digital11]

Sure, but you can't push the ISO too much without getting undesirable noise.

It's fun.

[Duncan3]

Having seen this stuff in action first hand, it's cool as heck. Also a tad scary. Miniblinds not closed 100% then you can see in, tree in the way no problem.

Basically what we see as solid with 2 eyes, may not be solid at all. So much like the IR/UV cameras, this new toy has a dark side.

Re:It's fun.

[mabinogi]

More to the point - what we see with one eye.

With two eyes you can already see the effect - holding a hand in front of your face doesn't stop you seeing what's behind it until you completely cover both eyes, etc.

Re:It's fun.

[mrmojo]

Even with one eye you see the effect. Leonardo da Vinci noted that if you hold a pin close to one eye, it disappears because it's significantly smaller than your pupil.

Re:It's fun.

[Anonymous Coward]

and wtf was da vinci doing sticking pins in his eyes?

Re:It's fun.

[Felmir]

What implications does this technology have for image recognition and processing? Does anyone know enough about both image / object recognition and photography to offer some insight?

Would it require rewrites of current algorithms or only simple modifications? What would be the added benifits?

Re:It's fun.

[Splab]

I guess thats why he figured it also works with only one eye ;)

Re:It's fun.

[Alsee]

I bet it had something to do with the great(x20) grandfather of Mr. Goatse.

-

Re:It's fun.

[quisph]

So much like the IR/UV cameras, this new toy has a dark side.

Sweet.

Just like in movies and TV!

[malraid]

Have you seen how in movies and TV they can zoom and then sharpen any image using software? We'll it seems that technology is finally comming to real life!

Re:Just like in movies and TV!

[Adult film producer]

ya, reminds me of a CSI (Vegas) episode. They're examining some blurry security video from a crime scene, one of the crime technicians says 'focus in on his eye!'... so they zoom up and guess what, they can read a newspaper or something reflecting in the perps eyeball.. lol

Re:Just like in movies and TV!

[Hast]

When you sharpen an image in Photoshop (or Gimp) you are only increasing the contrast. While this has the effect of making the image "pop out" more which looks sharp it can't extract more detailed information from the image.

Re:Just like in movies and TV!

[Anonymous Coward]

You're an idiot. No, really. You're stupid.

Re:Just like in movies and TV!

[HuguesT]

No, sharpening in Photoshop also enhances high frequencies in the image, which gives the impression it's more in focus because it does have sharp artifacts in it.

Re:Just like in movies and TV!

[ceoyoyo]

It is possible to truely sharpen an image (a little bit) using deconvolution. But it won't get you CSI style sharpening.

I wonder about the effect on resolution and sensitivity of this technique. Modern autofocus on little point and shoot cameras is pretty good at what it does... a lot of blurry pictures are due to camera shake because poor lighting requires long exposure times. The article even mentions "poor lighting" although it somehow assumes that this technique will fix that too.

Re:Just like in movies and TV!

[DMNT]

While this has the effect of making the image "pop out" more which looks sharp it can't extract more detailed information from the image.

No, you can't, and you're completely right about that. But in the out-of-focus picture all the information is (mostly) there, and the question is how to transform that desired subject in focus. If you have the convolution model, you can write an inverse function using Fourier transform. For a quick mathematic formula, see here and scroll a little down until you find section "Wrong lens focus". There's also software that does the trick. The downside here is that it increases the noise as you can see on the focusmagic examples, but nevertheless it's possible and already done. The original work that represents the degradation model is from the 60's.

Re:Just like in movies and TV!

[Anonymous Coward]

ENHANCE!

Some other info from MIT

[zegebbers]

can be found here

The "root" of innovation

[Anonymous Coward]

"The patent system was not designed to protect business methods, such as completing a sale using n clicks instead of n+1."

How abot the "rootkit" business method?

Re:The "root" of innovation

[Lord Ender]

No, the copyright system (in the form the DMCA) was (or is now) designed to protect those assholes.

Re:The "root" of innovation

[plalonde2]

The rootkit method? Of course not; it's just a trivial variant of the "disdain your customer" business method.

Intangible Pluralistic Brain-wave Phrenology

[millennial]

I can make up really technical sounding names, too!

Re:Intangible Pluralistic Brain-wave Phrenology

[Lil-Bondy]

Phrenology... on one of the bungie's (creators of halo) employee/company cards, the owner is an "Applied Phrenologist".. i laughed when i found that out

Obligatory...

[millennial]

Countdown until you hear about someone using one on CSI: 5... 4... 3...

Re:Obligatory...

[StikyPad]

Real tech on CSI? Better start from something a lot higher than 5, or start counting fractions like parents trying to scare their kids.

Amazing

[unik]

This is quite an amazing and exciting (discovery?). Should be very helpful to sports photographers.

You Don't Say

[Anonymous Coward]

I'm sick and tired of the sensational headlines here on Slashdot. I prefer my news delivered in the most boring manner possible.

At what price in resolution?

[ottffssent]

The linked article comments that there's an effective loss of resolution, but goes no further.

Obviously taking a camera that's designed to record light intensity and modifying it to record light intensity and direction isn't free. In the worst case, you're decreasing your effective resolution by the number of new lenses, or by a factor of 90,000. I don't think that's quite what happens though, because many of these lenses will be recording essentially the same information, and while only one may be perfectly focussed on part of the frame, nearby lenses can probably contribute color and intensity information as well. If we assume a 2Mpixel image is "good", the article's comment that the student's using a 16Mpixel camera but that an 8Mpixel camera might be good enough seems to support a roughly 4x to 8x decrease in effective resolution. Can the poster who claims to have heard the actual discussion at Siggraph comment?

That's a high price to pay for not having to use the viewfinder. It's cool tech, and I'm sure there are practical uses for it somewhere, but I don't think consumer cameras are the place for it just yet.

Re:At what price in resolution?

[Anonymous Coward]

I don't think consumer cameras are the place for it just yet

Is that you, Dvorak?

Re:At what price in resolution?

[mrmojo]

I'm one of the guys involved in this research at Stanford. I just posted below explaining that due to noise issues, you don't really fundamentally have to lose resolution, as you can just use an equally sized sensor with cell phone camera size pixels instead of good camera size pixels, and when you add up pixels to create a focused image, the noise goes down to match noise levels in a good camera.

Re:At what price in resolution?

[HuguesT]

From TFA, you end up with as many pixels as there are lenses, i.e. 90,000, and indeed the sample images in the article are about 300x300, i.e. 90,000 pixels.

Re:At what price in resolution?

[ottffssent]

Are you sure? That doesn't make sense (except perhaps for initial research). Each of the hundreds of sensor elements under each lens will be imaging a slightly different object, at a slightly different angle. Not taking this into account could explain the softness in even the in-focus parts of the images though.

Re:At what price in resolution?

[StikyPad]

You may think your +1 post has been overlooked, but the real explanation is that we at /. don't like our discussions clouded by stuff like facts, especially from firsthand accounts. We prefer all our discussions to be based on heresay or speculation grounded firmly in imagination.

Getting the least out of your 16MB camera

[Tsar]

Yes, the plenoptic camera has some neat benefits, including the ability to reconstruct the field of view from the perspective of any point on its objective lens. But for the image to contain all that information, it by necessity does NOT contain information that it otherwise would--in this case, resolution.

Look at the sample images. Even the sharpest-focused regions are soft-focused. This is a 16-megapixel camera with an effective resolution less than 1/3 that of VGA. Granted, the images can be refocused and depth information can be extracted, but do you really want to have to buy a 188-megapixel plenoptic camera to get sharp 1-megapixel images? Is focusing really that hard?

Re:Getting the least out of your 16MB camera

[jettoblack]

I don't think this technology will ever be useful to typical snapshooters or photographers. For the former, just stick an f16 lens on a small-sensor digicam and you'll have near-infinite DOF for most shots, and the latter generally prefer narrow DOF and know where they will be focusing before pressing the shutter.

However, I imagine this might be useful for some kinds of analysis photography, especially when dealing with high-speed motion. Those kinds of shots usually require a large aperture to gather enough light (due to the very high shutter speed), meaning a very narrow DOF. If you're shooting something which is very expensive or happens only once (say, explosion anaylsis, freezing bullet-time action, etc.) and getting the right focus or wide DOF is critical, this could be very useful.

Re:Getting the least out of your 16MB camera

[blincoln]

Is focusing really that hard?

It is if you want the exact same shot with different depths in-focus.

TFA has some good examples of this in the form of splashing water, but imagine how much more information you could extract from e.g. the Zapruder film if it had been captured this way. It's not like you can ask Kennedy to go back for another take.

Re:Getting the least out of your 16MB camera

[ozzee]

I suspect that there is now use for that 300-megapixel sensor.

Considering that we already have gigabit memory chips, I can see that it's plausible to have gigapixel light sensors (sometime in the future).

Given that 4 (good looking - low noise) megapixels would satisfy most non-professional type photographers, I think this is not that unreasonable to sacrifice pixel count for ease of use.

Re:Getting the least out of your 16MB camera

[asuffield]

Think special cases. Security cameras. Unrepeatable events. Quasi-autonomous robots.

In general, focussing *is* that hard if you're not there or don't have time to do it.

Re:Getting the least out of your 16MB camera

[Anonymous Coward]

Oh cripes, MEGAPIXELS MEAN NOTHING.

I have a 3 megapixel Fuji S1 pro SLR and I take photos that make most 5-11 megapixle cameras look downright horrible... Yes even a Canon 20D looks like crap compared to my camera.

The lens in front of the sensor is 90% of the picture. I print fantastic 11X17 full color prints that dazzle and enthrall my customers from that lowly 3 megapixel camera.

I simply have a $1300.00 lens on the front of it which kicks the crap out of everything else. (so the 20D would look better with my lens, but no camera without a removeable lens will)

If you put an extremely sharp and good image on the CCD you get a better picture. Megapixels mean nothing to clarity and quality.

Everyone but the dumbest know this...

Re:Getting the least out of your 16MB camera

[zippthorne]

Obviously you've never tried any of sony's dissapointing consumer cameras. A low-noise CCD is critical. (or at least a CCD capable of recording the "dark frames" whenever you want)

so I would peg the percentages at: good lens 100%, good CCD 100% since bad either will guarantee terrible photos.

Re:Getting the least out of your 16MB camera

[Viceice]

You obviously aren't a photographer. Many award winning shots are accidents. Taken during such times where, for example, a photographer is running for his life in a hail of bullets, simply pressing the shutter as he runs, not even looking in the viewfinder.

What I'm getting at is, some moments happen in literally in the blink of an eye and they only happen once in a lifetime. So in that split second where you are trying to take a shot and have no time to double check, won't you be sorely disappointed if your ticket to a Pulitzer was ruined by the wrong f-stop setting? Or the wrong focus?

Back in the day of 8mb CF cards, a 6megapixel 6mb RAW was insane. But in this day of 4GB CF and memory prices what they are, 6 or even 16 mb RAWs are but a drop in the bucket. Heck even with today's memory capacities, if you had a camera that produced a 188mb RAW, it'd still be perfectly acceptable to any photographer, considering the possibilities for photography this new technology gives you.

Re:Getting the least out of your 16MB camera

[Wheely]

Try finding a lens that can resolve anything near that amount of resolution on a 35mm format sensor.

Current guesses suggest that 10MP is about the safelimit on DX sized sensors unless you use professional lenses and anything over 15MP would finish those off too.

35mm format sensors like the 16.7MP Canon has already require high quality lenses to resolve enough detail. 22MP would seem around the limit for those.

The mega pixel race is nearly finished as the lenses are becoming the limiting factor.

Re:Getting the least out of your 16MB camera

[ipfwadm]

What I'm getting at is, some moments happen in literally in the blink of an eye and they only happen once in a lifetime. So in that split second where you are trying to take a shot and have no time to double check, won't you be sorely disappointed if your ticket to a Pulitzer was ruined by the wrong f-stop setting? Or the wrong focus?

There are a lot more types of photography than just photojournalism. And yes, there are certainly cases in, say, wedding photography where if you mis-focus an important shot, things are going to be bad. But that's why wedding photographers actually look through the viewfinder before they shoot. It's also why you pay for an experienced wedding photographer who is very unlikely to screw up, rather than asking your Uncle Jimmy who just bought a new-fangled D-SLR to do it. My guess is that given the current resolution trade-offs, most pros would not go for this system. I sure wouldn't. The percentage of times I take a shot that's out-of-focus due to mis-focus is very small, and I'm sure pros take even less mis-focused shots than I do.

Heck even with today's memory capacities, if you had a camera that produced a 188mb RAW, it'd still be perfectly acceptable to any photographer, considering the possibilities for photography this new technology gives you.

Maybe... it depends on how fast it can actually write that 188MB of data to the card, or how big the memory buffer is so I can still take more shots while it's writing that data.

Can't get something for nothing

[Deep Fried Geekboy]

The more potential focal points you want, the less resolution you can have for any particular one of them. You have to record information for all possible focal points on the CCD. Conceptually it's no different from, say, dividing the CCD into four parts and recording an image with a different focal point on each of the quarters, then post processing to combine them as required. I think. So photographically speaking the image is degraded compared to just getting the focal point right in the first place. Which isn't to say there aren't cool things you can do with it.

Re:Can't get something for nothing

[Fat Cow]

This method allows you to open up the aperture to get more light while retaining large depth of field.

An alternative is using deconvolution to retrieve the focussed image from the defocussed one. You'd have to know the point spread function, which I think you should be able to derive from knowledge of the optics.

Re:Can't get something for nothing

[Astro Dr Dave]

This method allows you to open up the aperture to get more light while retaining large depth of field.

Yes, but you lose resolution in the process. You could instead bin the pixels to get greater sensitivity, and simultaneously close down the aperture to get greater depth of field.

An alternative is using deconvolution to retrieve the focussed image from the defocussed one. You'd have to know the point spread function, which I think you should be able to derive from knowledge of the optics.

In general, that approach doesn't work. The out-of-focus parts of an image can't be recovered; there is a loss of information that can't be restored. Deconvolution is a nonlinear process and generally does not have a unique solution. And even if it did, the PSF for camera lenses varies dramatically with both focus and distance, and the photograph doesn't record the distance from the focal plane to the object seen at every pixel in the image.

Re:Can't get something for nothing

[Breakfast Pants]

Deconvolution works great for telescope pictures of stars... and that's about it.

Re:Can't get something for nothing

[ceoyoyo]

Deconvolution works a little bit on photos too. You just have to be realistic. You're not going to pull off CSI style effects. It's a wee bit better than Photoshop though. Deconvolution isn't about getting back information that's not there -- it's about reweighting what you do have to correct for some non-ideality (the part that does the convolution). For hubble that was a flawed lens. For a SLIGHTLY out of focus photo, it's the photographer. ;)

Re:Can't get something for nothing

[Graham Clark]

It's getting increasing use in light microscopy too.

Why stop at three dimensions?

[ian_mackereth]

Give the lenses a coating of resublimated Thiotimoline
http://en.wikipedia.org/wiki/The_Endochronic_Prope rties_of_Resublimated_Thiotimoline

This will not only ensure that your photo of Auntie May is in focus, but the camera will make sure that the image is captured at a time when her eyes are open and she's smiling.

Plenoptic eyeglass

[rewinn]

The next step is to pair the cameras and the LED image emitters, similar to night-vision goggles, to make a really kewl pair of corrective lenses. Truly the ultimate nerdwear!

Re:Plenoptic eyeglass

[Winterblink]

Oh you can do better than that. :)

How about a future where this technology can handle the focusing in realtime, extremely fast, and at high enough resolution to go into a bionic eye replacement. No more nerd goggles at all.

Re:Plenoptic eyeglass

[rewinn]

>bionic eye replacement.

Not a bad idea, but that requires two huge leaps in technology: implants that the body won't reject, and connecting to the optic nerve ... which as Wallace tells Grommit, would be "...just a harmless bit of brain alteration."

In contrast, going from a plenoptic camera to glasses requires no real fundamenta advances; just good solid moneymaking engineering based on existing nightvision, 3-d, and (assumed) plenoptic tech.

There is an old adage about photos.

[elgee]

If it is blurry, it is art.

Will this mean the end of artsy photos?

Re:There is an old adage about photos.

[laughingcoyote]

Yes, because in addition to the neat new lens, the package also includes a guy who'll come by and hold a gun to your head, forcing you to sharpen the image.

Re:There is an old adage about photos.

[Danimoth]

No, this way all images can be art. Anoying picture of you and Aunti May? *BLUR* ART!

Megapixels

[abertoll]

Finally, something more to cameras than megapixels.

Re:Megapixels

[(negative video)]

Yeah, kilolenses (kl). Yippee.

Fly Eye from the Fly Guy

[Tablizer]

using about 90,000 microlenses

Patents brought to you by the fly people.

Cool.. but...

[Anonymous Coward]

Very cool, but it's an other step of loosing privacy.

RTFA so early 90's

[Anonymous Coward]

Like, the sigfificance is that now you can get more mini-lenses on the CCD.

Whoa there, slow down a second...

[Anonymous Coward]

it's a prototype. They're just testing out the technology to see if it can be done. It can have many uses (the other reply to your post offers a few) but no one is saying that it's perfect yet. When you add in one respect you almost always lose something in another, it may take a while to get them both working well at the same time.

that is awesome

[drewxhawaii]

"...describing experiments on a compound, thiotimoline, that was so soluble that it dissolved in water up to 1.3 seconds before the water was added."

i burst out laughing when i read this. i love it

Re:that is awesome

[ian_mackereth]

The fact that I could remember the name of the substance 20+ years after reading the Good Doctor's story should tell you something about how much I liked the concept!

Sure, but...

[Theatetus]

Can they put it on a big frigging shark?

Re:Sure, but...

[Sarcastic nerd]

No, no, the laser story is down the page a bit, keep going.

Absolutely Amazed

[rhyder]

Just when I thought the last big thing in photography would be the multi-snapshot, multi-angle camera that allows action to be frozen, the spun around... Then there comes this. And I agree with the prior post, it is more or less obvious once you read the technique, but it never would have occured to me. No matter how many times I look at a focal-plane diagram.

What was that 3-D freeze and rotate camaera called? It has a bunch of camera on an arc, and it was used in the Matrix.

Re:Absolutely Amazed

[Twisted64]

Bullet time? That's what they liked to call it. The reason being, a bunch of cameras would be placed where the motion was to go. Then, a long unbroken string would be attached and run through the trigger of each camera. The other end of the string would be attached to a subsonic bullet, hence "Bullet Time." When the moment came, the bullet would be fired, triggering the cameras in sequence. The "Bullet Time" sequence gunmen would attempt to kill the directors as they fired, but failed miserably, hitting instead the story and script-writers, resulting in two miserable sequels.

Re:Absolutely Amazed

[Fembot]

I believe what you are refering to is this:

Timetrack

So much for thinking 32MB was decent storage.

[illumina us]

From the looks of it, this takes hundreds of images and stores them in one file. Then uses software to create a single, desired, image. This means that conventional storage will no longer be enough, for while one image now takes up several hundred kilobytes to a couple megabytes (JPEG compression), this new method will take up hundreds times that size. >.

Re:So much for thinking 32MB was decent storage.

[Anonymous Coward]

Uh... No.

The image will take up the space needed to record how ever many pixels are in the sensor. The problem is that you lose the uniqueness of the information at each pixel and thus lose resolution as has been mentioned elsewhere here.

Re:So much for thinking 32MB was decent storage.

[Fresh Mike]

Fool. The file is still unique, because the plenoptic system uses a normal digital camera, with a normal sensor. The filesize is exactly the same as if the photograph were taken without the microlenses. It doesn't change the quantity of information captured, it modifies the organization and use of the captured information i.e. for each array of pixel sensors behind one microlens, we choose to store information about the direction of the incoming light rays instead of a finer resolution without regard for direction of the light rays.

Re:So much for thinking 32MB was decent storage.

[Anonymous Coward]

It seems that you did not understand what the process behind this plenoptic camera is.
What they do is record a SINGLE image which contains not just pixel colors but also directions. They then use an algorithm that takes that information and converts it into the final image.
The original image should take up rougly the same amount of space as a regular 16MP image but the quality of the final photo is less.

A blanket solution.

[Belseth]

You could always go to a pin hole camera and eliminate the problem entirely. Alright so you'd need 10,000 ASA film or .1 lux for video but focas would never be an issue. Always been a massive fan of pin hole cameras. It's also a handy trick for those of us with failing eyesight for reading fine print. There have also been lensless cameras that use a rotating slit. There are 360 cameras that use the principal. Fun with optics

Re:A blanket solution.

[HuguesT]

Except real-world pinhole cameras are always blurry instead of always sharp...

This is due to the fact that the pictures sharpen as the size of the hole diminishes (i.e. large hole = very blurry, small hole = less blurry), but there is a limit to how small the hole can be until it becomes counter-productive due to diffraction.

Large-area plenoptic plates

[Namarrgon]

So, now that we have arrays of microlenses & software-based focusing, why do we need a conventional lens at all?

With some improvements to the manufacturing technology, we could have a revolutionary camera based on a large, flat plate of microlenses, scaled up to whatever the manufacturing allows, or even multiple plates tiled together, with the software allowing for the seams. And that's it - you could stick them to anything - phones, for example, or walls or whatever.

The linked paper shows how to use Fourier processing for (re)focusing the data on any point, but with a large enough plate you could also get enough parallax for a limited 3D result.

It'd also avoid the limited-aperture optics issues we're running into with sensor size vs lens size. You can scale them (or tile them) up, collect a lot more incident light & get a lot more resolution too, without having to shrink the sensor elements further or use a ginormous conventional lens. The bigger you make these things (or the more that are tiled together), the better & more flexible your result. There's more data to process, of course, but that's what Cell is for, isn't it?

Re:Large-area plenoptic plates

[HuguesT]

Even better than a Cell, a Retina.

Essentially a CMOS Retina is a massively parallel computer with processing logic at each and every pixel location.

Re:Large-area plenoptic plates

[Wheely]

Bit irritating when you wanted to zoom in or out though.

More like oh so this past summer at SIGGRAPH 2005

[baxissimo]

Sounds like this is a popularized writeup about the work that was just published at SIGGRAPH in July. So it's more recent than 1996.

Application for holographic movies?

[Dr. Spork]

The best 3D technology we have now still sucks. Its basic premise consists in showing different images to each of your two eyes, but those images are taken with standard photo equipment, so some portions are blurry and others are sharp. This really makes people nauseous at 3D movies. It sucks, really takes away the realism! So I wonder if a retinal projection of a 3D movie shot with these cameras could make the focus more natural. Basically, it would read the depth of your retinal focus and adjust the image focus on the fly so that it looks just right. Of course, each frame would have to encode all that depth data, and that would be lots of data. But who thought that Blu-Ray would be our last movie format? I say we don't rest until movies are 100% holographic, and this may just be the technology that makes it work!

Re:Application for holographic movies?

[Anonymous Coward]

Uhhhh, the whole idea of 3D is to show each of your eyes different things, if you saw the same thing, it would be 2D. I don't think you've been to a 3d movie for a long time, Red Blue glasses have been completely removed and replaced with lcd shutter glasses in digital theaters and polarized ones in others. There is no inconsistency between right and left channel in say an imax movie, because the imax 3d camera is a very expensive piece of equiptment that exposes equally in 2 lens, creating the 3D effect, it's all very precise. Now watching that 3D Imax film with a bug the size of a bus in it can trip some people out and make you all fucked up and dizzy, not due to the crappy quality but more the photorealism messing with your head. Crappy 3D results in headaches usually, which is usually a side-effect of the flickering.

Re:Application for holographic movies?

[Dr. Spork]

I've seen plenty of polarized 3D movies, in IMAX and other settings. They don't look real. They make the assumption you will be focused on a certain depth of field. If you choose to focus elsewhere, you can't. The thing you're trying to look at will be blurry no matter how you tweak your eyes, because it was out of focus on the film. If a display technology could sense where your eyes focus and then sharpen up that field, then it would look like a real landscape. This would require lots of extra data and some extra sensors, but the result would be perfect 3D. I say it's worth trying!

FUCK YOU MAN

[Anonymous Coward]

What the fuck? 3D sucks? You suck!

Smarter thinking

[goombah99]

Even better you can do this with a single lens and an ordinary camera. just take one photo in focus and one photo at a different focal plane. Voila. all the information you need to reconstruct the phase front for perfect focus. Bonus is that if it is ion focus then one of the photos is probably good enough right from the start without any signal processing. This whole 90,000 lenslet camera seems liek the hard way.

Re:Smarter thinking

[goombah99]

By the way, in case somebody doubts this, I did this in 1996 and used my mac laptop to compute the refocused image. I was experimenting with imaging from earth imaging sattelites. I still have the code, which actually was a relatively slow script written in Igor Pro.

Re:Smarter thinking

[Anonymous Coward]

Hey, do you have any additional information about this? It sounds pretty interesting. Any web page or something like that?

I couldn't google it up without a term to define it :)

Re:Smarter thinking

[(negative video)]

Even better you can do this with a single lens and an ordinary camera. just take one photo in focus and one photo at a different focal plane.

That doesn't work so well for video.

Re:Smarter thinking

[ghoti]

Paul Haeberli did this in 1994 - it's a neat idea for some experiments, but not a very practical solution.

Re:Smarter thinking

[goombah99]

I don't have a web page. The chief Terms to google for is phase recovery. In the monochromatic regime look at synthetic aperature, speckle imaging as google terms. A lot of the eray work on this (70s and 80s) happened at defense contractors in the US and the soviet union. Amusingly the literatture with the mathematics was published WITHOUT mentionain the application. It was just math on how to recover phases from two systematicially transformed images (i.e. out of focus). The first people to use this in the non classifed world were the solar astronomers.

Re:Smarter thinking

[goombah99]

Cute method and quite practical. But not the same thing as true phase recovery. Once you recover the phase of the image from the two photos you can synthetically compute any other possible image taken with any optical train along the same optical path. That is you can refocus at any plane or distort it imaginary ways (like changing the aperature). This method you link to is a effieicnt trick for hybridizing two images to get the best (infocus) parts of both into a single image.

You don't really lose resolution

[mrmojo]

I'm one of the guys who works on this stuff at Stanford. I should point out that it's not fair to say you lose resolution, because good cameras have large pixels to reduce noise over a finite exposure time. Lightfield cameras, because they add up a whole lot of individual pixel samples to produce an image pixel, can get away with much much smaller pixels, because the noise goes down as you sum up the pixel values.

The best way to think of it is take a standard good quality camera with big pixels, subdivide each pixel into a grid of 12x12 or so tiny pixels - more like the size of pixels in cell phone cameras - and put a microlens over it. You get the same spatial resolution as the good camera, roughly the same noise characteristics, and the ability to refocus and pull other light field tricks like hitchcock zooms.

You just have to be aware that treating the data as a light field it's very noisy, like a crappy cell phone camera, but when you add up pixels to make a focused image, the noise drops back to regular good camera levels.

It's just harder to deal with the amount of data you get off a large sensor with tiny pixels, and they're also harder to build, but neither point is a showstopper and these are mere engineering issues...

Re:You don't really lose resolution

[(negative video)]

The best way to think of it is take a standard good quality camera with big pixels, subdivide each pixel into a grid of 12x12 or so tiny pixels - more like the size of pixels in cell phone cameras - and put a microlens over it. You get ... roughly the same noise characteristics, ...

The space between the pixels tends to be hard to shrink, so as you add pixels an ever-increasing fraction of the image sensor tends to become dead zones. Using Foveon-style stacked detectors instead of a filter mosaic would, of course, help quite a bit.

A question: can you refocus colors independently to correct chromatic abberation of the lens?

Re:You don't really lose resolution

[ottffssent]

That's not completely fair. If I understand you, what you're saying is that in fact you DO lose resolution, but the loss in resolution can be compensated for by higher-resolution sensors and because you don't have to increase the physical size of the sensor, the production costs won't go up too much. I don't know enough about CCDs and CMOS sensors to know what the probable increase in cost would be, but it sounds fairly minor. At least for CPUs, I know die size is a stronger indicator of manufacturing cost than transistor count, though the latter obviously plays a role.

The other problems that you've swept under the rug seem to me to be more important, at least in the near term. If you take a CCD and replace each of its sensor sites with a 12x12 array, as you suggest, you're talking over a 100-fold increase in the data to be processed. While I haven't read the technical papers on the subject, it seems like the processing is more complicated than the processing that goes on in a standard digicam, which probably means at least a 200x increase in processing requirements. If you wait for Moore's law to save you, that's 10 years. Budgeting for a more expensive image processor will shave maybe a year or two off that number, but it's still fairly long-term research.

You could reduce the processing needed in-camera by storing closer-to-raw data and doing the processing at a workstation later, but then you have the problem of a data stream that's ~100x as large. Even with very fast flash storage, that would take 30+ seconds to write a single image, and you could only fit a few onto a 1GB card. Also, you introduce the problem that the photographer doesn't get feedback as to what he or she actually shot, and unless you can also post-process to correct for motion blur, abberation in color, etc. you still need that functionality.

It all sounds interesting, and I applaud research into what useful things could be done with likely future technology, but (and maybe I'm misreading the situation) it sounds like the core research is being cast as a thing we could be doing RSN, which I highly doubt. As a technique to make use of sensor densities that would normally exceed the capabilities of the lens they're attached to in order to do something useful, this is interesting. As a technique to be applied to today's or near-future sensors and cameras, I find it less interesting.

Re:You don't really lose resolution

[mrmojo]

Good point, and an excellent question. We're looking into it.

Re:You don't really lose resolution

[Anonymous Coward]

Maybe this is completely wrong, but if the end result is kind of "averaged" over sub pixels, wouldn't that reduce chromatic abberation anyway? It seems like if the subpixels are interleaved, then with the aforementioned 12x12 grid, the abberation would be no more than 1/4 of a pixel (or something like that).

Re:You don't really lose resolution

[Crspe]

Sure its fair to say you lose resolution - the output image of your 16MP camera is 0.08MP. How do you want to pass this off as "no loss of resolution???"

Maybe you can reduce the size of the pixels without increasing the noise in the final image, but be warned - as these pixels get noisy, the quality of your refocusing will drop. There is always a cost of reducing pixel size.

It also doesnt work to make really tiny pixels, you basically cant go below 2umx2um for a pixel, otherwise the pixel size is too close to the wavelength of light (0.7um) and then they stop working.
A 12x12 array of 2umx2um pixels - thats BIG and physics will stop you getting them any smaller ... inside a mobile phone you wont get more than 0.025 megapixels!! - useless! In a big expensive 35mm camera, you wont even get 2MP - who is going to pay $5000 for a 1.5MP camera?

Re:You don't really lose resolution

[ceoyoyo]

Except that recombining images gives you about a sqrt(N) reduction in noise, while partitioning your pixel will give you about an N increase in noise. Each subpixel of your 12x12 paritioned pixel will be about 1/144th as sensitive to light as the full pixel and summing up the parts will only get you back to 1/12th the sensitivity of the original.

Re:You don't really lose resolution

[ashot]

who is going to pay $5000 for a 1.5MP camera?
I would gladly pay for it if it had the refocusing ability. So would any sport photographer. Imagine, you can just snap away and focus later! Forget sports, add weddings and any other even semi-critical photography and this becomes extremely useful.
Not to mention the fact that many modern digital cameras (*cough* canon *cough*) have a difficult time with focusing accuracy with ultra wide lenses, this is especially true for digital enlargements. So, perfect focus will actually get you a bit of resolution in many cases (where a regular camera misfocuses even slightly)

Also, near-night photography is now possible on digital without a flash in high-ISOs, creating many situations where the only impedement to taking a nice photo is the inability to focus.

Re:You don't really lose resolution

[foniksonik]

If you're not concerned with focus... then wouldn't a simple thumbnail of the photo be sufficient to provide feedback 'on camera'? Have a separate low quality or small sized image captured in a more 'traditional' digital method and use that as the preview.... while storing the final image as raw, which is what you really want anyways.

Re:You don't really lose resolution

[mrmojo]

Think of it this way, adding up the 12x12 pixels under a microlens is exactly the same as having a single larger sensor there, in terms of the physical process occuring. Sensor noise per pixel is not significant compared to photon noise.

Therefore, because your sqrt(N) argument is correct, partitioning the pixel should only increase noise by a factor of 12, not 144.

Re:You don't really lose resolution

[ceoyoyo]

Photon noise: http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/f ip-Photon.html

Note the last line though -- "For very bright signals, where T exceeds 105, the noise fluctuations due to photon statistics can be ignored if the sensor has a sufficiently high saturation level."

Consumer digicams aren't counting individual photons. They most certainly are dominated by electronic noise, as anyone who does astrophotography with them knows because the colder your sensor, the less noise you get. So when you make your pixels smaller you're amplifying a smaller signal that has a constant or increased noise level. Adding those signals back together after amplification is not the same as amplifying the higher SNR signal from the larger pixel.

Re:You don't really lose resolution

[stienman]

A cheaper fix for the low resolution problem would be to mount the ccd on a servo stage that can vibrate it. Take 3-5 pictures in a few mS while the stage is vibrating. Since the camera is creating the vibrations it knows where the stage is and post-processing can extract 2-3 times more resolution out of the camera than the sensor's native resolution.

-Adam

Why .avi?

[EternityInterface]

Would o been nice instead of .avi to have it in Macromedia Flash, where you can change zoom as you wanted. (And compared to 77mb - with vastly lower filesize?)

"4D light field" sounds interesting though, if only for the "4D" part. I'm thinking and I can't really come up with anything it would be useful for, for the average photographer. I am though reminded of a cam (by the company that did Painter / Kais Power Tools [and eventually sold it all away to concentrate on some web technology...]) which took 2 photos at once, 2 lenses, was a horizontal thing, anyway, the 2nd photo was a 3d image - a radar kinda image. It's a shame we have 5.1 - 3d sound, and still only 2d images - it seems the only technology is in higher reses, not higher dimensions?

Re:Why .avi?

[EternityInterface]

Ok - woah. The .avi illustrates what "4D" means - compositing all the views gotten - a virtual viewing space can be created - you can look to the sides and also back and forth - low/wide angle (Do note that this is in low amounts - kind a bullet-time lite, to impress friends?) - and having the whole image in focus.

Now, the Wired article - could herald the end of fuzzy, poorly lit photos. POORLY LIT? I saw no mention of that improving.

Is the Wired article for real? The student, Ren Ng, ran out of patience with taking pictures the traditional way - adjusting the distance between the camera lens and sensor or film before snapping each shot. Dude, autofocus?

A camera equipped with Ng's device wouldn't need the motors that focus lenses, so the camera would have fewer moving parts. This sounds good though, because autofocus can be a slow and noizy annoyance.

Reinnovation

[Wilson_6500]

All sorts of optical technologies have been around for a long time, and only nowadays are people saying "Of course! Add this to this and I get a great thing!"

Off the top of my head... parallax barrier cameras and newer parallax barrier 3D displays... same thing with lenticular screens... Um... This was so much easier a year ago.

At some point, extra resolution is pointless

[supersat]

Ren Ng gave a talk on this work last April at the University of Washington, and IIRC, he argued that the resolutions of CCDs are increasing exponentially, and after a certain point, the extra resolution is pointless, so why not use that extra resolution to encode additional information not normally captured? I believe he also speculated that the rate of resolution improvements isn't nearly as high as it could be, and technology that could take advantage of the extra resolution would motivate development of extremely high-resolution CCDs.

Re:At some point, extra resolution is pointless

[ceoyoyo]

That point has been reached. Four or five megapixels is about the most you're likely to get out of a point and shoot quality lens. But the standard sensor these days seems to be six, seven or even eight. That's just a waste of memory.

Re:At some point, extra resolution is pointless

[TheLink]

I believe if you make the pixels smaller they get more prone to noise.

If you keep the pixels big and add more pixels, the array gets bigger (and more expensive).

But yeah, I guess it'll still be cool.

I wonder if that's how many insects see the world. Maybe the conventional wisdom is wrong about insects needing to be pretty close in order to see things in focus.

I already find it hard to track some flying insects with my crappy mammalian eyes, but it seems dragonflies manage to do that pretty well.

Has some useful features

[Wilson_6500]

Is focusing really that hard?

I was working a year ago on a 3D imaging system that used parallax barriers. We would've killed to have had the kind of continuously-focusable output this camera could produce; one of our biggest problems was deciding what part of the image to focus upon, and then keeping the camera and light steady between shots--especially outdoors. We would have huge problems on cloudy days because the ambient light would change so much between shots at different depths of focus.

Combine this continuous focusing with dynamically sized parallax barriers, and the results might be interesting.

3D imaging

[Wallslide]

If one can get multiple focal points from just a single exposure, it seems like this technique could speed up systems which take multiple "slices" of an object to build a 3D representation of that object's structure. Brain scans for instance.

Re:3D imaging

[ceoyoyo]

Except that brain scans are completely different than photography. Photography captures light rays that are unmolested between reflecting off an object and entering the camera. X-rays capture rays that are modified (attenuated) the entire time they pass through the object. CT uses clever reconstruction of these projections taken from different angles (all around a circle, not just from one side) to figure out what sort of object created them. MRI uses an entirely different method that is really a kind of interferometry.

Movies- multiple effects in a single take.

[Anonymous Coward]

This is huge. Imagine this in the video cameras used for movies. The cinematographer can now try multiple focus effects in a single take!!!

X-Ray enhancement?

[nacturation]

Could this be used to sharpen what we see in an x-ray image of a person? Take an x-ray of the whole body and then refocus to concentrate on one particular cross-sectional plane?

Re:X-Ray enhancement?

[HuguesT]

Not with this camera, as X-Rays are hardly ever focussed (they don't bend easily!). Here is an image of a rare and expensive X-ray focussing mirror. You have to use grazing incidence for it to work.

Medical X-ray photographies are simply taken by placing film (or these days a digital detector) behind the body and lighting with X-rays. No focussing is involved.

Re:X-Ray enhancement?

[jayegirl]

Erm, wouldn't that be a CAT scan?

Re:X-Ray enhancement?

[Hal-9001]

Take an x-ray of the whole body and then refocus to concentrate on one particular cross-sectional plane?

Congratulations, you've proposed X-ray computed tomography, which has been used clinically for over 30 years!

Re:X-Ray enhancement?

[nacturation]

Congratulations, you've proposed X-ray computed tomography, which has been used clinically for over 30 years!

Thank you, I'll be here all week. Tomorrow night: using the combustion of petroleum products to produce locomotion!

Don't insects have prior art?

[ArsenneLupin]

Insects' eyes are made op zillions of individual "facets", with each its own microlens and microretina.

Idea has no practical application!

[Crspe]

I saw this article about a week back. I am quite sure that this will never see a practical application ... They take a 16MP input image to produce a 0.08MP output image!!! They are using a $15000 camera system to produce images one quarter the size of VGA!!! Say what you want, but there are better ways to improve DOF.

They reduce resolution by a factor of 180, but only improve depth-of-field by a factor 7. This is particularly silly because the only reason they have a bad depth-of-field is because they are using a huge expensive sensor. If they would switch to a small cheap sensor like you find in any cheap digicam (1/1.8"), they would get the same improvement, and save $14800.

The light-performace of this small sensor would be just as good as their large one - if you use the same huge pixels that they do (to produce a 0.08MP image), you will get the same low light performance.

If you want more details on why this idea has no use, check out this thread:
http://luminous-landscape.com/forum/index.php?show topic=9354

Interesting article, no practical application.

Re:Idea has no practical application!

[Cederic]

If the idea has no practical application, why are you complaining about a specific implementation of it?

Ten years ago digital cameras were pretty naff. Low quality images, expensive to make.

Now they're better than film for many uses.

Another ten years of improvement and suddenly we have 16 megapixel images using this technology. It's still better image quality than your screen can display or your printer produce on paper, and you've been able to play with the focus, the depth of field, everything else this gives to you.

Get off the internet, you're clearly not qualified.

Re:Idea has no practical application!

[cyclomedia]

an i expect a decade or two ago people like yourself were saying that it'd be impractical to attempt to build a device that fits comfortably in the palm of your hand that allows you to talk to other similar devices anywhere in the (urbanised) world, play several hours of stored music, take and transmit photographs instantly (without all that waiting a week to get them developed, and in color too, unlike yer average fax machine).

if this technology is combined with the High Dynamic Range technology and storage sizes jump up by a factor or two (which they're doing all the time anyway) i can see the day when digital photos are the same actual output res as they are today but when you take a photo of someone stood inoors by a window with a bright sunshine-lit citiscape outside that both can be in focus and both can be well lit. frickin about time, after all my eyes can do it. or at least, they make my brain think they can

Re:Idea has no practical application!

[XMode]

The Laser also had no practical applications when invented. The world doesn't stand still at today's technology, and what might be useless today may be a revolution in years to come. Good luck to them.

Re:Idea has no practical application!

[Crspe]

If the idea has no practical application, why are you complaining about a specific implementation of it?
umm, I guess it was to point out that a $10000, 4lb/3kg, 0.08 megapixel camera is not very practical. You planning on buying one?

Another ten years of improvement and suddenly we have 16 megapixel images using this technology
NO, we are not going to have 16 megapixel versions of this. The laws of physics dont change very quickly, the wavelength of visible light is pretty stable and as a result, the minimum size of pixels are not going to get much smaller than they already are in mobile phones! This technology cant go beyond 1 megapixel, not because we dont have the technology yet but because the laws of physics will limit this.

Now, what exactly were your qualifications? They certainly dont include anything about physics, electronics or image sensor design! So shut up and let people who actually know about this technology write something relevant about it.

Re:Idea has no practical application!

[Crspe]

No, a decade or two ago people like me were saying that nothing can go faster than the speed of light, and guess what - it is still true. This is not a question of whether they can improve the technology, it is limited by the laws of physics.

Re:Idea has no practical application!

[Hal-9001]

They reduce resolution by a factor of 180, but only improve depth-of-field by a factor 7. This is particularly silly because the only reason they have a bad depth-of-field is because they are using a huge expensive sensor. If they would switch to a small cheap sensor like you find in any cheap digicam (1/1.8"), they would get the same improvement, and save $14800.

Actually, depth-of-field is entirely controlled by the imaging optics. Using a larger or smaller array increases the field of view, but has no effect on the depth of field. One can improve the depth of field by stopping down the lens, but this also trades away lateral resolution (assuming the optics and sensor have equal resolution at full aperture, you're now wasting some of the resolution of your sensor), and it throws away light (which is never good for signal-to-noise ratio). Focal error also trades away lateral resolution and wastes the resolution of your sensor. Admittedly, this technique also trades away some of the resolution of the sensor for its refocusing capability (which can become extended depth of field with appropriate image processing), but it does so without throwing away light. If you're limited in what you can do with the optics, this technique lets you circumvent those limitations to some extent with the sensor and with image processing, so I think it's a bit hasty to say this technique can never find a practical application.

Re:Idea has no practical application!

[Anonymous Coward]

Don't ever become an entrepreneur! I've never seen a comment displaying a more limited sense of vision.

Re:Idea has no practical application!

[Goaway]

Ok, the wavelength of light is around 0.5 micrometers. Let's say we want to fit a couple of those in a pixel, so 1 micrometer pixels. That gives you one square millimeter per megapixel. You can fit quite a number of square millimeters in your average-sized CCD sensor, no?

As was pointed out elsewhere, the limiting factor for pixel size is actually noise. And since this method combines many pixels anyway, you can get away with noisier individual pixels.

I'm not sure why you seem to be personally insulted by this technology, with all the shouting and ranting.

Re:Idea has no practical application!

[Shotgun]

I am quite sure that this will never see a practical application ...

Can you say 'targetting computer'?

Not only will it have a pratical application, it has application in one of the few areas where money is practically no object.

Removing fingers?

[GhaleonStrife]

But will it take your finger out of the way of the lens when you snap the picture?

I laughed my %$* off when I read this...

[OneSmartFellow]

The mini lenses measure all the rays of incoming light and their directions of origin...

That's quite an accomplishment

prior art

[paulwomack]

I'm not enough of a scientist/mathematician to know the answer, but this sounds very similar to "wavefront coding" from the U. of Colorado.

This is already being used commercially, and the company was formed in 1996 - so I assume the research substantially pre-dated that.

http://www.cdm-optics.com/site/

        BugBear

Obtaining a focused-everywhere image?

[4D6963]

That thing reminds me of High Dynamical Range imaging when it gets to the software-focusing thing (as HDR imaging uses software-controlled-exposure). In HDR you can "mix" together pictures at different exposures to obtain one good looking picture. Could you do the same with that thing to obtain a picture where all objects at all distances are focused, by "putting together" parts of the image when focused? If so that would be kind of interesting to obtain out of it something that's not normally possible, unless you use an impossibly small lens.

Necessity and frustration

[srussia]

are the parents of inventions. The patent system is just the pimp. "The student, Ren Ng, ran out of patience with taking pictures the traditional way" This is the reason why he came up with the idea, not the patent system. What does "protecting" an idea mean anyway? What patents do is protect a monopoly on an idea.

I had this basic idea myself

[eXocomp]

About 7 years ago, I was fantasizing about the Ultimate Camera, and realized that it would record simply the location and direction of every photon entering its sensor. I didn't know how such a thing could be accomplished, but I was sure technology would eventually reach that point.

So this is awesome! I imagined this idea becoming practical only in the far future. Now it looks like it merely needs more megapixels.

Ideal for Ground-based Telescopes?

[fygment]

Could the principles be used to correct for atmospheric distortion of stellar images in ground-based telescopes? It would seem so if the image was spread over more than a few pixels. Could it also be used to correct for optical flaws in a lense?

file size ?!

[ericcantona]

using this sort of method may lead to a massive increase in image file size.
this is not necessarily a terminal problem
crucially, by what factor are files bigger ?, I presume O(n)
I hope not O(n^2), or worse

btw, well done graphics.stanford.edu, you have survived slashdotting !!!
... must be running freeBSD

We need finger removal.....

[SomeoneGotMyNick]

My Aunt(-in-law) has a HUGE collection of family pictures. But more than half of them have her finger in the middle. She can't be convinced to get an SLR or digital camera. She likes her instamatic. I think she buys her film by the three-pack so she can get a full usable (one roll) set of prints from them.

what if you had more than two eyes?

[peter303]

We discussed this in a thread about 2004 SIGGRAPH conferencew. Several researchers were asking the question "what can you do with dozens or hundreds of simultaneous imaging systems?" These would be be line or plane array of cameras. People at SIGGRAPH were demonstrating real-time 3D telkevision. Also you can image a 3D volume simultaneously and select any subsection of volume. Stanford's classic exmple is "x-ray vision" through a hedge. A single camera/eye can only see a few scattered holes through a hedge. But a planar array of cameras is likely to find some light path to every object on the other side of the hedge and successfully construct the whole scene.

Commodity cameras, lense, and computer clusters make the construction of real-time planar imaging systems with the reach of professor's research budget or a clever hobbyist.

Why stop at planar depth of field?

[geek42]

The nifty thing about refocussing light fields is that you don't have to do it the same way a camera does. A camera always has at most a single plane in perfect focus, and that focus drops off as you get further from the plane. But some recent research has shown that you can place an extended region of depths in perfect focus, with a falloff in focus as you get further from that region.

You could, for example, put a 2-meter region of depths containing your subject in perfect focus, and have a sharp transition to blurriness at the edges of that 2-meter region.

Works fine for video

[goombah99]

Sure this is easy to do with video. Just a beam splitter that adds extra path length to a second CCD or to half of the main CCD

Re:Works fine for video

[(negative video)]

I stand corrected. Good idea.

What I don't get from the (real) article

[Vadim Makarov]

is whether the invention enables to take images past the diffraction limit of the conventional optics or not. Suppose you take a tripod-mounted image of a static object at f/45 with long exposure. The resulting image has a great depth-of-field but the details of objects within this depth-of-field are uniformly not sharp, due to light diffraction on the small aperture. Now, would the new design in principle allow to take the image with the same depth-of-field but less diffraction blur for objects within the depth-of-field?

Original paper by Ren Ng that describes the device

[Cochonou]

Available here. I think this will explain the device much better than the other links that were posted.

yes, removing fingers

[r00t]

As long as the finger isn't fat enough to cover the whole lens, yes, you could remove it.

Re:Original paper by Ren Ng - dupe

[Cochonou]

Dupe.
This is what you get when you don't click carefully on each one of the links. :)

Plenoptic Zapruder film?

[Tsar]

...imagine how much more information you could extract from e.g. the Zapruder film if it had been captured this way.

Actually, it would likely have been rendered completely unusable. The Zapruder Film was taken on a Bell & Howell Zoomatic using Kodachrome II Super 8mm safety film, which had a stated resolution of 63 lines/mm. That gives each frame an effective resolution of about 85,000 pixels.

A plenoptic camera of the type described here, using this film as the detector, would have an effective resolution of less than 480 pixels, or about a 25x19-pixel field. The result might let you determine roughly how far away the Lincoln Continental was from the camera, but that datum wouldn't be 0.00001% as valuable as the details you sacrificed to get it.

Not sure you get it...

[Animaether]

If you feel that the 'problem' is that there is a depth of field.. then yes, a pinhole camera will do the trick - to a point, as the pinhole gets too small you get diffractive issues - but that's another topic.

However, you can never -change- the focus on a pinhole camera image properly. The reason for this is that you have no depth information whatsoever. The best you can do is manually mask out bits of the image and (gaussian) blur it to your liking.

This technology, however, allows you to create perfect focus all over... or just on what's near, or just on what's far. You can even, as they show in the video, change the point of view - if you so desired. Or perhaps increase / decrease the perspective (simulate a closer distance with a larger field of view / a larger distance with a smaller view of view)

So if you don't find unfocused parts in itself a problem, just the distance at which the focused part is, the tech is for you. If you just want near-100% sharp pictures... use a camera with a 'fast' lens, large sensor and a high an f-stop (small an aperture) as you can get it. Or, also use this tech, with a much cheaper sensor/etc.

One FANTASTIC Possibility

[STLSegway]

A few years ago I went to Las Vegas and stay at Luxor hotel. I found out that the hotel has an iMax theater, so a friend and I thought we'd stand in line for about an hour to see "Mario Andretti's Super Speedway". (http://www.audio-ideas.com/reviews/dvd/super-spee dway.html) It had been my first time in an iMax so the thrill was out of this world. So, large, so close, great sound, etc., but there was one thing seriously missing and think this lens technology is the closest thing to fix the problem.

The problem is that when a camera focused on a subject, observers were forced to look at it and lot allowing a single observer to enjoy viewing other parts of the video. For example, Andretti was cruising down a long country road and the camera position was about 100 yards away at low altitude. The car and Mario were the subject; however, the scenery beyond the road appeared beautiful, but out of focus.

Here are two scenarios of usage:

    1) Using the statue pictures on wired, imagine that you are looking at the far statue. Light-weight Eye-tracking glasses could track the x/y coordinates and quickly adjust the "z" focus my switching the image (on the video end, not the obsrever's glasses). When your eyes move to the right to the near statue, the software switches to that image. Very simple actually, but, this idea is only good for one observer. Think of the virtual reality possibilities!

  2) Now back to Mario's scenario and the most expensive (I would think) solution. The observer is a "lens shifter" if you will. Essentially, the image the observer is seeing is either all out of focus to serve as a "common focus" or just focused on the subject. (I think all out of focus will work better.) A headset is worn by the observer containing thousands, if not millions, of small lenses and eye tracking capabilities. Auto-magically, the lenses adjust x/y viewpoints of the eye.

Perhaps we've been thinking too much about putting TVs on our heads rather than lenses and using a commonly-focused scene as a base. Kudos to Ng, this is the beginning of a great era in photography and movie making.

Re:One FANTASTIC Possibility

[STLSegway]

I found this, perhaps they should talk to Ng. http://www.smarteye.se/home.html

Its got that shiney-invented-in-1994 look

[AmericanInKiev]

Looks like an idea that been around the block for a while.

1994