Lens-Free Flat Cameras Make Use of Pinhole Technology (npr.org)

RhubarbPye writes: As reported on NPR, "Engineers in Texas are building a camera that can make a sharp image with no lens at all." By incorporating millions of individual pinholes with photoreceptors and postprocessing software, this camera system has been reduced to minimal thickness. Cameras in the wallpaper? A new phase of wearable cameras? What other applications for this technology could be developed?

Comments

Re:I must be missing something...

[Frosty Piss]

So, let's see if I get this right. They rediscovered something, that everyone from the 1990's and 80 years prior learned to make as part of science class...and simply applied modern technology to it.

No, you didn't get it right. But that's not surprising, clearly you didn't read the article.

Dynamic range?

[RightwingNutjob]

TL;DR. Most uncooled camera chips give you maybe 10 or 11 bits of dynamic range, and light is subject to Poisson noise, meaning the brighter a pixel, the noiser it is in absolute (not relative) terms. If you have to solve a big giant matrix inversion to do the job of a collimating lens, you're composing each pixel as a sum of many others instead of just itself, some of them being way brighter than the reconstructed image, meaning your reconstructed pixel is always noisier. Cool idea, and certainly has its applications, but the best images will always come from big fat optics.

Re:Dynamic range?

[Arkh89]

No. First, on these you are mostly limited by the thermal noise of the sensor which is miles above the photon noise for this application. Then you are still thinking that a pixel receives the same flux (power per surface area) as a traditional camera. This is not correct as each of the pixels collect flux from a much larger angular portion of the scene (due to the lack of optical focusing).

Re:Dynamic range?

[Ungrounded Lightning]

If you have to solve a big giant matrix inversion to do the job of a collimating lens, you're composing each pixel as a sum of many others instead of just itself, some of them being way brighter than the reconstructed image, meaning your reconstructed pixel is always noisier.

Not really.

When you average a large number of samples the noise tends to partially cancel out while the signal keeps adding up. Though the noise goes up with more samples, the signal goes up more, improving the signal to noise ratio. Even if you end up adding in some bright signals, with extra noise, that's still stomped by signal when you have enough samples.

Re:Dynamic range?

[Anonymous Coward]

No, the original poster was more correct. They're not averaging together a bunch of pixels, but applying an inverse matrix , which will weigh pixels differently, and quite frequently can involve very high weights assigned to noisier signals. This can result in an emphasis that amplifies noise. There is a lot of work done on different ways of clipping or modifying such matrix equations to make it slightly less accurate in an ideal world, but much less noisy in the real world.

Also, no averaging of noise will happen if you try to produce images with similar pixel count to the number of detectors. And if you do use more sensors than resulting pixels, the averaging increases signal to noise ratio only in general when the noise is independent, but in this case there will be some strong correlations that can lower that signal to noise ratio in some cases.

Re:I must be missing something...

[gl4ss]

I suspect that if you put a lens on it, you would end up with a light field camera.

aaaaanyways... this is wikipedia on light field camera: "A light field camera, also known as plenoptic camera, captures information about the intensity of light in a scene, and also captures information about the direction that the light rays are traveling in space. One type of light field camera uses an array of micro-lenses placed in front of an otherwise conventional image sensor to sense intensity, color, and directional information. Multi-camera arrays are another type of light field camera. Holograms are a type of film-based light field image."

which sounds almost exactly like a variation of this. it's the same exact concept.

Millions of pinholes * near zero size = ??

[erice]

Nothing in the article talks about what the resulting aperture is. To get a reasonable exposure time, you need to capture adequate light. Cameras in cell phones already suffer because their lenses are too small to capture enough light. Is this scheme worse because it lets less light through or better because a larger "lens" is practical?

The Irony

[djinn6]

I see all of the photos in the article were taken with a conventional camera, complete with lens blur.

Re:I must be missing something...

[shanen]

Actually it's more like the compound eye of insects, but 'wired' differently.

VR is the hot new thing

[fragMasterFlash]

So make a headband 360 degree camera to capture video you can view with your VR headset. Also, make police officers wear these instead of the silly badgecams they currently use.

Nothing is new under the sun?

[Lluc]

The way the NPR article describes this, it is no different from Uniformly Redundant Arrays, i.e. Coded Aperture Imaging: see https://en.wikipedia.org/wiki/... If you look at the 1998 paper, "Uniformly Redundant Arrays" by Busboom et al, the first sentence describes work from the 1960s:

Coded aperture imaging (CAI) (Mertz and Young, 1961; Dicke, 1968) has matured as a standard imaging technique in X–ray and Gamma-ray astronomy. It is capable of combining high angular resolution with good photon collection efficiency by using a mask consisting of transparent and opaque elements placed in front of a position sensitive detector (Figure 1).

So is the only innovation here using more pinholes, more pixels, and more processing than were around in the 1990s?

Re: Nothing is new under the sun?

[Lluc]

Yeah, reminds me of sparse imaging / compressive sensing too. Nothing new here except perhaps they aim for a consumer product. I would still welcome any device capable of imaging at high resolution and wide dynamic range without a lens. Lenses are so medieval, we should do better.

As their paper points out, these pinhole systems really suffer from the fact the the imaging aperture is TINY. This produces a low resolution, high depth of field image like a camera with a very "high / slow" F/# .
High degrees of multiplexing also require high bit depth measurements and low noise, driving sensor cost up.

Scalable

[DrYak]

Is this scheme worse because it lets less light through or better because a larger "lens" is practical?

Currently, it's worse (TFA mentions quality similar to first gen webcams).
But indeed, that technology is really scalable. TFA muses with large surface flatcams.
(They mention walls of it. Or boxes/cylinder in the middle of which you put an object, etc.)

So the whole back cover of a smart-phone could be a giant pinhole array.
Such a large surface even if covered with only pinholes (and even if some of the hole might get obscured by fingers holding the phone) would gather much more light and information and could produce better image. Exactly as you say : "a practical larger len" but taken to the extreme

Massively parallel pinhole camera

[DrYak]

So, let's see if I get this right. They rediscovered something, that everyone from the 1990's and 80 years prior learned to make as part of science class...and simply applied modern technology to it.

No, you didn't get it right. But that's not surprising, clearly you didn't read the article.

What? Even TFA itself mentions that the researcher decided to go back to the classical pinholes as inspiration:

To design their camera, Baraniuk and his colleagues looked to the past for inspiration.

"Back to really the very first cameras, pinhole cameras," Baraniuk says.

Pinhole cameras have been here for quite a while. According to some scholars, they were first described by the Chinese philosopher Mo Ti around 400 B.C

Basically they've taken the pinhole camera, and decided "We have computing power !"
Instead of putting just one camera (which would definitely NOT gather enough light at this size), they're massively parallelized it. They've put millions of pinholes camera on the chip and are using post processing to reconstruct the final image out of the tons of tiny pinhole views.

In away their doing exactly what light field photography does, except sans the micro-lens array usually associated with the technology. They've put a pinhole mask instead.

But the parent poster is right, the research have decided to take a well known old trick (the century old pinhole camera) and put a new spin to it (apply tons of computing, thus using tons of pinholes in massive parallelism, enabling to do something like light-field photography, but with no lens at all).

Re:Massively parallel pinhole camera

[stevelinton]

Instead of putting just one camera (which would definitely NOT gather enough light at this size), they're massively parallelized it. They've put millions of pinholes camera on the chip and are using post processing to reconstruct the final image out of the tons of tiny pinhole views.

I think there are more pinholes of a variety of different shapes, and fewer sensor pixels than your description might suggest.

Re:I must be missing something...

[91degrees]

No. They applied new technology to something they already knew about.

A pinhole camera is educationally interesting but not much use for practical purposes. You simply don't get enough light coming though. It's easily solved by optics. Most cameras use a lens to squeeze more light through the aperture. But this requires a certain thickness. Not a problem if you can afford a few CM of thickness, but unsuitable for flat cameras.

You can also solve the problem by using lots of pinholes. But if you use a single CCD, you get interference from lots of similar but different images. So you need to use some pretty sophisticated computer processing to unmerge these images.

Re:I must be missing something...

[BarbaraHudson]

A pinhole camera is more than adequate for taking pictures of eclipses of the sun, etc. It's also good enough for taking stills (longer exposure time) or objects lit with a very bright light (like a flash, maybe?).

Re:It will still be a bit bulky I guess

[geantvert]

I don't think so. Each pinhole can produce an image corresponding to a wide angle which only depends of the size of sensor of its distance from the pinhole. If the sensor is very close to the pinhole(s) then the image will have a wide angle (up to 180 degrees).

 

Integral MicroOptics from the 1980's

[eastjesus]

Interesting work with a lot of unobvious possibilities. "Lensless" is a little misleading. Pinholes are just the center circle of a zone plate. Zone plates are lenses that work by diffraction instead of refraction. They look like a bulls-eye (see http://www.eastjesus.net/tech/... for a quick and simple primer). The diameter of the hole determines the focal length - hence too big OR too small leads to fuzzier images. The have a couple of big drawbacks - the focal length is a function of wavelength hence objects in the image have rainbow edges and the aperture of a focused pinhole is small (the f-stop). The effective f-stop can be increased at will by adding additional zones around the pinhole but zone plates that work by blocking areas can only achieve efficiencies of around 10%. That can be improved to around 90% or more by replacing the opaque zones with tapered phase-shifting zones. Back in the mid 1980's I worked with a similar technology using arrays of zone plates which we called Integral MicroOptics. We used arrays of micro-zone plates (and pinholes) to capture image data over large sheets (hence from many angles at once) and then reconstruct that data with full parallax in 3D and color, both stored and in real time and sometimes with some optical computing applied - all using passive devices! They were the equivalent of full color holograms using a subtractive technology instead of an additive one, hence no lasers were required and the more diffuse the light the better. It was amazing what could be done with thin flat sheets of plastic and printing but the technology of the day was too crude to get very far. Today much more could be achieved. If interested you can see the original paper from 1986 at http://www.eastjesus.net/tech/... (with updated graphics and a link to the original) and http://www.eastjesus.net/tech/... for some images created using the technology at that time.