New Sensor Has Real Per-Pixel RGB Sensitivity

jonr writes: "Well, the holy grail of digital photography is finally found. A company named Foveon have developed a sensor that captures RGB colours on each pixel. So what you say? Well, for the past 30 years (or since the CCD was invented) we have been using CCD with with red, green & blue sensors (or cyan/magneta/yellow) and then used software to figure out the real colour. But Foveon is the first company to deliver RGB-in-each-pixel sensor. For those of you who are not into digital imaging, this makes a lot of difference, it's would be just as revelutionary if somebody would make a flatscreen with a real colour pixels, instead of the RGB dots. dpreview.com has the scoop. (No, it won't mean the death of film, but I suspect we'll see dramatic improvement in quality)."

Comments

Screw resolution

[Reality Master 101]

More resolution, while nice, is not what digital photography primarily lacks. Light and shadow sensitivity is what really sucks with digital cameras. Film has a logarithmic sensitivity to light, while a digital sensor has a linear sensitivity.

Just out of curiosity, does anyone know of any technologies in development to give better light/shadow sensitivity.

Re:Screw resolution

[esonik]

Yes. At the University of Heidelberg, Germany, Physicists have developed a log-response CCD chip (covering 6 decades of intensity). They want to use it as the frontend sensor for their "tactile vision substitution system" (a machine enabling blind people to "see" with their fingers). The Log response is achieved by operating the sensor transistors in their non-linear range (very crude description, it's been a long time ago since I attended a talk about that project). Links to publications can be found following the above link.

Re:Screw resolution

[Romeozulu]

A bigger problem then number of pixels is quality of glass. Lens of lowend digital cameras stink. Even on higher end digital SLR, the quality of your lens has a huge effect on the quality of the image. But this is nothing new to photography. My point is, more pixels won't help if you're still using cheap PS lens.

RZ

Re:Screw resolution

[llamalicious]

Had you read the info on the site, you would know they aren't taking specifically about increasing the resolution.
They are increasing the amount of light captured per element.
Succinct overview: (Let's say we have a camera with 16 pixel (4x4 matrix)

In a typical mosaic CCD pixels would be the following:

gbgr
rgbg
grgb
bgrg

That's 8 green pixels, 4 red and 4 blue.
Thus 50% green, 25% red and 25% blue.

Now, after the image is captured, the camera has to do some serious image manipulation to translate the average color per pixel into an RGB value. Depending on the "correctness" of the algorithm used, you'll get all sorts of fun... low light noise, color shifts, purple fringing, moire patterns.. etc. and all that processing chews cpu time and battery life, and slows down the speed from picture to picture.
(Yes, some cameras have higher speed processors... just means yet more battery drain)

In their new method of capturing the layout is as follows:

aaaa
aaaa
aaaa
aaaa

In this case, all 16 pixels capture red green and blue. This means NO processing to be done to calculate color per pixel.
It also means a 2x increase in the amount of green light captured. And a 4x increase in the amount of both red and blue light captured.

More light being captured = more light and shadow detail.
Not having to process (interpolate) pixel colors = no more fringing, moire or other funkiness.

Now, that doesn't change the fact that digital sensors rely on converting a given set of photons into an electrical charge, and that's tough work... but having more light detected at each photo element is going to give you a LOT more light and shadow definition.

But, I digrees. I don't know of anyone pioneering new ways of doing ADC for photo elements.

Speaking of resolution

[Reality Master 101]

If anyone is interested how photography resolution compares to digital, I found a great link once about this: http://www.users.qwest.net/~rnclark/scandetail.htm

It's pretty eye-opening if you think digital photography is getting close to film.

Re:Speaking of resolution

[Matey-O]

You're neatly thrashing the resolution limitatations and missing it's benefits.

I'll direct you to Philip 'Ex-Ars Digita' Greenspun's more balanced review here: http://www.photo.net/photo/digital/choosing.html

(Barring the fact he's talking about older digicams, there's newer stuff on photo.net, and the theory on colorspace is valid.

Further, having dont both film scanning and digital, there's NO DUST ISSUES in a picture that starts it's life out as a digital picture!

Re:Pixel count in camera specs...

[damiangerous]

That is exactly right.

No, it's almost entirely wrong. The most common configuration for a digital camera uses what's known as a Bayer filter pattern for discerning color. Each pixel will only sense red, green, or blue (there are as many green as there are red and blue combined), and it will use neighboring pixels to extrapolate the true color. While it's true that some of the pixels are lost from the stated resolution (stated pixels actually refers to photosites), it's only about 200,000 in a 2mp CCD.

If I may out-geek the original article...

[kindbud]

Please check out superconducting tunnel junction technology, which is the basis for detectors that can measure the frequency of impinging photons. No need for separate RGB pixels - stacked or not - because each pixel can determine the exact frequency or wavelength of each photon it detects. You can take a spectrum and create an image in one exposure with one detector, without using any diffraction gratings or RGB filters.

What about CMOS?

[Zarathustra.fi]

Before we all go crazy with Foveon's buzz talk, I think we should see what CMOS cameras have to offer. Although not yet very mainstream, the CMOS sensors are in many ways superior to CCD stuff:

• Low noise, higher quality
  
• Lower light sensitivity due to bigger amount of sensors per pixel (no big ugly photodiodes)
  
• Much lower power consumption (around 1% of a CCD sensor)
  
• Easy fabrication process, since it's all about transistors
  

There are already some (very high-end) digital cameras using CMOS technology, and judging on the sample images I've seen, they are awesome. Take a look at the review of Canon's EOS-D30, for example.

Depth of Field Limitations? Lens Requirements?

[human bean]

Each piece of the spectrum being at a different depth, will the modern apochromatic lens (designed to focus all colors in the same plane) be needing an adjustment to work well with this sensor?

Due to the sensor thickness, is depth of field going to be restricted to smaller stops in order to have the entire thickness of the sensor in focus?

All i can say is..

[Brat Food]

WOW. I worked on a project trying to do some pretty accurate work with digital cameras, and I can tell you this... Until you spend around $20,000US, you will not even get close to your original. Heres an example.

The subject is a GretagMacbeth color checker (a bunch of square swatches of color with a black boarder)

With a pro-sumer camera, say around 3k, the image overall looks OK, but zoom in to any "grayscale" swatch, and you'll see that the image is still very much little RGB dots blurred together, and your grey never has all the same RGB vales as a true grey should.

As you go up to the 20k price range, a variety of tech is used to get more accurate color. The best I have seen was a back for a large format camera (can't remember the name for the life of me) that, when used in a studio setting only, could capture exact grey values for each pixel. What this means, is that if you took the captured image in to Photoshop, with no image correction, and you used the eyedropper over a grey swatch, your RGB values would read (x , x , x) over the whole swatch without a hiccup (1 pixel sample).

The camera achieved this by physically moving the CCD array so that it took something like 3 or 4 shots of the image (hence needing to be in a studio set up).

Now, a single CCD camera setup that can be used in the field, probably generating the same results as above, is going to be HUGE.

I don't know the target price range to start, but cameras using this tech, if it lives up to its promises will be HUGE in the pro photo field. Capturing a more true color vs. totally interpolated has enormous impact on color correction and manipulation images. In my experience, images for lower end cameras don't always manipulate in ways you expect because of the interpolated nature.

Actually... it is revolutionary

[ka9dgx]

This is revolutionary. There is no alignment issue to worry about with this chip, the automatic gain and channel amplifiers will all be right next to each other, so the color accuracy is going to be phenominal. I would like to see the response curves for the different layers. The business about absorbing colors at different depths in the silicon sounds like typical marketing oversimplification, but gives enough of a clue to be useful. I can see this being extended down into a multispectral (including near infared) sensor in a generation or two, which would be even cooler yet.

This thing could also make one heck of a nice nightvision system, if used properly... so we could all have nice color pictures at night, just like the military folks have had for a long time. (Green screens are just for the media to consume).

--Mike--

logs

[purduephotog]

There are a couple of problems with logarithmic sensitivities in electronics- the little potential wells fill up too quickly. Make them too deep and they lose the low level light, make them too shallow and the electrons spill out.

Conventional AgX can capture around 14 stops of light (thats 2^14) - conventional paper can handle 8 stops or so... a typical scene has 2^11, give or take. Depends on the scene and the subject- obviously a shot of a barn with the door open in broad daylight is going to have a bit more range than a shot inside in a white room with light bouncing everywhere.

So, what you really want, is to have the SOFTWARE be cognizant of higher bitdepth images. When you have 8 bits to capture a 10 bit scene, information is lost. So you throw some out... and you end up with muddled highlights and muddled shadows, and something in the middle that looks decent.

Believe it or not, but alot of companies have spent alot of money trying to figure out the correct 'mental' representation of a greyscale- not even including colour. I'm partial to Kodak (I work there, but these views are mine).

I've worked with extended bit depth images quite a bit and know that there is none (read, big fat ZERO) ms support for anything over 8 bits.... in fact, ImageViewer simply locks up and crashes. So any sort of solution that gives you extended tonal rendering are going to have to be custom solutions... and that probably won't sit well with the average person- "what do you mean i have to process my pictures before I can view them?!?!? I'll just go buy another camera" etc etc. Even if the benefits are enormous, there is the simplicity factor that drives it.

I personally am interested in this sensor, but there seems to be the wrong website linked... which worries me...

Also invented integrated circuit compiler

[peter303]

Back in the late 1970s Carver Mead of CalTech and Lynn Conway of Xerox PARC computerized the design of integrated circuit chips. Before them chips were designed by mechanical drawing and hand-taped photo-masks. This often resulted in spaghetti-looking chip circuits. Mead & Conway reduced chip design to a hierachical set of physics and geometry issues, and wrote a compiler to issue these from higher level descriptions. Chip design was then transformed more-or-less into a computer language. People then added optimization and simulation-testing tools to further automate the process. It got so simple that chip design labs were offered in engineering colleges with same-semester turn-around. Some guy in my class twenty years ago designed a "homogeneous coordinate multiplier" which become the geometry engine of a startup called Silicon Graphics.