Using The GIMP (or Photoshop) to Improve Photos?

Nom du Keyboard asks: "Is it possible to use The GIMP (or Photoshop) to improve my digital photos? I have a mid-range 7.1MP Olympus camera capable of shooting in Raw mode. When I inspected a section of clear blue sky on a bright, sunny day (which I've long believed to be relatively good reference of uniform color and brightness) I was surprised (disappointed, since I expect digital perfection) at the variance in adjacent pixels. It's also a quick way to identify any bad pixels. Surprisingly, actual photos from this camera look pretty good despite this variance so far. Moving on from that point it led me to wonder that, if you shot a uniform white surface, perhaps blurred as much as possible to avoid any imperfections in the surface itself, could a correction (adjustment) layer be created in GIMP or Photoshop exactly tuned to your camera that fixed the variations in your CCD sensor and improved the image quality in the process. Any thoughts?"

Comments

Interesting idea

[Aladrin]

That's an interesting idea, but it assumes some pretty clean conditions. The light has to be absolutely the same over the entire surface, it would probably need to be blurred as you said, the surface would have to be absolutely the same color everywhere (no dust, no marks), the surface would to be completely non-reflective, and probably some other things that I haven't thought of. It would be extremely hard.

It also assumes that the variations are always the same, and that the variations in your photos are from defects and not from the natural color differences in the real world and the digital camera's attempt to map them to a very restricted color palette.

Learn about photography

[linuxbert]

What you describe is normal, and your question exhibits a lack of understanding about white ballence.
essentially, if your white is right, then all the other colors will be as well. your camera has several settings to compinsate for various light types (Tungsten, Flourescent, Daylight)Yours is probably set to AWB (Auto) which is easy - as the camera will figure it out pretty well and a Custom - which you can configure based on the lighting by shooting a grey card - which is a card that is 15% grey (Or there abouts) that the camera can then use to figure out what true white is.

The variation in pixels can also be the result of the ISO setting you are using. 100 has the least noise, but also requires longer exposures. higher settings react faster, but have more noise (400,800,1600) This is a tradeoff between desigered exposure and ambiant light.

I would suggest reading Strobist for more on lighting. There are also several other sites dedicated to post processing images, that you may find helpfull. it also might be worth looking at the various pool discucssion groups on Fliker.

-Peter

Re:Learn about photography

[Frobnicator]

You are right that light balance and natural noise are both very important.

Take for example the camera-assistant production slates (those little boards you see movie makers use with the clapper on top). They do a lot more than just showing the script location and film location, but they also have little black and white (and gray) lines on the clapper. Those are amazing tools that are deceptively simple. The clapper makes a sharp noise that lets you sync and balance the audio, digital boards will record the sync for individual film frames, and the lines provide for image calibration.

The black, gray, and white boards allow you to balance the brightness in post production exactly the way the original post was looking for.

Most boards also have calibrated colors to help balance those, as well.

Shooting slate is a very important step in good photography, both for stills and motion pictures.

And to the posters suggesting trying to eliminate all natural noise in photos, you don't really understand what you are talking about. Your eye expects noise in the real world.

Photos need natural noise, they look unnatural or cartoonish without it. Traditional photographs are full of noise because the silver halide gelatin and other chemicals are not perfectly uniform. The chemicals naturally clump up and form noise. (This property makes it easy to identify tampered photos since the natural noise is different between two areas.) Even digital photos get noise when you print them or display them on your screen. If your camera automatically smoothed out all the noise, the image would look like a cartoon or a naively ray-traced image.

As far as using image editing apps such as the GIMP or Photoshop, yes they are able to do a great job with digital images but they are limited by the knowledge and skill of the human using them.

Noise ninja

[Illusion]

See Noise Ninja for well-known commercial software that does this. There's apparently now also a Linux version.

While playing with it a while ago, I found that JPGs compress something like 25-33% better after you remove the CCD noise. Improving the image quality while making the images take less space seems like a nice combination. :)

This seems like it would be great to get in the hands of more people as a free software app or plugin, but I'm not aware of any.

-- Aaron

Good idea, but use black instead of white.

[NereusRen]

Using the sky or a white piece of paper may be interesting, but it probably won't give you anything you can use to calibrate the rest of your photos.

A better bet for isolating the noise your camera generates is to take completely black photos, using the lens cap and some extra covering (and a dark room) to make sure absolutely no light hits the sensors. This will let you make raw images of the "dark noise" and "bias noise" that your camera generates, and subtract those images from your real photos before doing any other processing.

Details of this method can be found here: http://photo.net/learn/dark_noise/.

Yes.

[oskard]

If we saw a sample of the photos, it would be easy to determine if they could be fixed. Its hard to understand what the exact problem is from a text description, but the general answer is: Yes, anything can be done with The GIMP / PS.

Yes Exactly! Only Backwards....

[DonnarsHmr]

You're almost right. The method you're using is called Dark Frame Subtraction. The idea is that you photography the non-random noise inherent in the sensor and then take that out of the captured images. To do this, you make an image that is completely black (i.e. body cap on the front of the camera and viewfinder cover on the back) at the same temperature conditions and for the same length of shutter speed as the image you are trying to fix. Then you add that as a layer in photoshop, subtract it from the real image, and the non-random noise disappears.

However, it is MUCH more likely that the noise you are complaining about is random thermal noise, which is not treatable via Dark Frame Subtraction. Because it's, well, random noise, it'll be different in every shot. There are several photoshop plugins that can address this issue. In my opinion, the most effective and easiest to use of them is Noise Ninja.

Re:Yes Exactly! Only Backwards....

[hankwang]

Then you add that as a layer in photoshop, subtract it from the real image, and the non-random noise disappears.

I doubt that that will work. Once in the computer, the pixel values are not proportional to the absolute brightness, see gamma correction on wikipedia. You would need to do the substraction on linearly encoded data (12 or more bits rather than 8). Maybe photoshop can indeed do this, provided you find the right settings, but GIMP as far as I know doesn't.

Sure Can.

[philibob]

You already can. Some cameras let you shoot against a blank white area to compensate for dust particles on the CCD. It's called "Dust Reference" in Nikon Capture, which works with most of their DSLRs.

Something like this?

[Quixotic137]

An article about correcting CCD variation

Not quite...

[Joe Decker]

Discounting truly bad pixels, variations in the sensor readings on an even sky have two sources--pure sampling noise from the fact that the sensor is only reading a finite number of pixels, and a more constant, but still varying per-pixel offset. It's likely with a daylight shot that you're primarily seeing the former, the latter effects tend to be more significant during long exposures doing astrophotography. Check out the "Digital Rebel" astrophotography page here, it outlines a procedure for measuring and subtracting off this varying per-pixel offset, but notices you need to essentially compute the "dark frame" (or offset) for a particular set of conditions (temperature, ISO, exposure time). That subtraction could be done in PS, but again, you really need a new "dark frame" for each shot.

It is possible to smooth rough skies and such in Photoshop, I can't speak from personal experience with the GIMP but I'd expect something similar would work. I'd take the image, duplicate a regular (non-adjustment) layer on top of the main image, call that second one "smoothed"), blur it (Gaussian blur, fiddle with the radius to keep the effect gentle), add a layer mask to "smoothed" and paint it so that it only targets the sky in a shot. You may end up finding that you want to leave a little noise in the resulting image to avoid posterization, if your results are too smooth you can always adjust the opacity of the smoothed layer downward.

Yes

[YGingras]

Scale down the picture, choose cubic interpolation and you're done. You can't fix the original, the information is scrambled already, but you can use the information of the larger image to average the pixels of the small image to get something clean. When you read X mega-pixels, you should know that this is a scam. There are no camera out there that will give you an image usable at X resolution but you can still have pretty pictures at X/2 (which is roughly 3/4 of the side on the original).

Something similar

[ceoyoyo]

Something similar is done in astrophotography. There are two kinds of fields you can remove from your images. A dark frame (taken with the lens cap on) is subtracted to remove things like pattern noise, hot pixels and amp glow that appears in images. A flat frame is then used to remove multiplicative effects, like vignetting and dust spots. Acquiring a flat frame can be tricky. One of the best ways is to use a translucent lens cap and a fairly bright light that provides a fairly uniform illumination.

However, the effects (unless there's something seriously wrong with your camera) are really only noticeable for long exposures.

Yes

[Ankh]

First, the biggest improvement you are likely to see in the Gimp is if you go to Colour->Layers (in older versions of the Gimp it's Layers->Colours->Levels) and click Auto. For pictures that should contain some black and some white this will usually make a noticeable improvement.

Second, yes, Canon (for example) includes (Windows only, proprietary, secret, closed-source) software to compensate: you shoot a 25% grey surface. You can also use this inside the camera itself: there it will use the data for white balance correction.

In practice, though, it's fairly hard to do this yourself. One difficulty is that the amount and position of colour aberrationswill probably vary depending on the lens you use, or, with a fixed lens, the amount of zoom and the aperture size. I know I found that when my Casio developed some dark spots.

There are some programs that are used with hugin, the panorama stitching UI, that help with some lens corrections; it might be you could ask those people. However, a lot of the variation you are seeing is likely to be digital noise. Try taking 3 shots usinga tripod and timer or remote, and comparing them.

Or GREYCstoration

[dschl]

GREYCstoration. Ugly name, but does the same job, and is open source. Haven't tried it, but there appear to be several plugins for various open source digicam programs and image editors (bottom of their downloads page).

Good thing you don't shoot with film

[Overzeetop]

You'd be just devastated if you blew a film image up to the level where you could see the grain.

Here are two questions for you:

1) Do you find that you are printing your images at sizes larger than 12x18?

If you are, then you probably ought to have more pixels (i.e., a better camera). I'm okay with digital pictures down to about 150dpi, others swear that you need 300+. Then again, there are people who swear that $3000 unobtainium coated silver strands wrapped in virgin PTFE and assembled when the planets are in alignement make their music sound better.

2) Presuming you are actually printing at at least 200dpi, can you really see the difference without a loupe on the final prints? I'm not worried about your monitor, because I'm going to bet that if you have a consumer-level camera, you're not doing photoediting on a 7.1MP monitor.

You see, if you can't tell, don't worry about it. Let your geek side go and spend more time in the field and less time in the darkroom. Seriously - unless you have significant image problems you can see in your final output, the camera and imaging is good enough. Go take some great pictures, and worry a bit less about having digitally perfect pixels.

Why do you care for photographs?

[dlevitan]

Yes, your $1000 digital camera is not going to have a perfect CCD. There is no such thing as a perfect CCD. And I don't understand why you care unless you're trying to do science work with it. Look at it this way, no one is ever going to look at your picture and say its horrible because one pixel is slightly different than the one next to it. You look at the content of the whole photograph, not three pixels.

If you are trying to do science, then a DSLR is not what you need. DSLRs use Bayer interpolation to create a color image. This inherently kills your accuracy since not every pixel in the image is actually a pixel on the camera. CCDs used for astronomy (which cost more than your whole camera) do not do this and they still suffer from the effects you mentioned. Every exposure used for scientific work goes through a whole data reduction process that tries to remove as much noise as possible. Others have mentioned most of the process (bias frames, dark frames, and flat fields), but most astronomical CCDs also have an overscan region which is part of the CCD that is not exposed to light and is used to record the thermal noise on the CCD. This changes from exposure to exposure and from temperature to temperature (and yes I am a researcher in astronomy).

In short, there's no reason for you to care about this, and there's no chance of fixing this completely (CCDs are not digital - they're analog). There's also no way of applying the same solution to every photograph (and CCDs can change over time). Don't worry about pixel-to-pixel variations and just take photographs for their content. If you're really interested in how CCDs work, read the Handbook of CCD Astronomy by Steve Howell. Its a great introduction to CCDs and how to use them for astronomy.