Using The GIMP (or Photoshop) to Improve Photos?

Nom du Keyboard asks: "Is it possible to use The GIMP (or Photoshop) to improve my digital photos? I have a mid-range 7.1MP Olympus camera capable of shooting in Raw mode. When I inspected a section of clear blue sky on a bright, sunny day (which I've long believed to be relatively good reference of uniform color and brightness) I was surprised (disappointed, since I expect digital perfection) at the variance in adjacent pixels. It's also a quick way to identify any bad pixels. Surprisingly, actual photos from this camera look pretty good despite this variance so far. Moving on from that point it led me to wonder that, if you shot a uniform white surface, perhaps blurred as much as possible to avoid any imperfections in the surface itself, could a correction (adjustment) layer be created in GIMP or Photoshop exactly tuned to your camera that fixed the variations in your CCD sensor and improved the image quality in the process. Any thoughts?"

Comments

It's out there.

[Anonymous Coward]

My digital rebel xti has software that lets you do what you describe. It's used to correct blemishes caused by dust particles on the sensor. In a regular imaging tool you could probably work out a similar fix by creating a mask that does some enhancements and whatnot where there are darker pixels. Just a guess, I've never done it.

Learn about photography

[linuxbert]

What you describe is normal, and your question exhibits a lack of understanding about white ballence.
essentially, if your white is right, then all the other colors will be as well. your camera has several settings to compinsate for various light types (Tungsten, Flourescent, Daylight)Yours is probably set to AWB (Auto) which is easy - as the camera will figure it out pretty well and a Custom - which you can configure based on the lighting by shooting a grey card - which is a card that is 15% grey (Or there abouts) that the camera can then use to figure out what true white is.

The variation in pixels can also be the result of the ISO setting you are using. 100 has the least noise, but also requires longer exposures. higher settings react faster, but have more noise (400,800,1600) This is a tradeoff between desigered exposure and ambiant light.

I would suggest reading Strobist for more on lighting. There are also several other sites dedicated to post processing images, that you may find helpfull. it also might be worth looking at the various pool discucssion groups on Fliker.

-Peter

Re:Learn about photography

[Frobnicator]

You are right that light balance and natural noise are both very important.

Take for example the camera-assistant production slates (those little boards you see movie makers use with the clapper on top). They do a lot more than just showing the script location and film location, but they also have little black and white (and gray) lines on the clapper. Those are amazing tools that are deceptively simple. The clapper makes a sharp noise that lets you sync and balance the audio, digital boards will record the sync for individual film frames, and the lines provide for image calibration.

The black, gray, and white boards allow you to balance the brightness in post production exactly the way the original post was looking for.

Most boards also have calibrated colors to help balance those, as well.

Shooting slate is a very important step in good photography, both for stills and motion pictures.

And to the posters suggesting trying to eliminate all natural noise in photos, you don't really understand what you are talking about. Your eye expects noise in the real world.

Photos need natural noise, they look unnatural or cartoonish without it. Traditional photographs are full of noise because the silver halide gelatin and other chemicals are not perfectly uniform. The chemicals naturally clump up and form noise. (This property makes it easy to identify tampered photos since the natural noise is different between two areas.) Even digital photos get noise when you print them or display them on your screen. If your camera automatically smoothed out all the noise, the image would look like a cartoon or a naively ray-traced image.

As far as using image editing apps such as the GIMP or Photoshop, yes they are able to do a great job with digital images but they are limited by the knowledge and skill of the human using them.

Noise ninja

[Illusion]

See Noise Ninja for well-known commercial software that does this. There's apparently now also a Linux version.

While playing with it a while ago, I found that JPGs compress something like 25-33% better after you remove the CCD noise. Improving the image quality while making the images take less space seems like a nice combination. :)

This seems like it would be great to get in the hands of more people as a free software app or plugin, but I'm not aware of any.

-- Aaron

Good idea, but use black instead of white.

[NereusRen]

Using the sky or a white piece of paper may be interesting, but it probably won't give you anything you can use to calibrate the rest of your photos.

A better bet for isolating the noise your camera generates is to take completely black photos, using the lens cap and some extra covering (and a dark room) to make sure absolutely no light hits the sensors. This will let you make raw images of the "dark noise" and "bias noise" that your camera generates, and subtract those images from your real photos before doing any other processing.

Details of this method can be found here: http://photo.net/learn/dark_noise/.

Re:Good idea, but use black instead of white.

[zippthorne]

Better cameras already do this, by taking a picture with the shutter closed. You can sometimes select between taking a "dark frame" for every picture and taking a single "dark frame" to apply to all subsequent frames. Best to read the manual for your particular model to see if you have these features.

Yes Exactly! Only Backwards....

[DonnarsHmr]

You're almost right. The method you're using is called Dark Frame Subtraction. The idea is that you photography the non-random noise inherent in the sensor and then take that out of the captured images. To do this, you make an image that is completely black (i.e. body cap on the front of the camera and viewfinder cover on the back) at the same temperature conditions and for the same length of shutter speed as the image you are trying to fix. Then you add that as a layer in photoshop, subtract it from the real image, and the non-random noise disappears.

However, it is MUCH more likely that the noise you are complaining about is random thermal noise, which is not treatable via Dark Frame Subtraction. Because it's, well, random noise, it'll be different in every shot. There are several photoshop plugins that can address this issue. In my opinion, the most effective and easiest to use of them is Noise Ninja.

Sure Can.

[philibob]

You already can. Some cameras let you shoot against a blank white area to compensate for dust particles on the CCD. It's called "Dust Reference" in Nikon Capture, which works with most of their DSLRs.

Something like this?

[Quixotic137]

An article about correcting CCD variation

Re:Something like this?

[fossa]

Another: Noise, Noise, Noise by M. Colleen Gino.

yes..

[slashkitty]

I don't know about your particular problem, but other camera flaws have been fixed with processing. For example, if your camera adds a vignette, you shoot a piece of white paper, then remove that shading from all the photos. This is gives you an automatic, scriptable way to do that with ImageMagick:

Vignettation Removal

Re:yes..

[YttriumOxide]

Note to anyone that plans on doing this - good digital SLRs have this kind of function built in and you should only consider this if your camera doesn't. The quality of the adjustment will actually be significantly worse unless you ensure:

1. The light is hitting the white paper evenly
2. The white paper is a nice bright and clean white (don't even THINK of using standard office copy paper)
3. The paper is of a very short grain
4. There is no curving or folding in the paper

It might be better to consider the alternative black reference - while a good system would use both, a system which has neither is easier to do with a black reference than a white one.

(disclaimer: I don't know much about photography, but I know a LOT about paper, colour theory and image editing)

Not quite...

[Joe Decker]

Discounting truly bad pixels, variations in the sensor readings on an even sky have two sources--pure sampling noise from the fact that the sensor is only reading a finite number of pixels, and a more constant, but still varying per-pixel offset. It's likely with a daylight shot that you're primarily seeing the former, the latter effects tend to be more significant during long exposures doing astrophotography. Check out the "Digital Rebel" astrophotography page here, it outlines a procedure for measuring and subtracting off this varying per-pixel offset, but notices you need to essentially compute the "dark frame" (or offset) for a particular set of conditions (temperature, ISO, exposure time). That subtraction could be done in PS, but again, you really need a new "dark frame" for each shot.

It is possible to smooth rough skies and such in Photoshop, I can't speak from personal experience with the GIMP but I'd expect something similar would work. I'd take the image, duplicate a regular (non-adjustment) layer on top of the main image, call that second one "smoothed"), blur it (Gaussian blur, fiddle with the radius to keep the effect gentle), add a layer mask to "smoothed" and paint it so that it only targets the sky in a shot. You may end up finding that you want to leave a little noise in the resulting image to avoid posterization, if your results are too smooth you can always adjust the opacity of the smoothed layer downward.

Yes

[YGingras]

Scale down the picture, choose cubic interpolation and you're done. You can't fix the original, the information is scrambled already, but you can use the information of the larger image to average the pixels of the small image to get something clean. When you read X mega-pixels, you should know that this is a scam. There are no camera out there that will give you an image usable at X resolution but you can still have pretty pictures at X/2 (which is roughly 3/4 of the side on the original).

Something similar

[ceoyoyo]

Something similar is done in astrophotography. There are two kinds of fields you can remove from your images. A dark frame (taken with the lens cap on) is subtracted to remove things like pattern noise, hot pixels and amp glow that appears in images. A flat frame is then used to remove multiplicative effects, like vignetting and dust spots. Acquiring a flat frame can be tricky. One of the best ways is to use a translucent lens cap and a fairly bright light that provides a fairly uniform illumination.

However, the effects (unless there's something seriously wrong with your camera) are really only noticeable for long exposures.

Yes

[Ankh]

First, the biggest improvement you are likely to see in the Gimp is if you go to Colour->Layers (in older versions of the Gimp it's Layers->Colours->Levels) and click Auto. For pictures that should contain some black and some white this will usually make a noticeable improvement.

Second, yes, Canon (for example) includes (Windows only, proprietary, secret, closed-source) software to compensate: you shoot a 25% grey surface. You can also use this inside the camera itself: there it will use the data for white balance correction.

In practice, though, it's fairly hard to do this yourself. One difficulty is that the amount and position of colour aberrationswill probably vary depending on the lens you use, or, with a fixed lens, the amount of zoom and the aperture size. I know I found that when my Casio developed some dark spots.

There are some programs that are used with hugin, the panorama stitching UI, that help with some lens corrections; it might be you could ask those people. However, a lot of the variation you are seeing is likely to be digital noise. Try taking 3 shots usinga tripod and timer or remote, and comparing them.

Uniform pixel sensitivity

[Technician]

Pixels are not identical in their dark current and light sensitivity.

For information on correcting these issues which compound in long exposures, find a good astronomy photographers forum. They discuss taking long exposures of various times with the camera capped to identify bright (high dark current) pixels. They use these corrections in their star shots of the same exposure time to subtract out the brightness caused by high dark current pixels. In bright scenes the same thing can be done to correct for low sensitivity (low bright current) pixels. A way out of focus shot of a white screen with primary color filters or lighthing should be able to give you some good sensor correction factor data. Remember that the errors are temprature sensitive so a full correction may be hard to get.

Re:try it

[smallfries]

I'd try it twice, with two different but supposedly uniform surfaces. I'll bet that the fluctuations in pixel intensities aren't uniform across both surfaces as they're not caused by a systematic bias in the CCD. Rather they are caused by random noise in the circuit.

If it turns out that there is a systematic bias (ie one that you can correct in the GIMP with a static image) then you would be best off taking a picture of something as black as you can make it. The inside of a bad should do. And then as light as you can make it (not really sure about this one - lighbulbs maybe). From the two masks you can make the image that you want without needs to make a perfect surface first.

Re:Interesting idea

[Goeland86]

that's not too hard. Get a projection screen with a spotlight aimed at it. There's your screen. Then you can create your layer fairly easily, no?
Any meeting room, or multimedia classroom will have one of those, anywhere with a projector will work. You just need a pair of tripods, your camera and a fairly powerful wide range spotlight. Done!

Re:Not only those

[YttriumOxide]

Paint.NET is really great for those who need a quick and dirty image editor with a lot more power than MSPaint. However be careful - on most systems, it's a SERIOUS resource hog when dealing with large images (such as the 8 megapixel images from my camera). I find Paint.NET is great for anything that fits on my screen without scaling to less than about 50%, but go above that and my poor little work laptop (Dell Latitude D510 - 512MB RAM, 1.73GHz Pentium-M) will choke and die with lots of swap file use. Photoshop (CS) and the GIMP on the other hand hardly run like a dream, but they both deal with large images in a much nicer way.

I assume the problem is pretty much entirely RAM related and if you throw a decent amount of RAM at it, you'll be able to work with much larger images, but you'll quickly find you do have a very definite threshold and it'd be wise to avoid going above that.

Re:Good thing you don't shoot with film

[Andy Dodd]

"If you are, then you probably ought to have more pixels (i.e., a better camera). I'm okay with digital pictures down to about 150dpi, others swear that you need 300+. Then again, there are people who swear that $3000 unobtainium coated silver strands wrapped in virgin PTFE and assembled when the planets are in alignement make their music sound better."

More pixels is not necessarily better. More sensor area is usually more important.

This is why high-end DSLRs with only 4-5 megapixel resolution deliver better images than 7-8 megapixel consumer cameras - larger sensor elements result in higher signal to noise ratio at the sensor, which means less image noise. Considering the submitter's problem is image noise, a higher resolution camera with the same sensor size is NOT going to help them.