Slashdot Mirror
Scaling Algorithm Bug In Gimp, Photoshop, Others
Wescotte writes "There is an important error in most photography scaling algorithms. All software tested has the problem: The Gimp, Adobe Photoshop, CinePaint, Nip2, ImageMagick, GQview, Eye of Gnome, Paint, and Krita. The problem exists across three different operating systems: Linux, Mac OS X, and Windows. (These exceptions have subsequently been reported — this software does not suffer from the problem: the Netpbm toolkit for graphic manipulations, the developing GEGL toolkit, 32-bit encoded images in Photoshop CS3, the latest version of Image Analyzer, the image exporters in Aperture 1.5.6, the latest version of Rendera, Adobe Lightroom 1.4.1, Pixelmator for Mac OS X, Paint Shop Pro X2, and the Preview app in Mac OS X starting from version 10.6.) Photographs scaled with the affected software are degraded, because of incorrect algorithmic accounting for monitor gamma. The degradation is often faint, but probably most pictures contain at least an array where the degradation is clearly visible. I believe this has happened since the first versions of these programs, maybe 20 years ago."
Photographs scaled with the affected software are degraded, because of incorrect algorithmic accounting for monitor gamma.
Seriously!
I have a theory on why this has gone unnoticed for so long, but I'll keep it to myself...
Most scaling algorithms treat brightness as a linear space, so e.g. if you're doing downscaling to 1/2 the size in each dimension, collapse 4 pixels into 1 by setting the 1 pixel to the numerical average of the original 4 pixels. But, most images are displayed with an assumption that brightness is a nonlinear space, i.e. gamma > 1. Therefore, scaling changes the perceived brightness, an unexpected result.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
Excellent point. Just to be safe though, I'm going to take another look through my porn crypt to see if that's true.
BRB.
Well, I am SURE glad I'm using Linux^H^H^H^H^HWindows^H^H^H^H^H^H^HMac^H^H^Hshit.
It is pitch black. You are likely to be eaten by a grue.
I've been telling people for years that I look better in person.
I told them that there's something wrong with pictures of me.
HA!
Now I know.
It's the Scaling Algorithm BUG!
The data in the pictures is not linear data. It assumes that it will be displayed on a system that introduces a gamma of 2.2. (If your display system does not do that physically, it should correct for this.) That is, a gray 127 should not display as halfway between a white 255 and a black zero, in terms of light output. (It should *appear* halfway between them visually, because your eyes aren't linear — that's (part of) why gamma is in use in the first place.) So, a checkerboard pattern of white / black squares will have half the luminosity of the white squares. When scaling down, software will turn it into a bunch of gray pixels. But they should be gray pixels of value 186, not 127.
The page is not well written, but his example images make the issue very clear. It's not about your monitor gamma; it's about the "standard gamma" that all image files assume your monitor has.
But most people who use images expect to look at them eventually. And most image files are meant to be viewed at gamma 2.2. (Printer drivers will at least approximately emulate a gamma of 2.2, and LCDs emulate it intentionally.) If you view the image at some other gamma, you don't see quite what was intended.
Another way of looking at it is that most standard image formats are stored with a nonlinear representation, and people who do math should realize that. For an untagged image, gamma=2.2 is a good bet. gamma=1.0 is a terrible bet.
Of course, if we really want our software to do a good job, then that software should be aware that specifying colors like #FF0000 isn't a good idea -- they look very different on different screens. What the user probably meant to do was specify a particular color, which means that the numbers need to be marked with a color space. (For a great demo, get an HP LP2475w or some other good wide-gamut display, don't install a profile, and look at anyone's photo album. Everyone looks freakishly red-faced.)
Come on, this isn't news...
Helmut Dersch (of Panorama Tools fame) certainly posted about this before;
http://www.all-in-one.ee/~dersch/gamma/gamma.html - Interpolation and Gamma Correction
There's no factual error in the scaling algorithm, as the /. headline would like you to believe - it's a color space (linearity) issue; you have to do your calculations in linear space which means a typical photo off of a camera/scanner gets the inverse of an sRGB curve applied (a gamma of 0.454545 is 'close enough' if you can't do the proper color bits). Then scale. Then re-apply the curve.
And no - for real life imagery, nobody really cares - the JPEGs out of the cameras and subsequent re-compression to JPEG after scaling will have 'destroyed' far more data than the linearity issue.
They're nice example images in the story, but they should be called 'academic'.
I am sure the Chinese government prefers the current implementation.
Actually, any well-specified file format will specify the gamma. Not all allow you to set it per-file, but they do specify it. Normally this is a line in the spec that reads something like "color values use the sRGB color space" or similar — which specifies a gamma of 2.2 (roughly). And sRGB with it's nearly 2.2 gamma has become so standard that assuming anything else (in the absence of a clear spec) would be idiotic.
The basic issue here has to do with gamma curves and the way they're being handled (they're not).
Most image files on your computer (BMP, JPG, PNG, etc.) are stored in the sRGB color space. sRGB defines the use of a gamma curve, which is a nonlinear transformation applied to each of the components (R, G, and B). The issue here is that most scalers make the assumption that the components are linear, rather than try to process the gamma curve. While this does save processing time (undoing the gamma curve then redoing it), it does add some error, especially when the values being scaled are not near each other.
So does this matter? Well, in some pathological cases where there are repeated sharp boundaries (such as alternating black-white lines or fine checkerboard patterns), this would make a difference. This is because the linear average of the pixels (what most image scalers use) yields a different result than if the gamma value was taken into account. For most images (both photographic and computer generated), this shouldn't be a big problem. Most samples are close in value to other nearby samples, so the error resulting from the gamma curve is very small. Sparse light-dark transitions also wouldn't be noticeable as there would only be an error right on the boundary. Only when you exercise this case over a large area does it become obvious.
One final point: this gamma scaling effect would occur regardless of the actual scaling algorithm. Bilinear, bicubic, and sinc would all have the same issue. Nearest neighbor interpolation would be unaffected, but in these cases, the output would look far worse.
Ok, so he made a very informative page about it, but this is still a well known effect. It affects practically everything you can do in image editing. Blurs, etc. Most people neither notice nor care. It's rooted in the fact that most images come with undefined black and white points and a gamma chosen for artistic effect rather than physical accuracy. Thus correctly converting to linear gamma is hardly ever possible. You can still correct for monitor gamma to avoid some rarely seen inconsistencies and artifacts, but most people don't even notice, so why bother? However, Photoshop does have everything you need to avoid the effect completely, even in the ancient Photoshop 6.0.
Here's how to properly resize in Photoshop:
1. Convert mode to 16 bit (to avoid tone aliasing in the next step, no other influence on the calculations)
2. Convert to profile, select "Custom RGB", set Gamma to 1.0 (this converts the internal image data to linear gamma, no visible change because the image is color managed and corrected back to monitor gamma on the fly)
3. Image Size
4. Convert to profile, select "Custom RGB", set Gamma to 2.2 (default)
5. Convert mode to 8 bit
Done. You can substitute your favorite image filter for the image resize. Unsharp mask works much better at gamma 1.0, for example. Of course you can use several filters before converting back to monitor gamma and 8 bit.
If you're looking for lcd test images, http://www.lagom.nl/lcd-test/ is probably better. It's got a whole bunch of images dedicated to various monitor problems, along with explanations.
My software has been calculating in linear space for over a decade now (this is the Nuke Compositor currenlty produced by The Foundry but at the time it was used by Digital Domain for Titanic). You can see some pages I wrote on the effect here: http://mysite.verizon.net/~spitzak/conversion/composite.html. See here for the overall paper: http://mysite.verizon.net/~spitzak/conversion/index.html and a Siggraph paper on the conversion of such images here: http://mysite.verizon.net/~spitzak/conversion/sketches_0265.pdf, in fact a lot more work went into figuring out how to get such linear images to show on the screen on hardware of that era than on the obvious need to do the math in linear. Initial work on this was done for Apollo 13 as the problems with gamma were quite obvious when scaling images of small bright objects against the black of space.
For typical photographs the effect is not very visible in scaling, as the gamma curve is very close to a straight line for two close points and thus the result is not very much different. Only widely separated points (ie very high contrast images with sharp edges) will show a visible difference. This probably means you are trying to scale line art, there are screenshots in the html pages showing the results of this. Far worse errors can be found in lighting calculations and in filtering operations such as blur. At the time even the most expensive professional 3D renderers were doing lighting completely wrong, but things have gotten better now that they can use floating point intermediate images.
One big annoyance is that you better do the math in floating point. Even 16 bits is insufficient for linear light levels as the black points will be too far apart and visible (the space is wasted on many many more white levels than you ever would need). A logarithmic system is needed, and on modern hardware you might as well use IEEE floating point, or the ILM "half" standard for 16-bit floating point.
Well?
It seems crazy to me to embed a particular Gamma value into an image. ...In fact it seems so crazy I must be missing something. Am I?
The article actually touches on this point. The sensitivity of the human eye isn't linear. If you use a linear scale to store luminosity information for an image, you waste a lot of bit depth at high luminosities - the eye has difficulty distinguishing between very bright and very bright plus a little tiny bit. On the other hand, the eye is very good at telling the difference between very dark and black. You need a lot of finely-graduated steps at low luminosity or else your shadows get jaggy.
If you uniformly (linearly) space out luminosities on an 8-bit (256-shade) scale, you store a lot of uninteresting information at the high end, and lose out on visible detail at the low end. A scale with gamma of 2.2 (typical these days) fits a full twenty-eight grey values between 0 and 1 on our hypothetical linear scale. To maintain that kind of luminosity resolution (down where it matters), you'd have to store an extra five bits on your linear scale. An extra sixty percent costs.
~Idarubicin
Well?
Somehow I don't think you've given him enough time.
Several people have spoken about "linear" RGB. That's nice and gets rid of some small level of distortion introduced by the non-linearity. However, it only starts there. For example, the eye sees R, G, and B differently. It is more sensitive to green than red, and to red more than blue, but it's not even that simple as the equations in your eye's processor are much more complicated. Many algorithms that treat the three "equally" are going to change the perceptual mixture. One can use other color spaces, such as HSV, Yuv, xyY, etc. with different advantages and disadvantages
Sound makes a good analogy. When you play music through any given combination of source, amp and speakers, it sounds different. Sometimes we actually like a particular type of sonic "distortion". It's never exactly like the "original" live music, though.
Likewise, any graphics manipulation is "distorting" the original. In fact, when I take a digital image and run it through Lightroom, do a range expansion/equalization, and do a bunch of tweaks to make the image look good, I'm making much larger changes than those little scaling problems listed in the article. The point is, do you think the result looks good?
There's other important variables, such as what colors are next to other colors in the image, how long you look at the image, what else is around you, how tired you are, etc. There's no such thing as color fidelity, there's only approximations to it. Color is hard, and I mean, really hard. See Hunt, "The Reproduction of Colour", or any number of other fine texts to learn more.
Join the window installer's union, where prosperity is a brick throw away!
meanwhile, I see a grey rectangle in firefox, and I still don't get what that signifies.
Right-click the image, then click view image. You'll see the image full-scale, like the first image. Scaling it down 50% shouldn't make it gray.
1. Spend a lot of time calibrating your camera yourself. This is only cheap if your time is worthless
Typically this should be done during the initial set-up of any new camera that a photographer purchases. A Gretag Macbeth colour checker is cheap, the required shot to evaluate the performance of the sensor is quick and easy to set-up and the processing of this test image is fast with the right tools (like this script for photoshop). It should take under an hour to do it right and get it as part of the automatic stage of processing your RAW files (basically setting ACR/Lightroom's demosaicing stage), but the benefit is that every picture taken from then onwards does not need extra calibration. Thus your prints look like your shots, assuming the rest of your workflow is equally as calibrated.
While your time is valuable, if you do not calibrate like this, you're wasting time further down the line for EACH image, and thus it's more expensive to not do it...