Slashdot Mirror
Scaling Algorithm Bug In Gimp, Photoshop, Others
Wescotte writes "There is an important error in most photography scaling algorithms. All software tested has the problem: The Gimp, Adobe Photoshop, CinePaint, Nip2, ImageMagick, GQview, Eye of Gnome, Paint, and Krita. The problem exists across three different operating systems: Linux, Mac OS X, and Windows. (These exceptions have subsequently been reported — this software does not suffer from the problem: the Netpbm toolkit for graphic manipulations, the developing GEGL toolkit, 32-bit encoded images in Photoshop CS3, the latest version of Image Analyzer, the image exporters in Aperture 1.5.6, the latest version of Rendera, Adobe Lightroom 1.4.1, Pixelmator for Mac OS X, Paint Shop Pro X2, and the Preview app in Mac OS X starting from version 10.6.) Photographs scaled with the affected software are degraded, because of incorrect algorithmic accounting for monitor gamma. The degradation is often faint, but probably most pictures contain at least an array where the degradation is clearly visible. I believe this has happened since the first versions of these programs, maybe 20 years ago."
Most scaling algorithms treat brightness as a linear space, so e.g. if you're doing downscaling to 1/2 the size in each dimension, collapse 4 pixels into 1 by setting the 1 pixel to the numerical average of the original 4 pixels. But, most images are displayed with an assumption that brightness is a nonlinear space, i.e. gamma > 1. Therefore, scaling changes the perceived brightness, an unexpected result.
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
The data in the pictures is not linear data. It assumes that it will be displayed on a system that introduces a gamma of 2.2. (If your display system does not do that physically, it should correct for this.) That is, a gray 127 should not display as halfway between a white 255 and a black zero, in terms of light output. (It should *appear* halfway between them visually, because your eyes aren't linear — that's (part of) why gamma is in use in the first place.) So, a checkerboard pattern of white / black squares will have half the luminosity of the white squares. When scaling down, software will turn it into a bunch of gray pixels. But they should be gray pixels of value 186, not 127.
The page is not well written, but his example images make the issue very clear. It's not about your monitor gamma; it's about the "standard gamma" that all image files assume your monitor has.
But most people who use images expect to look at them eventually. And most image files are meant to be viewed at gamma 2.2. (Printer drivers will at least approximately emulate a gamma of 2.2, and LCDs emulate it intentionally.) If you view the image at some other gamma, you don't see quite what was intended.
Another way of looking at it is that most standard image formats are stored with a nonlinear representation, and people who do math should realize that. For an untagged image, gamma=2.2 is a good bet. gamma=1.0 is a terrible bet.
Of course, if we really want our software to do a good job, then that software should be aware that specifying colors like #FF0000 isn't a good idea -- they look very different on different screens. What the user probably meant to do was specify a particular color, which means that the numbers need to be marked with a color space. (For a great demo, get an HP LP2475w or some other good wide-gamut display, don't install a profile, and look at anyone's photo album. Everyone looks freakishly red-faced.)
Gray rectangles in Picasa 3 and in Irfanview 4.25 :-(
Actually, any well-specified file format will specify the gamma. Not all allow you to set it per-file, but they do specify it. Normally this is a line in the spec that reads something like "color values use the sRGB color space" or similar — which specifies a gamma of 2.2 (roughly). And sRGB with it's nearly 2.2 gamma has become so standard that assuming anything else (in the absence of a clear spec) would be idiotic.
The example images that make it really clear are academic examples. But the scaled photos are all enough of a change to be worth noticing and caring about if you're a serious amateur photographer (never mind professional). And they don't look particularly unusual to me (I haven't looked for odd trickery, but I assume he's being honest here).
Ok, so he made a very informative page about it, but this is still a well known effect. It affects practically everything you can do in image editing. Blurs, etc. Most people neither notice nor care. It's rooted in the fact that most images come with undefined black and white points and a gamma chosen for artistic effect rather than physical accuracy. Thus correctly converting to linear gamma is hardly ever possible. You can still correct for monitor gamma to avoid some rarely seen inconsistencies and artifacts, but most people don't even notice, so why bother? However, Photoshop does have everything you need to avoid the effect completely, even in the ancient Photoshop 6.0.
Here's how to properly resize in Photoshop:
1. Convert mode to 16 bit (to avoid tone aliasing in the next step, no other influence on the calculations)
2. Convert to profile, select "Custom RGB", set Gamma to 1.0 (this converts the internal image data to linear gamma, no visible change because the image is color managed and corrected back to monitor gamma on the fly)
3. Image Size
4. Convert to profile, select "Custom RGB", set Gamma to 2.2 (default)
5. Convert mode to 8 bit
Done. You can substitute your favorite image filter for the image resize. Unsharp mask works much better at gamma 1.0, for example. Of course you can use several filters before converting back to monitor gamma and 8 bit.
If you're looking for lcd test images, http://www.lagom.nl/lcd-test/ is probably better. It's got a whole bunch of images dedicated to various monitor problems, along with explanations.
It's basically an implementation issue. The algorithms may be fine as intended ... in linear space. The programmers that implemented them didn't understand linear vs. gamma, or didn't care, or had a fire breathing PHB on their back. Hence we get junk software.
At least all MY image processing code always works in linear space. Bu merely converting 8-bit gamma to 8-bit linear is no good because that now introduces some serious quantizing artifacts (major banding effects happen). So I convert the 8-bit gammas to at least 30 or 31 bit integer if I need processing speed, or all the way to double precision floating point if I need as close to correct as possible. After processing, then I convert back to 8-bit gammas. Even then, you can't totally eliminate some banding effects that result from being in 8-bit. If you can get more bits from the raw images from your camera, that's the best to use. Apparently many JPEG compressors are also doing their DCT calculations in the non-unit gamma space instead of the linear space, too (which reduces the effectiveness of the compression somewhat, and may add more compression artifacts).
now we need to go OSS in diesel cars
I think the author specifically isn't stating whether the scaling is correct or not - it is; the whole story doesn't relate to scaling at all, but rather color space and how -it- affects, among other, scaling. Yes, with filtering - scaling without filtering can hardly be called scaling at all as you're just discarding data - and for anything but multiples of 2 (4x, 2x, 0.5x, 0.25x, etc.) that'd have a whole 'nother set of problems.
The author, I think, is suggesting, quite rightly so, that while...
The desired result for scaling down likely being that of the same visual image as when you simply stand further back.
( although at some point the resolution limit of a display and the image itself being presented on that display prevents that concept from being applied to "moving your eyeballs closer to the screen" for scaling up. )
It seems crazy to me to embed a particular Gamma value into an image. ...In fact it seems so crazy I must be missing something. Am I?
The article actually touches on this point. The sensitivity of the human eye isn't linear. If you use a linear scale to store luminosity information for an image, you waste a lot of bit depth at high luminosities - the eye has difficulty distinguishing between very bright and very bright plus a little tiny bit. On the other hand, the eye is very good at telling the difference between very dark and black. You need a lot of finely-graduated steps at low luminosity or else your shadows get jaggy.
If you uniformly (linearly) space out luminosities on an 8-bit (256-shade) scale, you store a lot of uninteresting information at the high end, and lose out on visible detail at the low end. A scale with gamma of 2.2 (typical these days) fits a full twenty-eight grey values between 0 and 1 on our hypothetical linear scale. To maintain that kind of luminosity resolution (down where it matters), you'd have to store an extra five bits on your linear scale. An extra sixty percent costs.
~Idarubicin
Several people have spoken about "linear" RGB. That's nice and gets rid of some small level of distortion introduced by the non-linearity. However, it only starts there. For example, the eye sees R, G, and B differently. It is more sensitive to green than red, and to red more than blue, but it's not even that simple as the equations in your eye's processor are much more complicated. Many algorithms that treat the three "equally" are going to change the perceptual mixture. One can use other color spaces, such as HSV, Yuv, xyY, etc. with different advantages and disadvantages
Sound makes a good analogy. When you play music through any given combination of source, amp and speakers, it sounds different. Sometimes we actually like a particular type of sonic "distortion". It's never exactly like the "original" live music, though.
Likewise, any graphics manipulation is "distorting" the original. In fact, when I take a digital image and run it through Lightroom, do a range expansion/equalization, and do a bunch of tweaks to make the image look good, I'm making much larger changes than those little scaling problems listed in the article. The point is, do you think the result looks good?
There's other important variables, such as what colors are next to other colors in the image, how long you look at the image, what else is around you, how tired you are, etc. There's no such thing as color fidelity, there's only approximations to it. Color is hard, and I mean, really hard. See Hunt, "The Reproduction of Colour", or any number of other fine texts to learn more.
Join the window installer's union, where prosperity is a brick throw away!
This is one of many reasons why creative professionals prefer macs over PCs --- and I'm not saying this as platform evangelism -- for one, you'd be hard pressed to disagree that Mac OS X's font-rendering, kerning, and anti-aliasing abilities are far superior to those provided by Windows when presented with side-by-side examples.
Mac OS X's font rendering is different , but calling it "far superior" is simply platform evangelism.
OS X renders text so that the on-screen representation looks more like the printed representation, which is good for tasks like designing print advertisements (where you want to approximate the finished product as closely as possible). Windows takes liberties with the shape and spacing of on-screen text in order to line it up with the pixel grid, which is good for tasks like word processing and programming (where legibility on screen is more important). When you're used to Windows, Mac text looks blurry; when you're used to the Mac, I imagine Windows text looks thin and lanky.
Visual IRC: Fast. Powerful. Free.
Actually a good scaling algorithm should perform a lowpass filter when downscaling. This is similar to downsampling of digital audio where you do need to filter out frequencies above half the sampling rate. Leafing these higher frequencies in would cause noise because they can not be faithfully represented in a lower resolution file.
meanwhile, I see a grey rectangle in firefox, and I still don't get what that signifies.
Right-click the image, then click view image. You'll see the image full-scale, like the first image. Scaling it down 50% shouldn't make it gray.
No, I'm pretty sure that font display is measured by fidelity to the creator's intention and design
And I'm pretty sure you're wrong. Sorry. You might choose to measure font rendering that way--fidelity at any cost, even if it means reducing text to illegible blobs--but I don't, and evidently Microsoft doesn't either. In fact, I'd wager that most people don't.
The technical ins and outs of photo editing and display are all about fidelity; why would that not be the case with typeface rendering and display?
Because in many (if not most) cases, the primary purpose of rendering text is to make that text readable to the person sitting in front of the screen, and the formatting of that text is a secondary concern. In such cases, making "click here" distinguishable from "dick here" is more important than preserving the font designer's artistic vision.
So the question is, if Microsoft had bothered to put some effort into proper rendering, would there be any meaningful loss of legibility?
Empirically, the answer is obviously "no". As even that article points out, the determining factor is familiarity.
Again, this is only true if you ignore what happens at small sizes.
Why the article pretends that Microsoft's decision was anything other than lack of interest in fine-tuning is rather curious. [...] Pixel grid rendering is simply easier to implement
Nope, you've got it backwards: Microsoft's approach involves more fine-tuning. Apple's approach is easier to implement: just scale the outline to the appropriate size and fill it in.
Again, font rendering is measured by fidelity to the creator's intent. If the typeface is illegible because of blurring, then it's a poor typeface.
Nonsense. A font that looks fine when rendered in a 15-pixel-high line (on a 300 DPI printer) may look illegible when rendered unchanged in a 5-pixel-high line (on screen). That doesn't mean the typeface is poor, it just means it's too intricate for such a low resolution.
But Microsoft's decision to abandon the design in favor of a simplistic snap-to-grid renderer does nothing to improve legibility.
You seem to be confused about how that renderer works. It's far from "simplistic snap-to-grid": simply snapping the font's vertices to a pixel grid would produce garbage. What it actually does is apply the TrueType hints found in the font file.
The OS X font renderer ignores nearly all the hints, which is why the outline it renders at small sizes looks the same as the one it renders at large sizes: the hints are placed there by the font creator to tell renderers how to distort the outline at small sizes.
(Indeed, Microsoft's ClearType renderer pays less attention to the hints than their old one, because the greater resolution of subpixel rendering makes horizontal pixel-fitting less important.)
I happen to be of the print publication era, and I cannot stand the state of Microsoft and Linux font rendering. I'm happy to concede those with a personal and subjective preference for that system, but the reality is that it is objectively inferior in every way.
No, not in every way; only in the particular way you've chosen to focus on. Your personal, subjective preference is showing.
Visual IRC: Fast. Powerful. Free.
IMO, what it actually means is that the so-called image is deliberately designed to be as catastrophically horrible as possible when scaled down.
Yes, the image is designed to exploit the bug in way that makes it very obvious that the scaling is wrong, but no, the gray rectangle is not the correct solution. The problem is that the browser or the image application assumes that the brightness is stored on a linear scale, while in reality its stored exponential one. Thus when you scale things or apply other filters the brightness will be messed up. In real photos it will be less noticeable then here, but it will still happen.
The correct way would be to transform the image to a linear scale before applying the filter and then restoring it the exponential one for display. A simple example of how to do that in Gimp (using gausian blur instead of scale, but it is the same bug) would be this. The left image is the original from the article, the middle one is blur applied and the right one is applying a gamma of 0.5 to get it to a linear scale, then the blur and then a gamma 2 to restore it to its original scale. As you can easily see the right one looks like the left one, but blurred, while the middle one has no resemblance to the original, all the information got lost due to the bug. In practice you would need a higher bit depth to make this trick practical, as else you would end with banding artifacts.