Is Google's AI-Driven Image-Resizing Algorithm Dishonest? (thestack.com)

The Stack reports on Google's "new research into upscaling low-resolution images using machine learning to 'fill in' the missing details," arguing this is "a questionable stance...continuing to propagate the idea that images contain some kind of abstract 'DNA', and that there might be some reliable photographic equivalent of polymerase chain reaction which could find deeper truth in low-res images than either the money spent on the equipment or the age of the equipment will allow." An anonymous reader summarizes their report: Rapid and Accurate Image Super Resolution (RAISR) uses low and high resolution versions of photos in a standard image set to establish templated paths for upward scaling... This effectively uses historical logic, instead of pixel interpolation, to infer what the image would look like if it had been taken at a higher resolution.

It's notable that neither their initial paper nor the supplementary examples feature human faces. It could be argued that using AI-driven techniques to reconstruct images raises some questions about whether upscaled, machine-driven digital enhancements are a legal risk, compared to the far greater expense of upgrading low-res CCTV networks with the necessary resolution, bandwidth and storage to obtain good quality video evidence.
The article points out that "faith in the fidelity of these 'enhanced' images routinely convicts defendants."

Comments

Depends on enhancement

[guruevi]

You can't really upscale resolution but you can "enhance" images (especially raw ones) to a point. A lot of shots may be over or underexposed with some details left in one or more of the channels but visually blocked out, having thousands of minuscule changes and filtering go through a human in the hope of seeing something would be nearly impossible and having a filter to weed them out is helpful.

JPEG and similar compression are like MP3 - you can filter out what the algorithm defines as outside of the human realm to perceive but a lot of those assumptions are faulty leading to noticeable artifacts. However it is very hard to recover the data lost in "lossy compression" although you can make some assumptions to recover them.

The other problem with using these filters is that they're called artificial intelligences. They are not intelligent and calling them that leads to an assumption of infallibility. They're a form of Bayesian filtering and we've been using that since at least the days of OS/2 to "enhance" images, I used a demo of a program back then that did just that: inferences on JPEG to make a type of vector image. We just use more powerful clock cycles and more storage to have them perform better but they're not and never will be magic.

Who put the stick up his ass?

[pavon]

All upscaling algorithms are making up data based on assumptions on what "typical" hi-res images should look like given their low-res counterparts. That doesn't mean they are lying or misrepresenting. Furthermore, some assumptions are most statistically valid than others, and some produce more aesthetically pleasing results than others, actually resulting in images that are genuinely more likely to be closer to the true image than nearest neighbor.

Nowhere in google's paper are they suggesting that these images be used for forensic purposes, nor claiming that they are finding "deeper truth" or additional information in the images than what actually exists. They developed an approach that produces better results for common classes of images than previous algorithms, which is useful for a large number of applications that don't require the same level of rigor that forensics do.