Identifying Manipulated Images

Jamie found a cool story at MIT Tech Review. (As an aside, it sits behind an interstitial ad AND on 2 pages: normally I reject websites that do that, but it's a slow news day, so I'm letting it through.) Essentially, software is used to analyze light patterns in still photographs. Once you can figure out where the light sources are, it becomes a lot easier to determine if an image has been photoshopped.

Comments

No ads, all on one page

[wile_e_wonka]

The printer-friendly version:

http://www.technologyreview.com/printer_friendly_article.aspx?id=20423

Re:No ads, all on one page

[trolltalk.com]

The link works fine if, instead of clicking on it, you cut-n-paste it into a new browser tab. Here's what you get, if you can't be arsed to go to the trouble:

Monday, March 17, 2008
Identifying Manipulated Images
New tools that analyze the lighting in images help spot tampering.
By Erica Naone

Photo-editing software gets more sophisticated all the time, allowing users to alter pictures in ways both fun and fraudulent. Last month, for example, a photo of Tibetan antelope roaming alongside a high-speed train was revealed to be a fake, according to the Wall Street Journal, after having been published by China's state-run news agency. Researchers are working on a variety of digital forensics tools, including those that analyze the lighting in an image, in hopes of making it easier to catch such manipulations.

Tools that analyze lighting are particularly useful because "lighting is hard to fake" without leaving a trace, says Micah Kimo Johnson, a researcher in the brain- and cognitive-sciences department at MIT, whose work includes designing tools for digital forensics. As a result, even frauds that look good to the naked eye are likely to contain inconsistencies that can be picked up by software.

Many fraudulent images are created by combining parts of two or more photographs into a single image. When the parts are combined, the combination can sometimes be spotted by variations in the lighting conditions within the image. An observant person might notice such variations, Johnson says; however, "people are pretty insensitive to lighting." Software tools are useful, he says, because they can help quantify lighting irregularities--they can give solid information during evaluations of images submitted as evidence in court, for example--and because they can analyze more complicated lighting conditions than the human eye can. Johnson notes that in many indoor environments, there are dozens of light sources, including lightbulbs and windows. Each light source contributes to the complexity of the overall lighting in the image.

Johnson's tool, which requires an expert user, works by modeling the lighting in the image based on clues garnered from various surfaces within the image. (It works best for images that contain surfaces of a fairly uniform color.) The user indicates the surface he wants to consider, and the program returns a set of coefficients to a complex equation that represents the surrounding lighting environment as a whole. That set of numbers can then be compared with results from other surfaces in the image. If the results fall outside a certain variance, the user can flag the image as possibly manipulated.

Hany Farid, a professor of computer science at Dartmouth College, who collaborated with Johnson in designing the tool and is a leader in the field of digital forensics, says that "for tampering, there's no silver button." Different manipulations will be spotted by different tools, he points out. As a result, Farid says, there's a need for a variety of tools that can help experts detect manipulated images and can give a solid rationale for why those images have been flagged.

Neal Krawetz, who owns a computer consulting firm called Hacker Factor, presented his own image-analysis tools last month at the Black Hat 2008 conference in Washington, DC. Among his tools was one that looks for the light direction in an image. The tool focuses on an individual pixel and finds the lightest of the surrounding pixels. It assumes that light is coming from that direction, and it processes the image according to that assumption, color-coding it based on light sources. While the results are noisy, Krawetz says, they can be used to spot disparities in lighting. He says that his tool, which has not been peer-reviewed, is meant as an aid for average people who want to consider whether an image has been manipulated--for example, people curious about content that they find online.

Cynthia Baron, associate director of digital media programs at N

Limited utillity

[johnjaydk]

In a studio or other arranged settings it's pretty standard to use multiple lighting sources. So this tool will mainly be usefull for outdoor settings. If it's up-close and personal then it's also very common to use lights or other tools outside. Sooo this tool should be used with moderation.

Re:Expert User Required

[general_re]

TFA says an "expert user" is required. This expert user inputs coefficients that drive the equations that analyze the picture.

So basically, if you want an image to be doctored, you use one set of values. If you want an image to be genuine, you use another set of values. Maybe somebody else's requirements differ from mine, but this is not the kind of flexibility I want in a tool that is supposed to tell me if an image has been altered or not. Ummm, what? FTA:

Johnson's tool, which requires an expert user, works by modeling the lighting in the image based on clues garnered from various surfaces within the image. (It works best for images that contain surfaces of a fairly uniform color.) The user indicates the surface he wants to consider, and the program returns a set of coefficients to a complex equation that represents the surrounding lighting environment as a whole. That set of numbers can then be compared with results from other surfaces in the image. If the results fall outside a certain variance, the user can flag the image as possibly manipulated. I mean, that's not even close to what you posted - "running the same analysis on different parts of the image and then comparing the results" is not the same as "you pick the results".

Re:That should help

[zotz]

Funny thing, you don't always have to shop things to get odd results:

My vids on youtube:

http://www.youtube.com/user/zotzbro

If you check the comments on the "UFO vs Paper plane test" you will see people talking of a real one.

Perhaps on some of the paper plane instruction vids too. If you watch those, as the camera pans in one of them, after the construction and before the flight test, you can see what the "UFO" really is.

all the best,

drew
http://zotzbro.blogspot.com/

Found the episode

[jimwelch]

Columbo and the Murder of a Rock Star, 1991 and yes Dabney Coleman was the bad guy.

Re:That should help

[sm62704]

It's not needed and won't help. Most of the UFO photos are pre-Photoshop and were done with different methods:

• Have a small model of the UFO and fling it into the air high enough that there's no context. Although those CAN be detected, they can't by this software.
  
• The objects are secret military aircraft, not alien craft. The hoax of alien craft is started by the government (pick one) to mask the true meaning of the object photoed. This software won't help with that, either
  
• It's something else flying around up there. Is it a bird? Is it a plane? Is it a weather balloon? Is it ball lightning? Who knows? If it's a flying thing and you don't know what it is, then it's an Unidentified Flying Object. This tool won't help here, either.

This tool can't do anything someone trained in art can't do. The first thing you learn in art school is how to see. You can't draw if you can't see, and that's usually the biggest reason most people can't draw.

As one of my instructors used to say, "I don't know what I like but I know what art is."

-mcgrew

Faked non-uniformly lit

[abqaussie]

The tool doesn't tell you if a photo is faked, it just analyzes whether there are light sources in the image that are not affecting different objects in the image the same way. From what I can tell it tries to tell if the way the light hits different objects in the picture "agree" with one another based on the position of the object, color, and probably other attributes not detailed in the article. If the photographer is controlling the light at all, using off-camera flash, focusing their light on some parts and blocking it from others, etc, then there would be components of the image that deliberately don't match when it comes to the lighting. People do that all the time, both deliberately and accidentally, when lighting a photo. Because the photographer has deliberately put a light on the subject that isn't hitting other elements, background, objects, the same way as it's hitting the subject. So it seems like the analysis would work great for cases where the light is ambient, and should affect all objects in the frame relatively the same. Otherwise it'd have a bad day.

Re:Steganography

[blueg3]

There's not much wrong with steganography of encrypted data, particularly if the data in the covert channel would have been statistically similar to random data anyway.

Most image steganography isn't that great, though, and steganography by a well-known means of cleartext data is fairly pointless.

Another good tool for detecting photoshopping

[Reziac]

...and for diagnosing damaged JPGs (I used it extensively when reconstructing mangled JPGs from someone's disk crash):

JPEGsnoop, by Calvin Hass
In very active development; suggestions and bug reports welcome. Free download from http://www.impulseadventure.com/photo/jpeg-snoop.html

Voodoo software

[Anonymous Coward]

The "tool" described in the article is a joke and reminds me of one (that also got posted on Slashdot, some time ago) whose author claimed to be able to tell if an image was a composite of several sources by "analyzing the compression patterns". Turned out all his tool did was check the quantization level of JPEG macroblocks... which is always defined by the last encoding. In other words, all it did was detect which parts the (last) encoder has decided to compress more, and which parts it had decided to compress less. Basically it worked like a (really bad) edge detection filter.

The description of the "tool" in this article is just as nonsensical. If a pixel is brighter than its neighbor, that tells you nothing about the position of the light source. If you want to determine that, you need to do some very complex spatial analysis, like the stuff done by Paul Debevec (and it's still only an approximation).

Human eyes and brains are much better at spotting lighting inconsistencies than any algorithm, because they can fill in 3D information from 2D images (which you simply cannot do without a powerful interpretative visual system). This tool (which, as the article states "has not been peer-reviewed" - what a surprise!) is just a hack put together to impress some wannabe "security consultants" and get a fat check for a quick patent buy-out.