Detecting Video & Audio Tampering

* * Beatles-Beatles writes "Dartmouth professor Hany Farid already devised software tools to detect when someone has tampered with digital photos. His next challenge: determining whether video or audio files have been retouched. "

Comments

FP

[Anonymous Coward]

I tampered with this first post, making it a later post.

Umm...

[dduardo]

How about just doing md5sum ?

Re:Umm...

[Schraegstrichpunkt]

Depending on who is doing the tampering, that doesn't necessarily work.

You can thank the Chinese for blabbing it to everyone.

Re:Umm...

[abandonment]

[quote]
Reuters, an international news wire service, caught heat by publishing a Beirut battle photo that contained an extra plume of smoke for dramatic effect. (Farid's software helped reveal that enhancement.)
[/quote]

they needed software to detect this? the 'enhancements' were so blatantly obvious that anyone that's ever used photoshop would have been able to see them...

hilarious

National Inquirer, Beware

[LifesABeach]

I'm thinking of those home movies of 'Bat Boy'; Now we will be able to find out the truth.

Re:National Inquirer, Beware

[diggum]

I believe Bat Boy is the sole editorial property of Weekly World News. Not that I've ever read it or anything...

Nevermind

[dduardo]

I was assuming he had the orignal picture/video/audio in hand, but he doesn't it.

Re:Nevermind

[Anonymous Coward]

I was assuming you had read the original article in hand, but you hadn't done it.

Re:Nevermind

[MustardMan]

You must be new here. Welcome to slashdot, I suggest you click that little linky up top and register, it's lots of fun!

No can do

[Anonymous Coward]

There's no way to know that an audio signal has been tampered with unless you have prior knowledge of the recording equipment and enviromental conditions. Consider the case of someone moving around during a cell phone conversation, it's not possible to factor in all the variables. You can detect a standard edit but even if you could detect expert manipulation, you still could not disprove that it was a convertor glitch, positional or level change.

You can even duplicate filtering of a given D/A with relative ease.

Re:No can do

[ResidntGeek]

You're right. It's too bad the Dartmouth professor didn't ask the ACs on slashdot about his work; they obviously know more about it than him.

Re:No can do

[Robot Randy]

Reminds me of the one and only Numb3rs episode I ever saw. It involved proving a "Suicide" was actually a murder. Seems there was a Zupruder style "Open Police Microphone" and the math whiz star is able to deduce that there was someone else in the room during the killing.

Funny thing is when we see the guy meticulously laying everything out for the test recording we have a policeman laying on the bed shooting a gun into the ceiling... Funny, but I though the dead person fired a gun into her head.

And they wondered why the acoustic signature was different???

Re:No can do

[Anonymous Coward]

He mentions it himself in TFA, do you have an issue with someone expanding on information presented in the mainstream press? Is someone who posts anon automatically less knowledgable by vitue of anonymity? Did you stop to think that perhaps there are people posting here as AC with degrees in acoustics?

Do us all a favor and save your funnies for the intellectual void known as myspace.

Re:No can do

[exp(pi*sqrt(163))]

Judging by the descriptions in the article this guy is interested in looking at tampering by fairly incompetent people. That's a more tractable problem. And in practice - many people tampering with photos and video are incompetent - well, at least the ones we know about are.

Re:No can do

[Gr8Apes]

He said that audio signals were easier to spoof since our auditory senses are more forgiving. Also, TFA only mentions video work in any detail whatsoever, with references that video techniques might be applicable to audio as well.

And might I mention that this work is more in the fields of mathematics or signals filtering/processing than acoustics? (BTW, who gives degrees in acoustics? I know of ME's with specialization in acoustics engineering, but that's a whole different field than this signals analysis)

Re:No can do

[Anonymous Coward]

Since you want to be pedantic, he didn't mention anything about spoofing.

"Audio is not that difficult to tamper with. Our auditory system is fairly forgiving,"

Your auditory system may be "more in the fields of mathematics or signals filtering/processing", but mine certainly isn't and who the hell does signals processing? I've known people do digital signals processing but to specialize in audio signals processing would require a working knowledge of acoustics, which according to you, this does not.

Re:No can do

[ResidntGeek]

He said it was _more_ in the field of signals processing than acoustics, not that it had nothing to do with acoustics. And he was perfectly correct.

Re:No can do

[Gr8Apes]

spoof, tamper, alter, fidget with, fuck with

OK, hopefully that clarifies a little something or two. As for finding fiddled with files (yes, that's another synonym) programmatically is going to require data processing by software more in line with that of signals processing of the sort done by... wait for it... signals processing applications, not "acoustics processing applications".

Re:No can do

[Anonymous Coward]

I know of ME's with specialization in acoustics engineering, but that's a whole different field than this signals analysis

to specialize in audio signals processing would require a working knowledge of acoustics, which according to you, this does not.

he was perfectly correct.

As was I but don't take an AC's word for it, go and ask a Dartmouth professor to explain the meaning of the term "whole different field"!

Re:No can do

[belmolis]

MIT has an ocean acoustics program. That at least used to deal to a large extent with submarine detection, with lots of military research money available. MIT also has an acoustics and vibration lab. And Amar Bose, who may well have designed your loud speakers, taught acoustics at MIT until his retirement in 2000. I took his intro acoustics course many years ago. I think that he may still teach a course. He is still listed in the MIT directory.

Re:No can do

[Anonymous Coward]

signals processing of the sort done by... wait for it... signals processing applications

Except that you are storing... wait for it... acoustic information. At least, that's the part I was discussing since TFA is about detecting tampering with images, video and audio data. FTA:

unexpected patterns in background noise and duplication detection could be employed in examining audio transcripts

The background noise here is clearly an acoustical phenomena, unless you're telling me that the noise from your computer fan is generated by DSP?

Re:No can do

[Gr8Apes]

Except that you are storing... wait for it... acoustic information. At least, that's the part I was discussing since TFA is about detecting tampering with images, video and audio data

Of course it is, that's what the data represents. But we're working on the data only. Which goes right back to signals processing. Because the processes in the article do not recreate the picture/movie/audio for examination, what they do is look for deviations in the data from what's expected. i.e., pixels that don't match/line-up, audio signals that suffer sudden phase disruptions or offsets. The little things you'd never see or hear.

All the time

[ericdano]

I see/hear this all the time. It's called a DUB. American companies taking a perfectly good foreign film or cartoon (Anime) and butchering it. Latest example would be Bleach.....

Re:All the time

[firebus]

the problem, for a violent or bloody (or sexy) title like bleach or naruto or (insert sexy title here) is that you can't sell it at walmart or put it on tv without censoring it. and if you can't sell it at walmart or put it on tv, it's not worth the money you have to pay to japan for the rights to the title.

i'd love it if american anime publishers would plan on an "uncut DVD" version from the start so that, once there's a big enough fan base, they can sell the real thing through niche market and online. it's a chance to make money twice on the same content! come on! just do it!

if you need the real thing, download the fansubs. if you're a real fan, buy the american version too.

Re:All the time

[Firefly1]

the problem, for a violent or bloody (or sexy) title like bleach or naruto or...

Good point; however, it's merely a symptom of another problem: the perception of 'animation = for kids' on this side of the Pac. To be fair, some projects (such as Cool World have attempted to break out of that mold...

i'd love it if american anime publishers would plan on an "uncut DVD" version from the start...

Yeah, why don't they? This is already done with quite a few movies, as I recall. And the popularity of fansubs leads me to conclude that such things are likely to find willing buyers.

Re:All the time

[Anonymous Coward]

....taking a perfectly good foreign film or cartoon (Anime) and butchering it.

Yeah, like that guy named after a cigarette did to Lord of the Rings!

security through obscurity

[TubeSteak]

Most likely, distribution will be limited: Photo editors, but not freelance photographers, at mainstream media outlets may get the software.

"You do diminish the power the software if you make it completely, widely available," he said.

Since they mention JPEG quantization tables as one of the main methods of identifying what program/camera a photograph came from... How hard would it be to replace the quant table with your own? Or even just tweak it enough that the program can't ID it.

It just doesn't strike me as a terribly reliable way to ID a picture's origin. Might as well rely on the EXIF data.

Re:security through obscurity

[Schraegstrichpunkt]

Hence the limited distribution. He's well aware that his mechanism is not foolproof. The underlying assumption is that even though you can defeat this detection mechanism, you won't.

Re:security through obscurity

[plover]

Quantization is only one facet of many means of detecting manipulation. If you're trying to claim a certain picture is legitimate, you may be requested to submit the camera for evidence. If anything but an image completely consistent with the camera's capabilities shows up, the image could be disregarded, and you may be liable for perjury.

A real world image will have a lot of artifacts tying it back to a particular camera make and model, and a base noise level that is a virtual fingerprint identifiable to a specific camera. Are you sure you can manipulate an image that still defeats ALL of these checks, even though you don't know what they all are?

Yes, it's absolutely a case of security through obscurity. He'd like to keep the program "away" from untrustworthy photographers who would use it as a photoshop-litmus test, and place it only in the hands of editors and police forensics labs. And researchers vetting data in submitted conference papers. And professional "expert witnesses." And the list of exceptions will grow as long as your arm, as if a program like this will never end up in "the wrong hands".

The real question is: did you do such a good job photoshopping that you'll score a "perfect 10" in a courtroom WITHOUT the luxury of first running his program yourself? Are you willing to bet 2 years in prison for perjury on that picture?

Of course, can anyone prove you do or don't have a copy of his program?

There'd be an awful lot of things you'd have to get right. Does the EXIF data exactly match the EXIF data the camera's firmware version produces? Are all the values legitimate (if the SD450 camera can't do 1/400th of a second exposure or f1.8, how'd they get set?) Do all the shadows line up correctly? Is the sun in the right position for the time of day and location the image indicates? Does the "noise level" change around the face where you may have used the 'magic wand' tool to cut and paste a second image? Do the foreground and background show evidence of differing resolutions? Does the image size match one of the possible image sizes produced by this camera? Do any black/white transition areas have "false colors" consistent with the arrangement of image sensors in that particular camera, and do they ALL match? Is depth-of-field consistent with the EXIF data, or is the background too sharp/blurry? Is flash evident, and consistent with the camera's make/model? Is the sensor 'noise level' consistent, and does it match the noise profile of the camera? Does the weather match the almanac for that day?

Those are just some off-the-cuff things I'd expect an analysis program to check for. It would be a lot of work to make an image pass all of these checks.

Re:security through obscurity

[mcmonkey]

A real world image will have a lot of artifacts tying it back to a particular camera make and model, and a base noise level that is a virtual fingerprint identifiable to a specific camera. Are you sure you can manipulate an image that still defeats ALL of these checks, even though you don't know what they all are?

Yes.

Just take a picture with the same camera. There may well be signs the image has been manipulated, but in terms of tying elements of a photograph back to an individual camera, why not just use the same camera? E.g. I have a picture of some independently verifiable event. I take a picture of myself taken with the same camera and photoshop my image into the event to construct an alibi.

That technique may not cover up signs of a "magic wand", but it would take care of matching all the idiosyncrasies of an individual camera. If I planned ahead, I could make sure my distance and position relative to the camera are consistent with my placement in the final photo. This step prevents signs of enlargement/reduction and would make the image correct in terms of artifacts, even though I don't know what they are.

I'm not saying I can produce undetectable fakes, but in terms of the 'aha! the data in this portion of the picture doesn't match the camera firmware' or 'aha! this area of the picture shows a slight blurring from a defect in the lens while this one face in the crowd is in perfect focus' I think I'm covered.

Re:security through obscurity

[plover]

in terms of tying elements of a photograph back to an individual camera, why not just use the same camera?

Even that might not be good enough. The camera will have to be in the same orientation, as that will affect both JPEG quantization and Moire artifacts. You may need the "paste-in" subject to be at the same part of the image sensor as he'll be in the final image. And it still has to be edited to match the other attributes -- depth of field, lighting, and focus, all of which are features experts commonly use to determine validity.

Think about what an expert would do if he were to verify your picture. He'd yse your camera to photograph a "white" card several times, and would examine the output carefully for dead pixels. Let's say that he found a green cell was out at (1000,500), leaving a single tiny purple pixel, and a blue cell was weak at (1010,550), leaving a yellow pixel. Next, he'd look at your photographic "evidence". If the cell at (1000,500) has any green component or the cell at (1010,550) has more than 25% blue, he'd suspect the image of having been manipulated. Similarly, if he finds an anomalous dead green pixel in your output image at (1500,750) and a weak blue pixel at (1510,800) he might question that as well.

Yes, I realize you'll remove (or obscure) many of the differences by using the same camera. It covers a lot of faults that his tamper detector supposedly can find. But without access to the analysis program, are you still really, really sure that you won't get caught?

If you're interested in the topic, I'd suggest googling for steganography and especially steganalysis. They've done a lot of work in finding "hidden" data, which is where a lot of the photoshop trickery becomes evident.

Re:security through obscurity

[mcmonkey]

Let's say that he found a green cell was out at (1000,500), leaving a single tiny purple pixel, and a blue cell was weak at (1010,550), leaving a yellow pixel. Next, he'd look at your photographic "evidence". If the cell at (1000,500) has any green component or the cell at (1010,550) has more than 25% blue, he'd suspect the image of having been manipulated. Similarly, if he finds an anomalous dead green pixel in your output image at (1500,750) and a weak blue pixel at (1510,800) he might question that as well.

That's why I'd have to plan ahead...

If I planned ahead, I could make sure my distance and position relative to the camera are consistent with my placement in the final photo.

The scenario in my mind is this: there's a bank heist at 12:05 on the East side. I'm a suspect with no alibi. Just so happens on that same day, a little before noon, there was a car accident on the West side, and someone on the scene took a picture of the clean-up with a digital camera.

If I'm in that picture, that would help in the alibi dept. Police and EMT logs would help establish the time line. So, a few days later at close to the same time of day, on a day with similar weather (to get the lighting right), I get someone to stand in the same spot as the original shutter bug (lucky for me she was standing in mud that still holds her footprints).

I then stand on the side of the road where a crowd of on-lookers had gathered, careful to stand where there was a small gap in the crowd, so I'm in a position I could have occupied in the original photo.

Any color shift, any dead pixels, and systematic anomalies should match between my torso and the surrounding crowd. I don't need to know what hidden data I'm matching. I just know I've matched the conditions creating that data.

Of course, I've only covered those systematic issues. Any random fluctuations that vary from picture to picture won't match up, and I still need to worry about direct evidence of image manipulation (magic wand effects).

My point is not that I could paste myself into a picture with a quality that I would stake my freedom on. My point is only that if you are faking pics and expect your audience to trace elements back to their source, make sure all the elements you have pasted together come from the same source.

Dithering

[Anonymous Coward]

Digital still cameras from different manufacturers, and often different cameras from the same manufacturer, operate under different JPEG quantization tables, Farid said. These tables determine that rate at which a camera will drop data in compressing a photograph. Farid's group has come up with software for examining the quantization tables among different cameras.

If someone removes or alters the EXIF data, the best this technique can do is identify which JPEG implementation was used to compress the image. PS output images can apparently be identified via their quantization table, if quantization data can be read after the fact, can it be reverse engineered and spoofed? Is this (from libjpeg) an example of what we're talking about?

static const UINT8 base_dither_matrix[ODITHER_SIZE][ODITHER_SIZE] = {
/* Bayer's order-4 dither array. Generated by the code given in
  * Stephen Hawley's article "Ordered Dithering" in Graphics Gems I.
  * The values in this array must range from 0 to ODITHER_CELLS-1.
  */
  { 0,192, 48,240, 12,204, 60,252, 3,195, 51,243, 15,207, 63,255 },
  { 128, 64,176,112,140, 76,188,124,131, 67,179,115,143, 79,191,127 },
  { 32,224, 16,208, 44,236, 28,220, 35,227, 19,211, 47,239, 31,223 },
  { 160, 96,144, 80,172,108,156, 92,163, 99,147, 83,175,111,159, 95 },
  { 8,200, 56,248, 4,196, 52,244, 11,203, 59,251, 7,199, 55,247 },
  { 136, 72,184,120,132, 68,180,116,139, 75,187,123,135, 71,183,119 },
  { 40,232, 24,216, 36,228, 20,212, 43,235, 27,219, 39,231, 23,215 },
  { 168,104,152, 88,164,100,148, 84,171,107,155, 91,167,103,151, 87 },
  { 2,194, 50,242, 14,206, 62,254, 1,193, 49,241, 13,205, 61,253 },
  { 130, 66,178,114,142, 78,190,126,129, 65,177,113,141, 77,189,125 },
  { 34,226, 18,210, 46,238, 30,222, 33,225, 17,209, 45,237, 29,221 },
  { 162, 98,146, 82,174,110,158, 94,161, 97,145, 81,173,109,157, 93 },
  { 10,202, 58,250, 6,198, 54,246, 9,201, 57,249, 5,197, 53,245 },
  { 138, 74,186,122,134, 70,182,118,137, 73,185,121,133, 69,181,117 },
  { 42,234, 26,218, 38,230, 22,214, 41,233, 25,217, 37,229, 21,213 },
  { 170,106,154, 90,166,102,150, 86,169,105,153, 89,165,101,149, 85 }
};

It looks to me like a simple program could make this forensic technique redundant by re-quantizing an image to match a specific fingerprint. Am I missing something or is it really so easily defeated?

Simple.

[Rob T Firefly]

If Paris Hilton is fully dressed, seems fully aware of her surroundings, and/or is singing well, it's been tampered with.

This formula can also be adapted to Lindsay Lohan, but hasn't been tested on Tara Reid or others yet.

Nobody familiar with this?

[Anonymous Coward]

Here are some more interesting comments from the libjpeg source code.

* Heckbert, Paul. "Color Image Quantization for Frame Buffer Display",
* Proc. SIGGRAPH '82, Computer Graphics v.16 #3 (July 1982), pp 297-304.
--
* Heckbert-style quantizers vary a good deal in their policies for choosing
* the "largest" box and deciding where to cut it. The particular policies
* used here have proved out well in experimental comparisons, but better ones
* may yet be found.
--
* To improve the visual quality of the results, we actually work in scaled
* RGB space, giving G distances more weight than R, and R in turn more than
* B. To do everything in integer math, we must use integer scale factors.
* The 2/3/1 scale factors used here correspond loosely to the relative
* weights of the colors in the NTSC grayscale equation.
* If you want to use this code to quantize a non-RGB color space, you'll
* probably need to change these scale factors.

* These routines are concerned with the time-critical task of mapping input
* colors to the nearest color in the selected colormap.
*
* We re-use the histogram space as an "inverse color map", essentially a
* cache for the results of nearest-color searches. All colors within a
* histogram cell will be mapped to the same colormap entry, namely the one
* closest to the cell's center. This may not be quite the closest entry to
* the actual input color, but it's almost as good. A zero in the cache
* indicates we haven't found the nearest color for that cell yet; the array
* is cleared to zeroes before starting the mapping pass. When we find the
* nearest color for a cell, its colormap index plus one is recorded in the
* cache for future use. The pass2 scanning routines call fill_inverse_cmap
* when they need to use an unfilled entry in the cache.
*
* Our method of efficiently finding nearest colors is based on the "locally
* sorted search" idea described by Heckbert and on the incremental distance
* calculation described by Spencer W. Thomas in chapter III.1 of Graphics
* Gems II (James Arvo, ed. Academic Press, 1991). Thomas points out that
* the distances from a given colormap entry to each cell of the histogram can
* be computed quickly using an incremental method: the differences between
* distances to adjacent cells themselves differ by a constant. This allows a
* fairly fast implementation of the "brute force" approach of computing the
* distance from every colormap entry to every histogram cell. Unfortunately,
* it needs a work array to hold the best-distance-so-far for each histogram
* cell (because the inner loop has to be over cells, not colormap entries).
* The work array elements have to be INT32s, so the work array would need
* 256Kb at our recommended precision. This is not feasible in DOS machines.
*
* To get around these problems, we apply Thomas' method to compute the
* nearest colors for only the cells within a small subbox of the histogram.
* The work array need be only as big as the subbox, so the memory usage
* problem is solved. Furthermore, we need not fill subboxes that are never
* referenced in pass2; many images use only part of the color gamut, so a
* fair amount of work is saved. An additional advantage of this
* approach is that we can apply Heckbert's locality criterion to quickly
* eliminate colormap entries that are far away from the subbox; typically
* three-fourths of the colormap entries are rejected by Heckbert's criterion,
* and we need not compute their distances

The Four Horsemen of the Infocalypse

[ChaosDiscord]

A story on detecting photomanipulation apparently wasn't interesting in and of itself, so the author felt the need to drag in one of the Four Horsemen of the Infocalypse: "Child pornographers also employ photo retouching to skirt felony laws."

No amount of photo retouching makes sexual abuse of a child legal. The only way I can see to "skirt" the law would be to transform the images so they plausible look artificial (Court rulings have upheld that as long as no children are involved, it's protected by the first amendment. Thus drawings, paintings, and 3-d models depicting child abuse are legal). Of course, if you're going to transform the image that much, you can legally get artificial images that look that way in the first place. Perhaps there are a few cases where this has been done, but I'm not buying that this is a real problem.

Not always necessary.

[crystalattice]

Reuters, an international news wire service, caught heat by publishing a Beirut battle photo that contained an extra plume of smoke for dramatic effect. (Farid's software helped reveal that enhancement.)

Like we really needed the software to show that a Photoshop clone tool was used. Nearly every person who saw it said it looked fake; even people who don't know how to use Photoshop said it looked "wrong".

Edit adult to look like a child

[Myria]

Different way: edit a pornographic adult picture to look like a child, like by cutting and pasting innocent child pictures onto a legal pornographic picture. Of course, it's not as simple as cutting and pasting, but you get the idea.

As sick as these people are, I don't see why we should throw them in jail for that...

Melissa