Slashdot Mirror

← Back to Stories (view on slashdot.org)

The Defense Department Has Produced the First Tools For Catching Deepfakes (technologyreview.com)

Posted by msmash on from the fighting-back dept.

Fake video clips made with artificial intelligence can also be spotted using AI -- but this may be the beginning of an arms race. From a report: The first forensics tools for catching revenge porn and fake news created with AI have been developed through a program run by the US Defense Department. Forensics experts have rushed to find ways of detecting videos synthesized and manipulated using machine learning because the technology makes it far easier to create convincing fake videos that could be used to sow disinformation or harass people. The most common technique for generating fake videos involves using machine learning to swap one person's face onto another's. The resulting videos, known as "deepfakes," are simple to make, and can be surprisingly realistic. Further tweaks, made by a skilled video editor, can make them seem even more real. Video trickery involves using a machine-learning technique known as generative modeling, which lets a computer learn from real data before producing fake examples that are statistically similar. A recent twist on this involves having two neural networks, known as generative adversarial networks, work together to produce ever more convincing fakes. The tools for catching deepfakes were developed through a program -- run by the US Defense Advanced Research Projects Agency (DARPA) -- called Media Forensics. The program was created to automate existing forensics tools, but has recently turned its attention to AI-made forgery.

45 comments

  1. Not just pasting faces, and not just video by Rick+Schumann · · Score: 1

    https://www.wnycstudios.org/st...
    The original image can be manipulated, and the audio even more convincingly manipulated to say pretty much anything you want.

    1. Re:Not just pasting faces, and not just video by admin7087 · · Score: 1

      That's one of the worse examples and wouldn't convince many people, but the problem is going to be massive once the techniques have been improved. The real problem is that people get their "information" from the most shady sources and tend to switch their brain of. Captain Disillusion's Youtube Channel is a great and entertaining way of becoming more skeptical.

    2. Re:Not just pasting faces, and not just video by Anonymous Coward · · Score: 0

      uploading a video which labels a five year old event as something that just happened today? Or the people shown in a politically sensitive event intentionally labeled as the opposition?

    3. Re:Not just pasting faces, and not just video by rtb61 · · Score: 1

      People will believe what they want to believe in order to favour their own personal biases and even their own particular personalities. They will claim to believe it, even if they don't and they will claim disbelief even if they believe, belibers (no accident) will be beliebers, idiot just-ins to the world of thoughts.

      Here's a news flash for you, prove the photo or vid is fake, so the fuck what, they'll just claim yours is fake about the fake because yeah, it can be. Unless you legalise the truth, legalise news as a profession and prosecute for lying in the news, with news as a protected word, you are stuck in a fake loop. Your fake, their fake, your fake, their fake, endless loop, of fake news.

      So the professional paranoid now have a way of detecting fakes, well, you know exactly what they are going to do with that, make sure their fakes pass the fucking test ;D (you know that is true, all of your, know it is 100% true, believe me ;D).

      --
      Chaos - everything, everywhere, everywhen
  2. I found a way to defeat it. by Anonymous Coward · · Score: 0

    AI to find fake news, needs to compare real news to fake news.

    So, the Ruskies just need to publish a story that says that all news is fake.

    And then a story stating that the news about fakes news is fake.

    That'll shut the AI down.

    I saw it once on a TV show. They told the computer that one guy always lies and then he tells the computer that he's lying.

    Another way to defeat the AI is to have someone in authority pontificate about humanity or some such thing and eventually the computer sees the errors in its logic and self destructs. It really screws it up if you put periods. at. the. end. of. every. word.

    1. Re:I found a way to defeat it. by Anonymous Coward · · Score: 0

      Referencing ST:TOS episodes

    2. Re:I found a way to defeat it. by Anonymous Coward · · Score: 0

      AI to find fake news, needs to compare real news to fake news.

      So, the Ruskies just need to publish a story that says that all news is fake.

      And then a story stating that the news about fakes news is fake.

      That'll shut the AI down.

      I saw it once on a TV show. They told the computer that one guy always lies and then he tells the computer that he's lying.

      Another way to defeat the AI is to have someone in authority pontificate about humanity or some such thing and eventually the computer sees the errors in its logic and self destructs. It really screws it up if you put periods. at. the. end. of. every. word.

      Is this a joke or are you stuck in 70s knowledge of computer??? The machine could only handle simple straight forward logics back then. Nowadays, machine learning helps a lot, so the machine nowadays wouldn't simply shut down.

  3. let me guess by Anonymous Coward · · Score: 0

    Trying to get ahead of damning video/audio evidence of crimes that will be coming out soon.

    1. Re:let me guess by Anonymous Coward · · Score: 1

      This.

      The whole reason this near real-time video editing was released by the CIA and/or NSA was because someone somewhere has some video of some politician doing something so incredible heinous, there needed to be a plausible deniability already in the ether.

    2. Re: let me guess by Anonymous Coward · · Score: 0

      You could have just said "the peepee tape".

    3. Re: let me guess by Anonymous Coward · · Score: 0

      If it was a peepee tape then no-one would care.
      The fact that Trump paid hookers for sex while his wife was pregnant didn't exactly gain any traction.
      A video of him watching hookers pee on a bed isn't going to damage him whatsoever.
      It has to be something way more incriminating than that.

      My guess is a video involving underage sex slaves. It would explain the pizzagate projections.
      But given his supporters it might be more damaging if it s a video of him having sex with a consenting adult male or a video of him dropping off some woman at an abortion clinic.

    4. Re: let me guess by Anonymous Coward · · Score: 0

      "the peepee tape".

      This level of projection is incredible. Trump is who I assume you are discussing. He is not the only politician. Also, I struggle to see how this allegation is so heinous, it necessitated burning CIA and/or NSA capability.

    5. Re:let me guess by Anonymous Coward · · Score: 0

      45,000 sealed indictments on the books atm. A storm is coming.

  4. But.... by Anonymous Coward · · Score: 0

    Clicking on the word deep fake in the summary shows the Germans already did it. I'm so confused.

  5. Scarier than it sounds by omnichad · · Score: 4, Insightful

    Any tool that can catch a deepfake can be used to help train the deepfake generator. This cover story just tells us that DARPA is working on training their own system for generating deepfakes. Dueling neural nets is the new normal and there's no reason to think they only developed one side of this.

    1. Re:Scarier than it sounds by Anonymous Coward · · Score: 0

      Any tool that can catch a deepfake can be used to help train the deepfake generator. This cover story just tells us that DARPA is working on training their own system for generating deepfakes. Dueling neural nets is the new normal and there's no reason to think they only developed one side of this.

      This is true. How do you calibrate the fake-catching tool without generating fakes for it to find?

      There is an analogy to be made between these dueling nets and computer security research.
      Security researchers know the same tools as the black hats, because it's the only way to know how to secure something against the black hats.

    2. Re:Scarier than it sounds by Aighearach · · Score: 1

      I hope this is true, because without deepfakes how are slashdot users going to use dating sites?!

      Think of the innocent neckbeards, people, think of the neckbeards.

    3. Re:Scarier than it sounds by nitehawk214 · · Score: 1

      But any tool that can generate deepfakes can be used to train the deepfake detector.

      It will eventually conclude that it must nuke humanity.

      --
      I'm a good cook. I'm a fantastic eater. - Steven Brust
  6. I'm Convienced by Anonymous Coward · · Score: 1

    That this will be a cat and mouse game of trying to evade detection. What we will need and likely eventually settle on is video & audio that includes digital certificates embedded in the video encoding. That way there is a certified chain of proof from the moment the video is recorded to the point of distribution. This will likely matter in cases where legal burden of proof is required - security camera feeds, official court & governmental documentation, and news anchor releases.

  7. O come on by Anonymous Coward · · Score: 0

    The tool is clever but it's not AI.

    Damnit not every novel script is artificial intelligence!

  8. That's the geek test. by Anonymous Coward · · Score: 0

    Referencing ST:TOS episodes

    Any geek who needed the parent's post to understand the GP ..... imagine Donald Sutherland at the end of the "Invasion of the Body Snatchers" 1970's remake.

  9. where this technology will go now by slashmydots · · Score: 3, Funny

    Well time to give it up and make technology for "dank fakes" where you take a Shiba Inu's head and render it onto someone's body in a video.

  10. oblig. bonequest by Anonymous Coward · · Score: 0

    spigot: HOW TO SPOT A DEEP FAKE!!!!!
    spigot: STEP ONE: LOOK AT IT

  11. Futurama FTW by fuzznutz · · Score: 1

    I saw it once on a TV show. They told the computer that one guy always lies and then he tells the computer that he's lying.

    You need to update your references...

    Robot Santa: Nice try, but my head was built with paradox-absorbing crumple zones.

  12. Re:Billionaire, Author, Tv Star, President.... by Anonymous Coward · · Score: 0

    Any comparison to... tax fraud, treason, daughter leering? Nothing. I am not able to be readily compared to the obese orange lying traitor. I get way more than 2 hours of sleep a night and won't die in Federal prison a traitor. Way different.

  13. How to spot Deep Fakes... by Anonymous Coward · · Score: 0

    Watch hours and hours of porn...AI loves porn.

  14. Any tutorials to use it for not-porn? by Wulf2k · · Score: 1

    Here's probably as good a place as any to ask.

    Any time I've tried some simple searching on how-to, it's started going down porn paths. Does anybody have a porn-free tutorial on how to get started on creating a deepfake?

    I don't care to create anything that would fool anybody, I just think it'd be neat to screw around with over a few lunch hours at work.

    1. Re:Any tutorials to use it for not-porn? by Gilgaron · · Score: 1

      Other than the whole NSFW aspect the methodology should be about the same either way?

    2. Re:Any tutorials to use it for not-porn? by Kjella · · Score: 1

      Not really, but I've seen the results of people trying it out and it seems extremely fickle, needs a pretty big training set and you have to mask out other faces if there are any. People use it on celebs because there's tons of photos/videos and someone OCD enough to compile the set. And when they're trying to morph it into porn all the scenes that don't work and all the times it loses track doesn't matter so much. You just pick the clips the virtual mask sticks and glue it together. You'll notice that in a lot of the scenes the woman isn't actually moving her head much so it's a relatively straightforward face replacement, it has trouble with all sorts of half obscured faces, partial faces entering/leaving the camera, accessories like hats or glasses and so on.

      In the end it's a really cumbersome process that probably won't work on that cool clip you think it will. Sure you can probably morph someone you know into a Hollywood movie for a few scenes, but it's still a lot of work for a gag. If you just want to put yourself in there it's probably just easier to shoot some footage on green screen and overlay it rather than deep fake it. It's literally for when you want the person being deep faked to take over that actor/actress' performance... Oh also it just works for the face itself, so if the body or hair color/length/style is a mismatch it looks like your faked person in a wig or worse. Another reason why porn is easier, with so much to choose from you can find a similar body that just needs a new face to be a convincing fake.

      --
      Live today, because you never know what tomorrow brings
    3. Re:Any tutorials to use it for not-porn? by Wulf2k · · Score: 1

      Gag-level quality was pretty much what I was going for, but it sounds like the effort/reward ratio really isn't there.

      I'll probably still poke at it sometime but I'll keep my expectations low.

  15. SFW - Safe For Work Tutorials by Anonymous Coward · · Score: 1

    Safe For Work Tutorials:
    https://www.deepfakes.club/

    The easiest way to get started in deep fakes is to use the Python scripts.

    1. Re:SFW - Safe For Work Tutorials by Wulf2k · · Score: 1

      At a quick glance that seems to be ideal.

      Thank ye, noble AC.

  16. I can't see this arms race lasting long by Anonymous Coward · · Score: 0

    Although the one guy who keeps on replying "this isn't AI" to every single "machine learning" article wouldn't agree.

    Soon it will be impossible to tell a fake apart from the real thing.

    1. Re:I can't see this arms race lasting long by HiThere · · Score: 1

      I don't know whether the process is AI or not. Do you need to train it? If so, it's probably AI. This doesn't mean it isn't a rather specialized AI, of course. But the higher the bar gets, the more general the AI is going to need to be. Adding in proper eye blinking requires a significantly more powerful AI than just matching head positions....of course, you could, in principle, do the entire thing with image processing matches, but the required dataset would expand at least greater than linearly, and probably exponentially. Then there's matching skin textures under various light conditions. Then...

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
  17. How to identify a deepfake? by PhunkySchtuff · · Score: 1

    At the moment it's pretty easy to identify a deepfake video. The AI's don't "know" that people regularly blink, they see this as a glitch in the data set.
    If you look at pretty much any deepfake video, no matter how realistic it is, the person who's been deepfaked will never blink.

    --
    Specialist Mac support for creative pros, Melbourne
    1. Re:How to identify a deepfake? by Anonymous Coward · · Score: 0

      I make mine blink like a DOS prompt.

  18. The thing about adversarial networks by goombah99 · · Score: 1

    If you build a better mousetrap an adversarial system builds a better mouse in response. Thus I wonder how they can make a detector that continues to detect. The whole idea of a GAN is to generate things that defeat the detector. So what's the strategy to make s detector the generator can't beat?

    --
    Some drink at the fountain of knowledge. Others just gargle.
    1. Re:The thing about adversarial networks by Anonymous Coward · · Score: 0

      Well, as generators and detectors get better and better we will eventually reach a point where the generator is pixel perfect and can't be distinguished from a non-edited video.

      The strategy to use for detecting deepfakes is to compare it to other videos. If you can find the videos it was based on then the generator would have to significantly modify the video to a point where it might be easier to just render the entire thing from scratch instead of merging things together.
      This only works when the video is publicly available.

      Either way the deepfake technology only matters when it comes to fake celebrity porn.
      For political reasons the gains are large enough to make it worth doing the editing manually like it was done to add Ariana Richards face to her stunt double in Jurassic Park.
      You can also set up a studio and film material with lookalike actors and angles and lighting that fits the faces you are going to add on it.
      Deepfakes is only a cost-saver. Anything larger than your local city politics and the money involved is large enough to do without it.
      As for detecting if a video is fake or not you can just look at it and see if it supports your message or not.
      If it aligns with your interests it's true. If it is a problem then it is fake.

      I'm just waiting for the first case where a cop uses deepfakes to fake video evidence against someone that he "knows is criminal but can't find any proof"

  19. adversarial by thygate · · Score: 1

    .. adversarial networks, work together ..

    what ?

  20. Caching by Anonymous Coward · · Score: 0

    I thought the title said Caching Deep Fakes. Oh well.

  21. Re:Billionaire, Author, Tv Star, President.... by Anonymous Coward · · Score: 0

    Traitor? Explain.

  22. Re:Billionaire, Author, Tv Star, President.... by Anonymous Coward · · Score: 0

    What treason?