Slashdot Mirror

← Back to Stories (view on slashdot.org)

Algorithm Clones Facial Expressions And Pastes Them Onto Other Faces

Posted by samzenpus on from the I-want-that-smile dept.

KentuckyFC writes Various researchers have attempted to paste an expression from one face on to another but so far with mixed results. Problems arise because these algorithms measure the way a face distorts when it changes from a neutral expression to the one of interest. They then attempt to reproduce the same distortion on another face. That's fine if the two faces have similar features. But when the faces differ in structure, as most do, this kind of distortion looks unnatural. Now a Chinese team has solved the problem with an algorithm that divides a face into different regions for the mouth, eyes, nose, etc and measures the distortion in each area separately. It then distorts the target face in these specific regions while ensuring the overall proportions remain realistic. At the same time, it decides what muscle groups must have been used to create these distortions and calculates how this would change the topology of the target face with wrinkles, dimples and so on. It then adds the appropriate shadows to make the expression realistic. The result is a way to clone an expression and paste it onto another entirely different face. The algorithm opens the way to a new generation of communication techniques in which avatars can represent the expressions as well as the voices of humans. The film industry could also benefit from an easy way to paste the expressions of actors on to the cartoon characters they voice.

20 of 31 comments (clear)

  1. compression approach for video chats by mspring · · Score: 2, Insightful

    This approach should be used to reduce the necessary bandwidth in video conferencing situations!

    1. Re:compression approach for video chats by Khyber · · Score: 1

      You probably generate and transmit more procedural data versus a video stream, sorry to break it to you. You also introduce lag, which many of us don't like (and this would suck balls for those using sign language to communicate via video chat.)

      --
      Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
    2. Re:compression approach for video chats by Jeremiah+Cornelius · · Score: 1

      Looks like he's pinching a loaf. Xerox that look.

      --
      "Flyin' in just a sweet place,
      Never been known to fail..."
    3. Re:compression approach for video chats by Knuckles · · Score: 3, Informative

      Matrox HeadCast in 2001.

      --
      "When I first heard Daydream Nation it quite frankly scared the living shit out of me." -- Matthew Stearns
    4. Re:compression approach for video chats by jblues · · Score: 1

      This approach was used in Vernor Vinge's 'A Fire Upon the Deep' published in 1992. In the book, bandwidth between faster than light network nodes is very limited, so an initial capture of the user is sent followed by facial expression data. . . its a pretty awesome book actually.

      --
      If it acquires resources on instantiation like a duck, then its a shared_ptr<Duck>
  2. Mathematical definition for by invictusvoyd · · Score: 1

    The term " Permanently shitfaced"

    nice

  3. Mona Lisa by AveryRegier · · Score: 1

    now anyone can see their Mona Lisa smile!

  4. already there by slashmydots · · Score: 1

    I do this all the time on Facebook to my friends' pic uploads. Except it's more like: http://manbabies.com/

  5. Hollywood wants this! by arielCo · · Score: 1

    Paging Keanu Reeves, Kristen Stewart...

    --
    This post contains no rudeness or derision of any kind. All arguments are friendly. Terms and exclusions may apply.
  6. "They said it was a break-through!" by westlake · · Score: 1

    The film industry could also benefit from an easy way to paste the expressions of actors on to the cartoon characters they voice.

    The Adventures of Mr. Incredible (with Commentary)

    The animator of a feature film begins with a study of an actor's vocal performance, facial expression and mannerisms but the end product will be shaped his own imagination and interpretation of the character

    One reason for Disney's recent string of hits is that the studio casts its nets widely and avoids using overly familiar celebrity voices in predictable ways.

  7. 3d animation already uses this technique. by tbttfox · · Score: 5, Interesting

    As a professional working on the technical side of 3d animation, I can tell you with 100% certainty that this is nothing new. This technique of scaling the individual parts of the face to match proportions, then applying a muscle simulation is used everywhere. It's how high fidelity, realistic facial mocap retargeting works.
    Another good technique is to sample FACS (Facial Action Coding System) style muscle activation data from the source head and just add the individual deformations together on the target head. The proportioning is already taken care of by the definition of the FACS shapes relative to each head.

    1. Re:3d animation already uses this technique. by tbttfox · · Score: 1

      Unfortunately this level of the technical process usually isn't documented in public, so there won't be any articles. And it's waaaay over the head of the DVD extras, but some glimpses do sometimes make it through. For instance, I was able to find this bonus clip from The Incredible Hulk showing a pass of the mocap retargeting: https://www.youtube.com/watch?...
      Also at 2:05 you can see the mucle set that was used to drive the deformations of the face.

    2. Re:3d animation already uses this technique. by tbttfox · · Score: 1

      Sorry to double post, but a screencap of an old coworker just showed up in my facebook feed saying "That's my shoulder". The more interesting (relevant) part was the screen showing the facial mocap process where a human is driving the shapes for an alien: https://www.youtube.com/watch?...

    3. Re:3d animation already uses this technique. by tbttfox · · Score: 1

      Unfortunately, without reading the actual scholarly paper, I can't really say. I can only comment on what made it into the linked article. However, from the pictures, it looks like they're manually defining a mesh on top of each face to perform the image warping. So they would be defining their own mocap dots to track the face image.
      You don't have to use the dots, it just makes things easier to track. Check out http://facewaretech.com/ , they do a pretty good job without the dots.

  8. Seen before? by wlj · · Score: 2

    I believe I saw a demo of this technology back at Uncertainty '99 when Matthew Brand presented a paper titled "Patter discovery via entropy minimization" (TR-98-21 from MERL ("A Mitsubishi Electronics Research Laboratory")). The demo was a video of an infant who started to lecture the audience on the technique. I was quite impressed. I recently found a copy of the paper via google.

  9. Look ma, no teeth? by aNonnyMouseCowered · · Score: 1

    None of the example pictures in the article and the paper @Arxiv show target/source person flashing a toothpaste smile. Does this mean their algorithm's only good for Mona Lisa? Maybe that's their secret (limitation)?

    1. Re:Look ma, no teeth? by gnupun · · Score: 1

      The target person's smile looks completely computer-generated and not natural at all. They are very far from a toothpaste smile. Real actors in movies need not fear Hollywood replacing them with no-acting-talent pretty faces.

  10. Emoji by penguinoid · · Score: 1

    :-)

    --
    Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
  11. Re:Mathematical definition for by BFDTweets · · Score: 1

    Could not have said this better myself, except for maybe today "Permanently Pi Shiftaced"

  12. Coming soon to GMOD? by wikthemighty · · Score: 1

    GMOD FacePoser in addition to GMOD w/Kinect would be a blast!

    --
    "There are people who do not love their fellow human being, and I _hate_ people like that!" - Tom Lehrer