Slashdot Mirror

← Back to Stories (view on slashdot.org)

Algorithm Clones Facial Expressions And Pastes Them Onto Other Faces

Posted by samzenpus on from the I-want-that-smile dept.

KentuckyFC writes Various researchers have attempted to paste an expression from one face on to another but so far with mixed results. Problems arise because these algorithms measure the way a face distorts when it changes from a neutral expression to the one of interest. They then attempt to reproduce the same distortion on another face. That's fine if the two faces have similar features. But when the faces differ in structure, as most do, this kind of distortion looks unnatural. Now a Chinese team has solved the problem with an algorithm that divides a face into different regions for the mouth, eyes, nose, etc and measures the distortion in each area separately. It then distorts the target face in these specific regions while ensuring the overall proportions remain realistic. At the same time, it decides what muscle groups must have been used to create these distortions and calculates how this would change the topology of the target face with wrinkles, dimples and so on. It then adds the appropriate shadows to make the expression realistic. The result is a way to clone an expression and paste it onto another entirely different face. The algorithm opens the way to a new generation of communication techniques in which avatars can represent the expressions as well as the voices of humans. The film industry could also benefit from an easy way to paste the expressions of actors on to the cartoon characters they voice.

31 comments

  1. compression approach for video chats by mspring · · Score: 2, Insightful

    This approach should be used to reduce the necessary bandwidth in video conferencing situations!

    1. Re:compression approach for video chats by Khyber · · Score: 1

      You probably generate and transmit more procedural data versus a video stream, sorry to break it to you. You also introduce lag, which many of us don't like (and this would suck balls for those using sign language to communicate via video chat.)

      --
      Still waiting on Serviscope_minor to wake up to fucking reality and realize that Jessica Price isn't going to fuck him.
    2. Re:compression approach for video chats by Jeremiah+Cornelius · · Score: 1

      Looks like he's pinching a loaf. Xerox that look.

      --
      "Flyin' in just a sweet place,
      Never been known to fail..."
    3. Re:compression approach for video chats by Knuckles · · Score: 3, Informative

      Matrox HeadCast in 2001.

      --
      "When I first heard Daydream Nation it quite frankly scared the living shit out of me." -- Matthew Stearns
    4. Re:compression approach for video chats by jblues · · Score: 1

      This approach was used in Vernor Vinge's 'A Fire Upon the Deep' published in 1992. In the book, bandwidth between faster than light network nodes is very limited, so an initial capture of the user is sent followed by facial expression data. . . its a pretty awesome book actually.

      --
      If it acquires resources on instantiation like a duck, then its a shared_ptr<Duck>
  2. Yes, I've seen that by Anonymous Coward · · Score: 0

    That was their software in Running Man, wasn't it? Ca. 1987, right?

    1. Re: Yes, I've seen that by Anonymous Coward · · Score: 0

      Yeah, but now Hollywood can support actors with less talent

  3. Mathematical definition for by invictusvoyd · · Score: 1

    The term " Permanently shitfaced"

    nice

  4. Mona Lisa by AveryRegier · · Score: 1

    now anyone can see their Mona Lisa smile!

  5. already there by slashmydots · · Score: 1

    I do this all the time on Facebook to my friends' pic uploads. Except it's more like: http://manbabies.com/

  6. Gonna have fun pasting by Anonymous Coward · · Score: 0

    congressmans' faces in 'compromising' places.

  7. 4chan for the win by Anonymous Coward · · Score: 0

    see this all the time on the seedy boards of 4chan

  8. meme generator 2.0 by Anonymous Coward · · Score: 0

    Mona Lisa doing the cool face in 2015.

  9. Easy for the Chinese... by Maow · · Score: 0

    Easy for the Chinese researchers, since... Chinese Faces All Look Alike.

    Bada-bum-tish.

    Thank you, try the waitress, tip the salmon, something-something "all week".

    (And for the humour-impaired, I certainly don't think "they all look alike", I live with a Hong Konger... Just playing with an old trope.)

  10. Hollywood wants this! by arielCo · · Score: 1

    Paging Keanu Reeves, Kristen Stewart...

    --
    This post contains no rudeness or derision of any kind. All arguments are friendly. Terms and exclusions may apply.
    1. Re:Hollywood wants this! by Anonymous Coward · · Score: 0

      Anyone who thinks that Keanu Reeves can't make more than one facial expression have not seen Bill & Ted's Excellent Adventure.

  11. Calling Eric! by Anonymous Coward · · Score: 0

    Eric Conveys An Emotion needs to make a comeback to take advantage of this.

  12. "They said it was a break-through!" by westlake · · Score: 1

    The film industry could also benefit from an easy way to paste the expressions of actors on to the cartoon characters they voice.

    The Adventures of Mr. Incredible (with Commentary)

    The animator of a feature film begins with a study of an actor's vocal performance, facial expression and mannerisms but the end product will be shaped his own imagination and interpretation of the character

    One reason for Disney's recent string of hits is that the studio casts its nets widely and avoids using overly familiar celebrity voices in predictable ways.

  13. 3d animation already uses this technique. by tbttfox · · Score: 5, Interesting

    As a professional working on the technical side of 3d animation, I can tell you with 100% certainty that this is nothing new. This technique of scaling the individual parts of the face to match proportions, then applying a muscle simulation is used everywhere. It's how high fidelity, realistic facial mocap retargeting works.
    Another good technique is to sample FACS (Facial Action Coding System) style muscle activation data from the source head and just add the individual deformations together on the target head. The proportioning is already taken care of by the definition of the FACS shapes relative to each head.

    1. Re:3d animation already uses this technique. by Anonymous Coward · · Score: 0

      Had you given some pointers to sites or articles would have enhanced your opinion. Otherwise, it is just a statement. But thanks anyway.

    2. Re:3d animation already uses this technique. by tbttfox · · Score: 1

      Unfortunately this level of the technical process usually isn't documented in public, so there won't be any articles. And it's waaaay over the head of the DVD extras, but some glimpses do sometimes make it through. For instance, I was able to find this bonus clip from The Incredible Hulk showing a pass of the mocap retargeting: https://www.youtube.com/watch?...
      Also at 2:05 you can see the mucle set that was used to drive the deformations of the face.

    3. Re:3d animation already uses this technique. by tbttfox · · Score: 1

      Sorry to double post, but a screencap of an old coworker just showed up in my facebook feed saying "That's my shoulder". The more interesting (relevant) part was the screen showing the facial mocap process where a human is driving the shapes for an alien: https://www.youtube.com/watch?...

    4. Re:3d animation already uses this technique. by Anonymous Coward · · Score: 0

      But do you do it without the motion capture 'dots' on the actor's face? I assume that's the difference here.

    5. Re:3d animation already uses this technique. by tbttfox · · Score: 1

      Unfortunately, without reading the actual scholarly paper, I can't really say. I can only comment on what made it into the linked article. However, from the pictures, it looks like they're manually defining a mesh on top of each face to perform the image warping. So they would be defining their own mocap dots to track the face image.
      You don't have to use the dots, it just makes things easier to track. Check out http://facewaretech.com/ , they do a pretty good job without the dots.

  14. Seen before? by wlj · · Score: 2

    I believe I saw a demo of this technology back at Uncertainty '99 when Matthew Brand presented a paper titled "Patter discovery via entropy minimization" (TR-98-21 from MERL ("A Mitsubishi Electronics Research Laboratory")). The demo was a video of an infant who started to lecture the audience on the technique. I was quite impressed. I recently found a copy of the paper via google.

  15. One interesting question is... by Anonymous Coward · · Score: 0

    ... the "translation" of culture related expressions.

    Facial expressions, just like hand gestures, vary from place to place.

    I suspect Westerners have richer mouth expressions than Asians. In the example provided, some expressions on the painted guy look uncommon while they seem very valid on the Asian dude.

    But even Westerners are not the same: just pick an international movie from a distant country to compare with the mannerisms you know.

  16. Look ma, no teeth? by aNonnyMouseCowered · · Score: 1

    None of the example pictures in the article and the paper @Arxiv show target/source person flashing a toothpaste smile. Does this mean their algorithm's only good for Mona Lisa? Maybe that's their secret (limitation)?

    1. Re:Look ma, no teeth? by gnupun · · Score: 1

      The target person's smile looks completely computer-generated and not natural at all. They are very far from a toothpaste smile. Real actors in movies need not fear Hollywood replacing them with no-acting-talent pretty faces.

  17. Emoji by penguinoid · · Score: 1

    :-)

    --
    Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
  18. Re:Mathematical definition for by BFDTweets · · Score: 1

    Could not have said this better myself, except for maybe today "Permanently Pi Shiftaced"

  19. Coming soon to GMOD? by wikthemighty · · Score: 1

    GMOD FacePoser in addition to GMOD w/Kinect would be a blast!

    --
    "There are people who do not love their fellow human being, and I _hate_ people like that!" - Tom Lehrer