Slashdot Mirror


GPU Accelerated Realtime Skin Smoothing Algorithms Make Actors Look Perfect

dryriver writes: A recent Guardian article about the need for actors and celebrities -- male and female -- to look their best in a high-definition media world ended on the note that several low-profile Los Angeles VFX outfits specialize in "beautifying actors" in movies, TV shows and video ads. They reportedly use a software named "Beauty Box," resulting in films and other motion content that are -- for lack of a better term -- "motion Photoshopped." After some investigating, it turns out that "Beauty Box" is a sophisticated CUDA and OpenGL accelerated skin-smoothing plugin for many popular video production software that not only smooths even terribly rough or wrinkly looking skin effectively, but also suppresses skin spots, blemishes, scars, acne or freckles in realtime, or near realtime, using the video processing capabilities of modern GPUs.

The product's short demo reel is here with a few examples. Everybody knows about photoshopped celebrities in an Instagram world, and in the print magazine world that came long before it, but far fewer people seem to realize that the near-perfect actor, celebrity, or model skin you see in high-budget productions is often the result of "digital makeup" -- if you were to stand next to the person being filmed in real life, you'd see far more ordinary or aged skin from the near-perfection that is visible on the big screen or little screen. The fact that the algorithms are realtime capable also means that they may already be being used for live television broadcasts without anyone noticing, particularly in HD and 4K resolution broadcasts. The question, as was the case with photoshopped magazine fashion models 25 years ago, is whether the technology creates an unrealistic expectation of having to have "perfectly smooth looking" skin to look attractive, particularly in people who are past their teenage years.

71 of 138 comments (clear)

  1. Comment removed by account_deleted · · Score: 4, Interesting

    Comment removed based on user account deletion

  2. Will it work on neckbeard geeks? by Anonymous Coward · · Score: 1

    If not, I don't see much need for it.

  3. Won't 4k kill this for a while? by wonkavader · · Score: 3, Funny

    "The question...is whether the technology creates an unrealistic expectation of having"
    blurry, featureless skin.

    1. Re: Won't 4k kill this for a while? by Anonymous Coward · · Score: 3, Funny

      So we've gone full circle, we have invented SD video

    2. Re: Won't 4k kill this for a while? by Anonymous Coward · · Score: 2, Interesting

      I saw a discussion with a few professional directors at Broadcast Video Expo a few years ago and the interviewer goes 'so are you excited for 4k'? and one answered that as most ladies wouldn't want to be seen in that sharp detail they would have to use smoothing filters (like real glass ones) so you didn't have 4k and so was a bit pointless. they were all more interested in HDR.

      on personal opinion.. whens the last time you watched a film and said 'that was quite good, but it would be better if it was sharper'. my complaints about movies dont involve being sharper (unless the focus puller fucked up)

    3. Re: Won't 4k kill this for a while? by Anonymous Coward · · Score: 1

      Ayup, I told people decades ago, that going to higher resolutions is only useful for nature movies. Sitcoms and anything with real actors in it, require lower resolution since nobody wants to see age spots, veins and wrinkles.

    4. Re:Won't 4k kill this for a while? by AmiMoJo · · Score: 2

      That's the default setting on most camera apps for phones. All they did was accelerate it to realtime.

      Unfortunately photoshopping is basically the default now. No skill required, the app magically makes you look better without even bothering to ask if that's what you want, before offering to upload to Facebook or Instagram.

      Either we have to build the tech into every mirror or start educating people how harmful it is pushing for natural beauty and imperfections to be the standard. Just as some magazines and fashion brands were doing that we got hit with the apps.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    5. Re:Won't 4k kill this for a while? by mentil · · Score: 2

      Not going outside without a box over my head with a QR code on it that leads to an app that lets you see a digitally-enhanced version of my face. Take that, reality!

      --
      Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
  4. Re:Probably not. by iggymanz · · Score: 4, Interesting

    It looks silly to me, especially knowing the actors' real life appearance, to see a bunch of living mannequins in a movie. Even the piled on makeup opera or play performers wear looks ridiculous, they're like clowns and I must laugh.

  5. Re:Technology Curve? by DontBeAMoran · · Score: 2

    Americans have been using 60-years-old as teenagers in sitcoms for decades, so it's not really going to change anything.

    --
    #DeleteFacebook
  6. The problem... by mholve · · Score: 4, Interesting

    This technique has been used for still photography for years now, and has the same issue under video (even more so)... There's a very fine line between skin smoothing - and making it look like plastic. When it's overdone, even slightly, you do notice it, and it doesn't look right.

    1. Re:The problem... by geekmux · · Score: 5, Insightful

      This technique has been used for still photography for years now, and has the same issue under video (even more so)... There's a very fine line between skin smoothing - and making it look like plastic. When it's overdone, even slightly, you do notice it, and it doesn't look right.

      Let's be realistic here. Any amount of digital manipulation isn't going to change the shock value once you meet the actor or actress in meatspace. They're all going to look different/look older/look like shit by comparison.

      And the fact that photo manipulation is still alive and well after decades tells you the popularity in which video manipulation will be welcomed with open arms. The world is full of lies, and people love it.

    2. Re:The problem... by ShanghaiBill · · Score: 2

      The next step should be to virtualize the concept of "celebrities" and just completely regenerate the appearance of the actors. So a filmmaker could hire anyone for the role, based solely on their ability to perform, and then switch their face and appearance in post-production to match the "star" of the film who doesn't actually exist.

      There are virtual celebrities like Hatsune Miku, but in the future they will look like real people, only perfect.

  7. Fucking stupid by reanjr · · Score: 5, Interesting

    If the resolution of movies is so high that they are essentially applying a blur filter everywher, then maybe we should back off the higher resolutions. What's the point of high res capture if you're just going to muddy the image in post processing?

    1. Re:Fucking stupid by Roger+W+Moore · · Score: 3, Insightful

      ...or maybe they should stop using blur filters because it is fine to see actors and actresses with a few wrinkles. However, I don't think this is a blur filter because otherwise, it will look terrible on a big movie screen where it is intended to be used.

    2. Re:Fucking stupid by Solandri · · Score: 4, Informative

      What's the point of high res capture if you're just going to muddy the image in post processing?

      In photography, the basic algorithm is to use a high-pass filter to identify high-contrast regions (places with lots of edges. Turn that into a mask and invert it. Apply a blurring algorithm to the picture, using the mask to exempt the high-contrast parts of the pic (places with lots of edges.) The final result is a pic where low-contrast surfaces (like skin and sky and blurred backgrounds) are blurred, but high-contrast edges which contain detail are untouched.

      I never liked doing it (I prefer realistic photos), but it was sometimes necessary to counter a sharpening algorithm run across the entire picture, and prevent skin blemishes from being exaggerated. Also, I found that if I first showed my female friends their photo after running it through the above algorithm, they were much less likely to threaten to kill me if I ever released that photo to the public. Who says flattery never got you anywhere?

    3. Re:Fucking stupid by ShanghaiBill · · Score: 1

      If the resolution of movies is so high that they are essentially applying a blur filter everywher

      It is not a blur filter, and it is not being applied "everywhere". Only on skin. More specifically, only on the skin of the good people. The villains will still have bad complexions.

    4. Re:Fucking stupid by phantomfive · · Score: 1

      What tool(s) do you use to do that?

      --
      "First they came for the slanderers and i said nothing."
    5. Re:Fucking stupid by ClickOnThis · · Score: 1

      If the resolution of movies is so high that they are essentially applying a blur filter everywher, then maybe we should back off the higher resolutions. What's the point of high res capture if you're just going to muddy the image in post processing?

      I suppose one point is that there are other things in a scene besides the actors' faces. Higher resolution increases the immersive impact of the entire image.

      And let's not dismiss the possibility that the art will adapt to the medium as it often has in the past (not the other way around). Maybe higher-res images of actors faces, blemishes and all, will become less of an issue if filmmakers, actors, makeup artists use high-res to give us better visual storytelling.

      --
      If it weren't for deadlines, nothing would be late.
    6. Re:Fucking stupid by careysub · · Score: 1

      One thing to recall is that the "close up" is an extremely artificial way of viewing someone. It does not exist in real life. Do you ever get inches from someone's face while they are talking (without getting smacked good and proper)?

      So doing a close-up in 8K with an actual human is going to have very unfortunate consequences unless enhancement if used, and we are used to that and expect it - we call it "make-up". But at sufficiently high resolution even physical make-up probably can't cut it.

      --
      Starships were meant to fly, Hands up and touch the sky - Nicky Minaj
  8. Was this driven, like so many other technologies, by mark_reh · · Score: 1

    by the porn industry?

  9. I can see by unrealmp3 · · Score: 1

    ..the porn industry having a field day with this.

  10. Re:Probably not. by Roger+W+Moore · · Score: 2

    I have no problem doing this for fiction.

    So, actually, you really don't have a problem with using it in political ads.

  11. Blur filter with motion tracking by Anonymous Coward · · Score: 1

    Just what we all want, 4k footage with blurry faces.

  12. Nothing new here by petes_PoV · · Score: 4, Insightful

    whether the technology creates an unrealistic expectation of having to have "perfectly smooth looking" skin

    As the article alludes, this is nothing more than a digital form of makeup. And that has been used for decades for TV and films - and even longer in the real world.

    There really aren't any additional issues here. If is simply a modern version of an old, old, tradition.

    --
    politicians are like babies' nappies: they should both be changed regularly and for the same reasons
    1. Re:Nothing new here by dryriver · · Score: 2

      Except that makeup is bound by "physical universe" constraints. Like photons bouncing off it and into the camera sensor or photochemical film. Image processing algorithms on the other hand are NOT bound by these constraints. Image processing can create "skin looks" that are practically impossible to achieve with even the best makeup. You are looking a celebs that have a "heavenly glow" or "saintly glow" to them anywhere, in all locations, under all lighting conditions. You are essentially creating "digitally enhanced Gods and Goddesses" out of slightly-better-than-ordinary looking mortals who are famous for doing whatever. People who look supernaturally amazing, always, in any situation. And of course this tech is eventually going to deliver "super beautiful skin looks" when it goes 3D-aware as well. You'll be able to play God with the specularity, reflectance, sub-surface scattering and other aspects of actor or model skin, resulting in skin that looks so "alluring" that no man or woman in real life can match it with ANY skin product. Thus lifting "celebrities" very high above "ordinary mortals" in terms of how they look. Today you can go out on the street and see real-life people who look just as good as the best looking celebs or better. Tomorrow, you may find that digitally enhanced millionaire or billionaire celebrities look so amazing that everyone else just looks tired or ugly by comparison. Is that a good thing?

      --
      Why did the chicken cross the road? Because Elon Musk put an AI chip in its head.
    2. Re:Nothing new here by argStyopa · · Score: 2

      It only matters until they replace the troublesome, expensive, neurotic human actors anyway.

      --
      -Styopa
    3. Re:Nothing new here by AmiMoJo · · Score: 1

      At CES this year they demoed a special applicator that uses something like an inkjet print head and a high speed camera to apply makeup to just the pores and imperfections on your face, smoothing your skin out and making you look younger in a fairly natural way. It didn't look like heavy foundation, it was the kind of thing that you would need an expensive makeup artist for, but easy to apply in a minute or two yourself.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    4. Re:Nothing new here by rnturn · · Score: 1

      So the makeup box from The Fifth Element is coming to a store near you?

      --
      CUR ALLOC 20195.....5804M
    5. Re:Nothing new here by mentil · · Score: 1

      Just turn on the beautification filter for your AR headset, and bam, everyone now looks like that.

      --
      Corruption is convincing someone that the selfless ideal is the same as their selfish ideal.
  13. Should be banned for beauty commercials by ET3D · · Score: 3, Insightful

    Anyone trying to sell a beauty product and using this should be sued.

    Other than this, yes, I think it's a problem that people can't be accepted as they are. Of course we already use makeup, etc., but if anything we should move in the direction of accepting how people look instead of trying to stylise them further.

    1. Re:Should be banned for beauty commercials by Anonymous Coward · · Score: 2, Informative

      If you watch the demo reel, you will see that it is being used with regards to beauty products. That is a level of deceit that should not be tolerated. If the industry cannot effectively prohibit this, then the government needs to step in.

      If you can achieve the result in the real world with your cosmetics, go ahead and do so. If the result shown requires digital effects, then it should be banned.

    2. Re:Should be banned for beauty commercials by TheGoodNamesWereGone · · Score: 1

      That's some strange reasoning, when makeup and beauty products themselves are an attempt to deceive...

    3. Re:Should be banned for beauty commercials by Ambvai · · Score: 1

      As I recall, there is legislation specifically in place for cosmetics that basically says 'The people featured in this ad got the results from actually using cosmetics.' There was one manufacturer a few years ago who got in trouble over accusations of this because people couldn't believe that she looked that good without digital enhancement. In the end, they got through it by documenting the process... and it really was just a good makeup job.

  14. Sigh by nospam007 · · Score: 2

    That's why I prefer English movies and series.
    There at least you see real people, not a bunch of 27 year old model types with fake hair, fake teeth, fake boobs and fake noses.
    That just makes it look unreal and this will make it still worse, if that is possible.

    1. Re:Sigh by helpfulcorn · · Score: 1

      Indeed, while it's been apparent for quite some time, it really slapped me in the face when I saw the first episodes of The Walking Dead and also Revolution, the people were so unrealistically good looking and well put together for living in a post-apocalyptic nightmare world with presumably limitations on health (especially in Revolution where they don't even have antibiotics or asthma medicine) that people being beyond perfect was so obnoxious that I couldn't even finish Revolution beyond the first episode, that and because the show sucked. I also stopped watching The Walking Dead in the middle of Season 2 because it got boring for me, but it wasn't nearly as bad as Revolution when it came to "sure the world as we know it is over and the conveniences of modern life are gone, but I look better than even people possibly could when society was intact, not to mention I'm immaculately groomed, and my clothes are stylish and clearly tailored."

      The remake of V was the same way, just: what the hell is going on? I realise there's a certain expectation on TV, especially American TV, but if I want to see anyone remotely "normal" looking I have to watch stuff the BBC and Channel 4 churns out.

  15. Re:Probably not. by ShanghaiBill · · Score: 1

    Using it in things like political ads, well, that's a different question.

    Why do you care? Do you base your vote on zit counts?

  16. Re:Probably not. by iggymanz · · Score: 1

    because young people don't have acne, moles, freckles, warts, scars etc.?

  17. Comment removed by account_deleted · · Score: 1

    Comment removed based on user account deletion

  18. Congradulations - Humanity Morns by BrendaEM · · Score: 1

    I great computing achievement that raises the bar of illusion higher over a person's head....requiring them to use even more sophisticated solutions to try to reach it.

    --
    https://www.youtube.com/c/BrendaEM
  19. Take my money! by paiute · · Score: 2

    I can't wait to get a pair of Google Glasses with this software installed so I don't have to loot at your all's ugly faces anymore. Is there a setting of 11 on this menu?

    --
    If Slashdot were chemistry it would look like this:Cadaverine
  20. How does it compare to traditional makeup? by quantaman · · Score: 1

    Actors and actresses already spend huge amounts of time in makeup, how does this method compare to that?

    Does it compound the effect of makeup so some actress in her late 40s can still get a lead role? Or is it one or the other, meaning that a lot of makeup artists will be out of work once this tech matures.

    --
    I stole this Sig
  21. Still does not give us good scripts and actors by Anonymous Coward · · Score: 1

    If blemishless looks carried that kind of importance, the movies featuring supermodels wouldn't flop the way they do. This is the action movie fallacy repeated: you can fill a movie with explosions and facial closeups all you want, if you don't actually cater for a convincing plot and convincing actors, you'll bore your audience.

    This is not what "here is looking at you, kid" was about.

  22. Comment removed by account_deleted · · Score: 1

    Comment removed based on user account deletion

  23. Re:Probably not. by ceoyoyo · · Score: 2

    Apparently the best predictor of electoral success is candidate appearance (not money spent). So many people DO base their vote on zit counts. Although beautifying might not be the best idea for political ads. The predictor for US presidential elections was the candidate who looked "most presidential," which might not be the same as "youngest" or "prettiest."

  24. When making a video of your product by quonset · · Score: 5, Informative

    Make sure you include snippets of other videos which bounce around so much you can't tell what you're looking at.

    That way you can show off your product to its full effect.

  25. Re:Probably not. by phantomfive · · Score: 1

    The real question I have is, when can I get it as an Instagram filter?

    Better yet, can I get it installed on my date's glasses? Come to think of it, maybe it'd be better to have installed on my glasses.

    --
    "First they came for the slanderers and i said nothing."
  26. Comment removed by account_deleted · · Score: 1

    Comment removed based on user account deletion

  27. Re:Probably not. by iggymanz · · Score: 1

    lolz no don't offer or suggest to show pics to random 54 year old guys on the internet

    your face might not be as perfect as you imagine, there is nonzero chance people with that username such as yours might have inflated opinion of themselves.

    no big pores, no wrinkles...heh......suuuuure

  28. Saving 4K pr0n 1 gpu cycle at a time by frazamatazzle · · Score: 1

    Does it really have any other purpose? Really?

  29. Re: Probably not. by Albanach · · Score: 1

    Honestly it looks like watching 480p upscaled.

  30. higer res tv/movies do this by p51d007 · · Score: 1

    If everything was stuck in low def, wouldn't be a "need", but with 1080p, 2k, 4k, 8k, higher definitions make it much easier to "see the flaws". Just get rid of the overhyped "actors" and use CGI. Movies would be easier to make, without the "drama" of the overpaid hollyWEIRD crybabies.

  31. lemme get this straight by Ol+Olsoc · · Score: 1
    We spend all of this time and effort and money for ultra high definition, then apply blurring so people look more fuzzy?

    Sounds legit smart.

    --
    The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
  32. Realtime by g01d4 · · Score: 1

    I think realtime is what's significant here given the frame rates and resolution. That's an impressive amount of computation whereas historically crude analog (blur or special filters on the lens) techniques had to be used if makeup wasn't available or enough.

  33. Re:Probably not. by iggymanz · · Score: 1

    I made no claims about beauty or lack dummy-tude or talent.

    Please read with more comprehension before posting. Better yet, holler up the stairs to your mommy to order you a pizza so you feel better and not so grumpy.

  34. Definition of beauty by nehumanuscrede · · Score: 1

    As if the definition of Beauty is not manipulated enough by magazines and digital retouching as it is. :|

    Whole generations of folks striving to reach the levels of Beauty their idols achieve, clueless to the fact it is all digitally and / or makeup enhanced bullshit.

  35. Re:Probably not. by SeaFox · · Score: 1

    Using it in things like political ads, well, that's a different question.

    If you're choosing a leader based on their physical attractiveness, you deserve to be disenfranchised.

  36. Shitty results. by SeaFox · · Score: 1

    The product's short demo reel is here with a few examples.

    If this supposed to be a commercial for the software it's a failure. The side-by-side results show an obvious muddy texture on the skin on the after side. If you're entire production is really soft focus, I guess it might blend in, but otherwise it looks like you hired a moron for a video compressionist.

  37. Uncanny valley by jrumney · · Score: 1

    Is this step one in getting us used to accepting the visual uncanny valley (autotune has already done this for audio)? When it gets to the point where we cannot tell the difference between a real actor and an animation on the screen, will Hollywood go the same way as Detroit?

  38. Webcams will get more freaky by TJHook3r · · Score: 1

    Maybe we can all maintain a facade by webcam? Perfect Tinder date, perfect salesperson or camgirl.... as long as we never meet in the flesh. Think my cyberpunk reading list is coming true!

  39. This is all just the ... by rnturn · · Score: 1

    ... CGI-ification of human actors so that, in the future when movie studio profits need a boost, they can get rid of the human actors altogether---by then we won't recognize the difference. There's no Actor's Guild pay scale like no pay at all.

    --
    CUR ALLOC 20195.....5804M
  40. Re:Probably not. by ShanghaiBill · · Score: 1

    Bubblehead? What the everloving fuck are you talking about. She is freaking awesome.

    I agree that she is awesome, and I agree with her on some issues.

    But when she talks, listen to the details. Write down the numbers. Then go to Google and do a reality check. Many of the "facts" she rattles off are off by a factor of ten or more. And some of her policy proposals are what destroyed Venezuela. She is reality challenged.

  41. Re:Probably not. by arglebargle_xiv · · Score: 1

    We've actually been doing this for decades, it's called a soft-focus filter, they've been used to make people's faces look better since at least the 1920s. Before that, you just left the lens slightly unfocused. So this is just an expensive GPU-accelerated (and thus hype-attracting) way of doing what you could do with a bit of glass for over a hundred years.

  42. Photo restoration by Waccoon · · Score: 1

    About 10 years ago I was doing photo restoration in a small camera store. I had a large stock of noise/texture patterns to put back into photographs after retouching them so they didn't look too perfect. Even modern retouching tools meant for still photographs don't do that well, so I'm not surprised these "AI" video tools do such a bad job.

  43. Just what our culture needs by saboosh · · Score: 1

    Oh great, this will make the girls on this planet even more insecure and boys have even more unrealistic expectations... just what we need :(.. Why cant we all just be ok with who we are exactly as we are? I love technology but this is just adding to a cultural phenomenon that is already pretty sick.

  44. Yet another technology built on proprietary GPUs by themusicgod1 · · Score: 1

    nt

    --
    GENERATION 26: The first time you see this, copy it into your sig on any forum and add 1 to the generation.
  45. Re:Probably not. by baegucb · · Score: 1

    old school technology: beer goggles, is likely cheaper

  46. Unrealism... by bradley13 · · Score: 1

    It strikes me that this is related to the pushback against high frame rate movies. Apparently a substantial segment of people like 24fps, because of the "cinematic feel" it gives you. It's an artifact left from a previous age, but some people want to preserve the lack of realism it portrays.

    Hiding actors freckles and blemishes is perhaps part of the same mentality. I remember when HDTV first came out: I was struck by the fact that I could actually see the wrinkles and freckles and little blemishes on people's faces. Apparently this is unwanted - too much realism?

    It's also probably sheer egotism on the part of Hollywood & Co.. An actor or actress is no one special - they just have a talent for playing pretend in a convincing way. Yet they feel entitled to tell us all about their views on politics and society, and how we should live our lives. They think they know better. They certainly don't want us to see that they are just people with zits and wrinkles, like everyone else - that might dent their mystique.

    --
    Enjoy life! This is not a dress rehearsal.
  47. not just faces by sad_ · · Score: 1

    it doesn't only work for faces, but the video also shows hands, complete bodies and...

    paper.

    in the youtube video one person is holding a wrinkled paper and in the 'processed' stream the paper is very much less wrinkled.

    i think the whole thing is stupid, if you need to modify people so much, might as well just use complete digital models instead.

    --
    On a long enough timeline, the survival rate for everyone drops to zero.
  48. Can I run it in reverse? by gshegosh · · Score: 1

    I want an algorithm that'll take these artificial faces and make them look real again.

  49. Re:Probably not. by ConceptJunkie · · Score: 1

    I would agree with "bubblehead". It's like they elected a college sophomore who took one poli-sci class and thinks she knows everything. Yes, I'm aware of her alleged degrees.

    --
    You are in a maze of twisty little passages, all alike.