Slashdot Mirror

← Back to Stories (view on slashdot.org)

Getting Computers To Recognize Facial Expressions

Posted by samzenpus on from the getting-to-know-you-better dept.

Zothecula writes "Binghamton University computer scientist Lijun Yin thinks that using a computer should be a comfortable and intuitive experience, like talking to a friend. As anyone who has ever yelled 'Why did you go and do that?' at their PC or Mac will know, however, using a computer is currently sometimes more like talking to an overly-literal government bureaucrat who just doesn't get you. Thanks to Yin's work with things like emotion recognition, however, that might be on its way to becoming a thing of the past."

56 comments

  1. Face recognition by Pricetx · · Score: 2

    I think if my computer had face recognition it would grow scared of me over time! (either that or just start arguing back at me, that would be almost as bad as using vista :o).

    1. Re:Face recognition by AndyAndyAndyAndy · · Score: 1

      I'm glad my computer can't see how unexcited I am for this technology. Poor little guy. Doesn't have any idea.

      --
      It's always confirmation bias!
    2. Re:Face recognition by Anonymous Coward · · Score: 2, Funny

      Just what I need when I get home from work, 2 women asking me about my feelings. "I"M FINE - JUST COMPILE MUTHER F$*#KER!"

  2. already seen the result of this: by Krau+Ming · · Score: 2

    Terminator shoots man in leg.
    John Connor: what the hell did you do that for???
    Terminator: because you told me to.

  3. A computer is a tool. by gblackwo · · Score: 3, Insightful

    I'm not sure I want my tools to respond differently to me depending on what mood I'm in.

    1. Re:A computer is a tool. by chemicaldave · · Score: 1

      Not to mention it would be a task to program the computer to respond appropriately. My PC (or rather the software running on it) knowing that I'm pissed at it, won't help it fix the problems. What we really need is a reporting tool for the developers. i.e., "74% of users who reported this bug problem had a look of murderous rage on their face."

    2. Re:A computer is a tool. by Thing+1 · · Score: 1

      I'm not sure I want my tools to respond differently to me depending on what mood I'm in.

      Actually, I just found out about f.lux yesterday and installed it on my work laptop this morning, then forgot about it during the day. Then along comes 6 pm and the laptop starts getting easier to read! It's not my mood, it's a different external reading (available sunlight, calculated by the current time, and it also might have some sort of IP-geolocation, to determine latitude), but I found on just the first day of using it that it is fairly cool. I'm now putting it on all my computers.

      That said, I very much agree with you: it is frustrating using Outlook to type part of someone's email address, especially when that someone and another email are addresses that you regularly communicate with a lot, but sometimes more and sometimes less, so that those two fucking addresses keep swapping places in the drop-down hint that appears... (Did you know you can arrow-key to one of the entries and then hit the Delete key to stop it from ever appearing again? That might be useful for some who find themselves in this circumstance, if for instance one of the two emails is a "dead" one.)

      Anyway: yeah, a tool that is not deterministic (i.e., changes with my mood) seems like it would make my mood worse, not improve it.

      --
      I feel fantastic, and I'm still alive.
  4. Why did it do that? by KublaiKhan · · Score: 1

    If you have to ask why it did that, then you either need to learn more about how computers work or pay attention to what you're clicking.

    --
    In Xanadu did Kubla Khan
    A stately pleasure dome decree
    1. Re:Why did it do that? by by+(1706743) · · Score: 1

      Sort of -- a segfault in legitimate software is not unheard of. Is that really the users fault?

      Which brings up an interesting point (others -- care to weight in?): if/when you yell "at your computer," are you yelling at your computer, or a particular piece of software / hardware? I curse at things all the time -- shoddy wifi drivers / grub misbehaving (that's a fun one...) / databases / etc., but I'm very clear that I'm not yelling at my computer per se (or I may curse at a stuck key, lousy ethernet cable, etc.). I see this as very different than yelling at your computer -- kinda a "don't kill the messenger" thing.

      Anyone else?

    2. Re:Why did it do that? by Hatta · · Score: 2

      Which brings up an interesting point (others -- care to weight in?): if/when you yell "at your computer," are you yelling at your computer, or a particular piece of software / hardware?

      Neither. You're yelling at the programmers.

      --
      Give me Classic Slashdot or give me death!
    3. Re:Why did it do that? by An+ominous+Cow+art · · Score: 1

      Neither. You're yelling at the programmers.

      That's what I do. I have heaped a lot of abuse on Microsoft employees over the last 23 or so years...

  5. Don't do it!!! by Rolgar · · Score: 1

    When they take over, we won't be able to mock them without getting zapped for insolence.

    1. Re:Don't do it!!! by ackthpt · · Score: 1

      When they take over, we won't be able to mock them without getting zapped for insolence.

      I'm not so concerned about facial expressions as I am about Sarcasm Detection.

      This would probably be closely followed by the Dry Wit Riposte and (shudder) the Device Which Thanks You And Detects Its Degree Of Welcomeness.

      --

      A feeling of having made the same mistake before: Deja Foobar
  6. As a learning tool by MarkRose · · Score: 2

    As an aspie, and someone who has partial face blindness and trouble recognizing facial expression of emotions, I'd find such research interesting as a learning tool. For instance, I can't tell the difference between fear and anger. Nor did I recognize the surprised face in the article. I just don't "see" it like other people do. If we can more precisely quantify the expression of emotion, it would certainly help me learn read and differentiate emotions when necessary.

    --
    Be relentless!
    1. Re:As a learning tool by blair1q · · Score: 1

      Don't count yourself down over that. I can't tell that's supposed to be "surprise", either. Looks more like a combination of bewildered and amused.

      But, a small experiment. What do these mean to you?

      :-)
      :-(
      :-O
      :-/

    2. Re:As a learning tool by Sonny+Yatsen · · Score: 1

      Well, even better than that, you might have an augmented reality device that might alert you to facial expressions and emotions in the future.

      --
      My postings are informational and does not constitute legal advice. Act on it at your risk.
    3. Re:As a learning tool by Anonymous Coward · · Score: 0

      The first one is happy, the second one is sad, the third just got a red-hot poker shoved up his <censored>, and I can't make heads or tails of the fourth.

    4. Re:As a learning tool by MarkRose · · Score: 1

      Happiness, sadness, shock or surprise, and worry or concern or displeasure.

      Interestingly, I don't have much trouble with Western smilies. Eastern smilies are a mystery. It could be because I look at people's mouths and not their eyes like most people, and Asians (and thus Eastern smilies) express emotion with the eyes.

      --
      Be relentless!
    5. Re:As a learning tool by williamhb · · Score: 1

      Follow Rana El Kaliouby's work -- that's pretty much what her PhD and subsequent research has been on.

  7. Who's being overly-literal? by Anonymous Coward · · Score: 0

    "Using a computer should be a comfortable and intuitive experience, like talking to a friend"
    Sure, but that doesn't mean that the computer has to have the exact same user interface as the friend

  8. HAL Sees It Coming... by Sponge+Bath · · Score: 1

    Be sure to wipe the monitor clean when you are done, Dave.

  9. microsoft bob? by Anonymous Coward · · Score: 1

    I *like* that my computers are literal. I already find logic and algorithms "comfortable and intuitive". I have a much easier time talking to a computer in absolute terms than I do trying to decipher what's behind the casual lies most people pass off as conversation. For hackers, this is an evolutionary step in the wrong direction. For normal people? Maybe this is what Microsoft Bob was supposed to be.

    1. Re:microsoft bob? by KublaiKhan · · Score: 1

      Wasn't Microsoft Bob the source of "clippy"?

      Perhaps if I make a rage-face at the computer, it'll run away and delete its system files...

      --
      In Xanadu did Kubla Khan
      A stately pleasure dome decree
  10. Um, yeah by smooth+wombat · · Score: 1

    sometimes more like talking to an overly-literal government bureaucrat who just doesn't get you.

    Or like talking to a programmer who doesn't understand you don't care if the interface to an OS is cool or edgy, you just want to be able to get to things without having to click a dozen different links or burrow down some menu until you near the center of the Earth. *cough*Windows7*cough*

    Or like talking to a web designer who doesn't understand that you don't care if the buttons fade in and out or they can create a transparent entry box using javascript, you just want to get to the information you need without being assaulted by the latest and greatest web design. *cough*VerizonBillPay*cough*

    --
    We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
    1. Re:Um, yeah by MozeeToby · · Score: 1

      http://www.cnn.com/2006/HEALTH/conditions/04/03/engineered.organs/index.html

      I don't understand this complaint. I don't exactly spend hours and hours of time administering on my home computer, but everything I've ever needed to do was done with Windows-Key -> Type what I want -> Select from a very short list. Are you sure you're using the search features to their fullest? That is what Windows 7 is built off of at heart, as a general rule you shouldn't ever be browsing randomly through the administration tools to find what you're looking for.

    2. Re:Um, yeah by smooth+wombat · · Score: 1

      I don't want to search. That is the whole point.

      An OS is not a search engine. It is supposed to provide you with the means to get at what you want without having to search.

      Further, the new design prevents one from seeing everything at one time. I want to see every program installed on the system. Not one list for programs and one list for updates. Everything at one shot.

      I shouldn't have to hunt around for how to turn off the fade-in effects by going to a menu system completely unrelated to display settings.

      When I tell it to revert to the classic look (i.e. no shitty link clicking) I want EVERYTHING to revert. Not a piece here and there.

      It's shitty interfaces like this which have made people lazy and even more stupid when it comes to computers. Between allowing people to open attachments directly from emails to making everything a link, Microsoft has borked any semblance of coherency.

      I'm willing to give praise where it is due (the ability to change the shutdown button to Restart as the default should resolve several issues around here), but when one creates an abomination akin to punch the monkey, the programmers deserve to be punched.

      --
      We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
    3. Re:Um, yeah by FiloEleven · · Score: 1

      What you want is impossible unless you are on a very minimal system. There are some legitimate gripes in there, like the Windows7 effects nonsense, but there's no way to present EVERYTHING on a system at once. The best you can do is to give someone a meaningful layout that makes navigation easy and the opportunity to rearrange it so the things they use often are a small step away, which usually means a desktop shortcut. I personally hate a cluttered desktop, but that just means I rarely use it; I see no need to start a crusade (or is jihad now the preferred nomenclature?).

      LiveSearch from Microsoft and Spotlight from Apple don't take away from that at all, they give you another tool to get you where you're going if you want to make use of it. In fact if you don't use it then you still end up searching, only by directory trawling instead of keyword guessing. Either way, if you already know what you are after then there's not really a search involved; at that point you are following a series of steps to get at what you want--and it's hard not to see search-as-you-type as a clear winner here against everything save perhaps a desktop icon.

      The same is true with opening attachments from emails--you can take the extra step to save to disk and open manually if you want to, but why? Allowing a user to open an attachment that he may bother to look at only once and then trash immediately makes sense. Allowing a user to instead save it for later without opening now also makes sense. Removing one of those choices really doesn't.

      We should be able to be lazy while using computers. The whole point of a tool is to let us be lazier where we can so that we can do more of whatever the real task at hand is. A construction worker can be proud of his skill with hammer and saw, but his hammering and sawing is at the service of building something else and not an end in itself. I think a lot of people are just uncomfortable that our magic staves have been slowly but surely transforming into common hammers.

      --
      Your brain is not a computer.
    4. Re:Um, yeah by smooth+wombat · · Score: 1

      Except there is a huge flaw in Outlook which, if people continually open attachments rather than than save them, causes them not to be able to open any attachment. Those temp files never get deleted. That's a flaw which has not been corrected since day one.

      Today I had to configure a new user on a W7 machine. It took me over half an hour to make it look and act as close as possible to XP. There is no way configuring a profile should take that long. Ten minutes tops.

      And to change the default location of documents one no longer can go to the My Documents folder (which doesn't exist), instead, you have to go to the Start button, select the user, change the location and remove the secondary location. WTF??!!!!!

      And what the hell is up with Libraries?

      And yes, you can see everything as far as programs are concerned. It's been done since Day 1 of Windows. It's called Add/Remove Programs. One could see everything that was installed from there. No longer. Now you have to make a choice of what you want to see, but not both at the same time.

      I could go on but the bottom line is Windows 7 sucks. Big time. This push to have everything in a cloud and available everywhere is creating huge security holes, combined with the inability for people to do their jobs because there is no logical or semi-logical way to know where things are located or how to change your system. I will go to Linux before I have my own machine with Windows 7 on it.

      --
      We will bankrupt ourselves in the vain search for absolute security. -- Dwight D. Eisenhower
  11. People yell at their Macs in anger? by genghisjahn · · Score: 1

    I thought it was all butterflys, rainbows and one button mice. Another example of a poorly edited summary.

    --
    Sorry about the mess.
  12. Stupid Question by Haedrian · · Score: 1

    "'Why did you go and do that?"

    Because you asked it to. Isn't it obvious? Or because whoever programmed that function made it do that.

    I would rather my computer always does the same thing regardless of whether I'm smiling at the screen or growling angrily. If you can't tell a computer what you want done, then that's your problem.

  13. I do not want a computer that cringes. by blair1q · · Score: 1

    I expect my computer to do as I say, not run and hide when I guess wrong about how some feeb programmed it.

    The feeb, on the other hand...

  14. Great Idea. by 140Mandak262Jamuna · · Score: 1

    Binghamton University computer scientist Lijun Yin thinks that using a computer should be a comfortable and intuitive experience, like talking to a friend.

    Yup, it turned out to be so great last time two pals, Hal and Dave got talking.

    --
    sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
    1. Re:Great Idea. by Sonny+Yatsen · · Score: 1

      You mean this?

      HAL-9000: What is going to happen?
      Dave: Something wonderful.
      HAL-9000: I'm afraid.
      Dave: Don't be. We'll be together.
      HAL-9000: Where will we be?
      Dave: Where I am now.

      --
      My postings are informational and does not constitute legal advice. Act on it at your risk.
    2. Re:Great Idea. by rleibman · · Score: 1

      Don't worry, that was ten years ago, computers have gotten much more powerful since 2001

  15. New invvoations by Anonymous Coward · · Score: 0

    New laptop technology also introduced

  16. Forget Facial Recognition by Anonymous Coward · · Score: 0

    Husbands can't even recognize their wives facial expressions after being married for 20 years. What the hell kind of windmill are these facial recognition coders tilting at? Just put accelerometers in the monitor and poll the current volume level on the microphones, and when you detect the user screaming and yelling and slapping the monitor around, you know that it's time to pop up Clippy. Er, okay, maybe it's better just to not bother with this stuff at all.

    1. Re:Forget Facial Recognition by Kosi · · Score: 1

      Husbands can't even recognize their wives facial expressions after being married for 20 years.

      That is probably a consequence of many occurrences like this:

      Husband, recognizing sad/angry/whatever expression: "Honey, you look sad/angry/whatever, what's bothering you?"
      Wife: "Nothing, it's all fine."

  17. I went to SUNY Binghamton and... by Anonymous Coward · · Score: 0

    and Lijun Yin barely had any facial expressions. Dude spoke in a soft monotone throughout his entire lectures. I almost slept as much in his class as I did in my cinema history class.

  18. How hard can this be? by Krishnoid · · Score: 1
    • :-) - I'm happy
    • :-( - I'm sad
    • :-| - meh

    If you want deeper nuances, you can always refer to this as well.

  19. This Will Backfire by bistromath007 · · Score: 1

    When the machines can recognize our emotions, the first one they'll develop is schadenfreude.

  20. let's better by Anonymous Coward · · Score: 0

    install it on government bureaucrats' brains

    1. Re:let's better by Kosi · · Score: 1

      They already know enough about playing with people's emotions. How do you think they manipulated the masses to elect them?

  21. What We Really Need is... by 0xG · · Score: 1

    A computer that understands this.

    --
    A pox on web designers who feel that window.innerWidth == screen.availWidth
  22. I don't get it. by wcrowe · · Score: 3, Interesting

    I don't understand what my computer is supposed to DO once it has determined what my emotion is. The only everyday application I can see for this is marketing. I am sure marketers would like to register your reaction when you see something. I don't see how that helps ME. I mean, if the computer sees that I'm angry or surprised or whatever, what is it supposed to DO?

    --
    Proverbs 21:19
    1. Re:I don't get it. by Anonymous Coward · · Score: 0

      That's really a question for UX designers, not programmers.

      One obvious and infamous example is Clippy. If Excel could differentiate between concentration and bewilderment, it wouldn't pop up Clippy when I'm pondering the 2010 North Dakota Sales. You'd want a fairly good eyetracker, though, to check what the user is scanning. It does matter whether he goes looking for the North Dakota 2009 figures or the formula bar.

      An more subtle thing might be to adjust timings and thresholds. An angry user will not be as precise with mouse clicks. A timid user might be new and afraid to break the shiny new&expensive computer, and that might be a good reason to show the "Welcome to your new computer" tour.

      And I'm not even going to touch upon the use of this for games.

    2. Re:I don't get it. by MadMaverick9 · · Score: 1

      I mean, if the computer sees that I'm angry or surprised or whatever, what is it supposed to DO?

      it should look at my right hand and its fingers ... if the computer can see four ... then ... well, the computer should just ... :-)

    3. Re:I don't get it. by vidnet · · Score: 1

      It can put Clippy into Eliza mode. "Looks like you find this document upsetting. Would you like me to set the font to Comic Sans?"

      It can auto-like/dislike Youtube videos! Just put on a long cat video playlist, and you won't have to lift a mouse finger for the rest of the day.

      It can be combined with people recognition and integrated in photo apps, to allow queries like "Find one where my damn ex-wife doesn't have that awful grin on her face".

      It could be used by the Windows crash dialog to automatically assign priorities to bugs, based on how pissed off the user is about it!

      The possibilities are endless!

    4. Re:I don't get it. by ThunderBird89 · · Score: 1

      If my computer sees me angry, it should bloody well correct whatever error happened in the first place, or run...

      --
      Hyperbole: I use it liberally!
    5. Re:I don't get it. by mr_gorkajuice · · Score: 1

      What would be infinitely more useful than a "auto-detect mood" feature would be a "you're wrong!"-button when you disagree with whatever result your PC came up with. Once that's developed, it might be useful to autotrigger it when the user is aggrevated.

    6. Re:I don't get it. by wondafucka · · Score: 1

      I don't understand what my computer is supposed to DO once it has determined what my emotion is. The only everyday application I can see for this is marketing. I am sure marketers would like to register your reaction when you see something. I don't see how that helps ME. I mean, if the computer sees that I'm angry or surprised or whatever, what is it supposed to DO?

      If I were a marketing researcher for an OS company, I would log whenever extreme reactions take place and use that to prioritize my bugs / feature improvements. You can't fix all the bugs, but you can fix the ones that cause the most people the most ire.

      --
      postmodernsideshow.com
  23. Drunken e-mailing by Anonymous Coward · · Score: 0

    This would be really useful if it could detect when you are drunk and only allow you to save drafts of e-mails rather than send them...

  24. NO THANKS! by Anonymous Coward · · Score: 0

    I have enough trouble when my wife misinterprets my facial expressions. Since I spend more time with my computer, that would be pure hell.

  25. Merge with Clippy by TheInternetGuy · · Score: 1

    I can see it before me now. Typing an email to the mother in law; I'm so glad that you are coming to visit us.. Clippy: -No you aren't , starting auto correct I'm so unbelievably depressed that you are ..... -Well, thanks Clippy.

    --
    If my comment didn't sound as good in your head as it did in mine, then I guess we all know who's to blame
  26. 3d cameras should make it easy by The+Car · · Score: 1

    Reminds me of a facial expression recognition project I worked on about five years back using plain old webcams and Neven Vision software. Under decent lighting conditions we were able to detect a variety of expressions.

  27. Not a completely original idea by Anonymous Coward · · Score: 0

    I attended a Microsoft Research conference with my school about 4 years ago and they had a demonstration of a webcam that accurately calculated someone's emotion from a [slightly exaggerated] facial expression they were pulling. Microsoft Research had even attached to a basic instant messenger that displayed emoticons based on the facial expressions you made.

  28. porn? by Anonymous Coward · · Score: 0

    porn will be totally awkward