Software Brings Eye Contact To Video Chat, With a Little Help From Kinect

← Back to Stories (view on slashdot.org)

Software Brings Eye Contact To Video Chat, With a Little Help From Kinect

Posted by Soulskill on Tuesday September 3, 2013 @06:55AM from the and-makes-you-look-really-funny-when-it-bugs-out dept.

Zothecula writes "Skype has been around for ten years now. Once a science fiction dream, the video calling service has 300 million users making two billion minutes of video calls a day. One problem: most of them can't look each other in the eye. Claudia Kuster, a doctoral student at the Computer Graphics Laboratory ETH Zurich, and her team are developing a way to bring eye contact to Skype and similar video services with software that alters the caller's on-screen image to give the illusion that they're looking straight at the camera."

69 of 111 comments (clear)

Min score:

Reason:

Sort:

there's always looking right at the camera by themushroom · 2013-09-03 06:57 · Score: 1

Which is hardly a natural act, so you should position your camera just above your screen if possible because that's where you're looking, at your screen. :)

--
Laughter is the Spackle of the Soul.
1. Re:there's always looking right at the camera by Atzanteol · 2013-09-03 07:19 · Score: 3, Insightful
  
  Then you're not looking the other person in the eyes. Hence no "eye-contact."
  
  --
  "Ignorance more frequently begets confidence than does knowledge"
  
  - Charles Darwin
2. Re:there's always looking right at the camera by Anonymous Coward · 2013-09-03 07:25 · Score: 1
  
  If you're looking at the camera, then you're not seeing the other person, which defeats the purpose of a video chat.
3. Re:there's always looking right at the camera by theIsovist · 2013-09-03 07:38 · Score: 1
  
  No where in the article does it say they are looking directly at the camera. And if you look at the photos they are showing, you should automatically realize that the viewer is looking away from the camera (presumably at the screen displaying the other person's face), and the image is adjusted to give the illusion that the viewer is looking into the camera (thus achieving digital eye contact). Come on folks, this is Slashdot. We used to be smart. Let's bring that back.
4. Re:there's always looking right at the camera by asylumx · 2013-09-03 07:51 · Score: 4, Funny
  
  If both people are only looking at the camera, why bother even displaying a picture at all? Nobody is looking at it... Except the NSA perhaps.
5. Re:there's always looking right at the camera by Culture20 · 2013-09-03 08:00 · Score: 1
  
  You should position a small camera directly in front of the screen between the eyes of the person on screen.
6. Re:there's always looking right at the camera by Lumpy · 2013-09-03 08:29 · Score: 1
  
  You mean like how Every Acer, Samsung, Toshiba, macintosh, and dell has the camera on their laptops and all in one pc's?
  
  --
  Do not look at laser with remaining good eye.
7. Re:there's always looking right at the camera by cusco · 2013-09-03 08:35 · Score: 1
  
  Back in 1999 my boss hung up the phone after several minutes of yelling at her kid. She sighed and said, "Remember Brian, when we were kids and they told us that in the year 2000 everyone would have phones where they could see the other person?"
  
  I agreed that yes, I did remember that. I said that with MSN Messenger and some of the other new software we were actually getting pretty close.
  
  She said, "I don't so much want to see the other person, as I want to be able to reach through and SLAP them!" The inventor of that will be be a gizillionaire overnight.
  
  --
  "Think about how stupid the average person is. Now, realise that half of them are dumber than that." - George Carlin
8. Re:there's always looking right at the camera by chihowa · 2013-09-03 08:41 · Score: 1
  
  They're all looking up, though, which would indicate that the camera is below the screen. Every setup I've seen puts the camera atop the screen, which makes more sense with regards to eye contact as the contact's eyes will be closer to the camera (and filming from slightly above is more flattering than filming from slightly below).
  I've done a little video chatting and it never looks as dramatically awkward as the pictures they show. You can tell that the other person is looking at the screen and not the camera, but it isn't as exaggerated as it appears in their setup. If you drag the window as close as you can to the camera, the situation gets much better.
  
  --
  If you want a vision of the future, imagine a youtube comments section scrolling - forever.
9. Re:there's always looking right at the camera by rahvin112 · 2013-09-03 09:10 · Score: 1
  
  Come on folks, this is Slashdot. We used to be smart.
  You must be using a different slashdot than me because that's not my experience at all.
10. Re:there's always looking right at the camera by theIsovist · 2013-09-03 09:33 · Score: 1
  
  Yeah... and I didn't help myself for not fully reading their comment. Read the article and not the comments, I should be banned, right?
11. Re:there's always looking right at the camera by ArsonSmith · 2013-09-03 09:49 · Score: 1
  
  Why isn't this done for video conferencing systems. would it be difficult to have a tiny camera embedded in the center of the screen? A small spot on the screen shouldn't be entirely too distracting. Would work great for this type of setup: http://www.zdnet.com/atlassian-builds-portal-for-video-chat-1339327884/
  
  --
  Paying taxes to buy civilization is like paying a hooker to buy love.
12. Re:there's always looking right at the camera by LinuxIsGarbage · 2013-09-03 14:38 · Score: 1
  
  I don't know if this should be rated funny or insightful.
13. Re:there's always looking right at the camera by SpectreBlofeld · 2013-09-04 02:24 · Score: 1
  
  "I see in Slashdot the strongest and smartest men who've ever lived. I see all this potential, and I see squandering. God damn it, an entire generation pumping gas, waiting tables; slaves with white collars. Advertising has us chasing cars and clothes, working jobs we hate so we can buy shit we don't need. We're the middle children of history, man. No purpose or place. We have no Great War. No Great Depression. Our Great War's a spiritual war... our Great Depression is our lives. We've all been raised on television to believe that one day we'd all be venture capitalists, and head developers, and IT project managers. But we won't. And we're slowly learning that fact. And we're very, very pissed off."
Beta Preview by Anonymous Coward · 2013-09-03 06:59 · Score: 1, Funny

http://i.ebayimg.com/00/s/MTYwMFgxNTM1/z/0DkAAMXQHeBSD6I4/$(KGrHqIOKpMFIJv(N8KvBSD6I3QPTw~~60_35.JPG?set_id=880000500F
Aversion by CanHasDIY · 2013-09-03 07:06 · Score: 1

I have an aversion to maintaining eye contact with people I don't completely trust, you insensitive clod!
Even if it is fake.

--
An enigma, wrapped in a riddle, shrouded in bacon and cheese
1. Re:Aversion by dmbasso · 2013-09-03 07:13 · Score: 1
  
  You just have to not activate the feature yourself. On the other hand, your peers will be relieved to have a more natural communication. I had a professor that while speaking with anyone never makes eye contact, looking to a corner of the ceiling... I cannot describe how disturbing that was.
  
  --
  `echo $[0x853204FA81]|tr 0-9 ionbsdeaml`@gmail.com
2. Re:Aversion by hedwards · 2013-09-03 07:29 · Score: 2
  
  I disagree, not looking at people in the eye during chat isn't weird at all. It's the way that video chat is done.
  By permitting the computer to change your eyes, you solve one problem, while taking away the meaning with which your eyes communicate with people. Suddenly, you can't just roll your eyes without telling the computer that you want to roll your eyes as the computer isn't going to know that it should break eye contact.
  If you happen to live in a culture where eye contact is maintained through out, that might be fine, but for those of us that are used to maintaining eye contact for short periods and breaking it as a part of effective communication, this is just going to make that more complicated.
3. Re:Aversion by CanHasDIY · 2013-09-03 07:52 · Score: 1
  
  I had a professor that while speaking with anyone never makes eye contact, looking to a corner of the ceiling... I cannot describe how disturbing that was.
  Having grown up very close to a deaf person, I've developed the tendency to stare at people's mouths when they speak.
  You want disturbing? Try learning to lip read.
  
  --
  An enigma, wrapped in a riddle, shrouded in bacon and cheese
4. Re:Aversion by kermidge · 2013-09-03 08:32 · Score: 1
  
  I haven't video called much and it took a little bit of getting used to but then was OK. At appropriate times I'll look directly into the camera and the other person sees that. And vice versa. Works, and only a little awkward. This new software is nifty but unsettling in its own way. Until I use it I'll reserve judgement.
5. Re:Aversion by hedwards · 2013-09-03 08:36 · Score: 1
  
  I've never had a problem with it, I just look at the screen as one is intended and focus on communicating. It's really freaky to be on video chat and focusing on the camera, it sounds like you were sitting too close to a camera with a screen that's too large. With my 11.6" screen, the difference between looking at the camera and looking at the screen is minimal.
  If you've got a larger screen then you're likely going to have to sit further away in order for the effect to disappear.
  Still, it's a damned sight better than having my computer spying on me via Kinnect.
6. Re:Aversion by kermidge · 2013-09-03 11:43 · Score: 1
  
  Didn't say I had a problem with it, only that it was a bit of getting used to. Camera is in bezel of laptop's 17" screen; sometimes I'll be leaning back in my office chair, other times leaning forward to type some thing, once with my nose to the screen, "What are you doing?" "Seeing if you got more wrinkles that I do." At a comfortable viewing distance there's still around 5 degrees of separation eyes-to-camera, so that's the "getting used to." Also, with several people, we'll each look directly at the camera from time to time, for emphasis, solidarity, whatever. I guess it's a human thing and we're all a bit different in how we do things.
  Unless the Kinect is phoning home, that part wouldn't bother me. Using Skype? Now that bothers me, so I use alternatives.
7. Re:Aversion by hedwards · 2013-09-03 13:14 · Score: 1
  
  This sounds a lot like people prior to video cameras becoming ubiquitous. If you've ever seen a Beatles film you'll know what I mean, it's all awkward because they're paying a ton of attention to the camera rather than just ignoring it and going about their business.
  The video conferencing equipment is the same way, you're not supposed to be changing your behavior to suit the device, you're supposed to be looking on screen. I'm not sure how that would look weird, because it's never looked weird on any device I've used.
  And yeah, Kinnect is great if you don't mind MS or random crackers invading your privacy, at least my webcam has a light on, and the laptop is supposed to be closed when not in use.
8. Re:Aversion by kermidge · 2013-09-03 15:45 · Score: 1
  
  I'm not sure about the "supposed to" so much; so far people and I just do what seems natural to us, but I think I know what you mean.
  If one could know that the Kinect is locked down then I'm all for it just 'cuz it's a nifty toy at the least. What these folks have done with it is way cool whether one wishes to use it or no.
  My laptop cam has a light also but I'm not sophisticated enough to rest totally assured that the software I use reports correctly as to whether it and the microphone are on or off - offhand, I don't even recall if the light is wired directly in circuit with the cam so it can't be spoofed. Drat, like I really needed something more to look into right now.
Sometimes the easy way is the better way by 3D-nut · 2013-09-03 07:06 · Score: 4, Interesting

Why not put a half-silvered mirror (plate beamsplitter) at a 45 degree angle to the screen, a piece of black velvet beyond the beamsplitter as a light trap, and point the camera so it sees your face reflected in the glass? Like a teleprompter.
1. Re:Sometimes the easy way is the better way by shadowrat · 2013-09-03 07:09 · Score: 1
  
  Why not put a half-silvered mirror (plate beamsplitter) at a 45 degree angle to the screen, a piece of black velvet beyond the beamsplitter as a light trap, and point the camera so it sees your face reflected in the glass? Like a teleprompter.
  Not a bad idea really. Software could remove the ghostly reflected face probably, and it would be a far better use of software than manufacturing where your eye is looking. That just sounds like it would look creepy.
2. Re:Sometimes the easy way is the better way by a_nonamiss · 2013-09-03 07:14 · Score: 2
  
  Terrific idea, but you'd have a pretty hard time getting that setup into a laptop bezel. Video-conferencing didn't really seem to catch on en masse until the cameras came built into every piece of hardware sold. As a dedicated device, I could see this idea working really well, but I don't think people are willing to sacrifice price and portability just to be able to see someone they're talking to. Of course, there could be exceptions, such as when people are doing high-value business transactions, where eye contact can make or break a deal.
  
  --
  -Arthur
  Cave ne ante ullas catapultas ambules
3. Re:Sometimes the easy way is the better way by Russ1642 · 2013-09-03 07:18 · Score: 1
  
  Difficult to do on a phone or laptop but you'd make fortune.
4. Re:Sometimes the easy way is the better way by Lumpy · 2013-09-03 08:32 · Score: 1
  
  Large size? I have one for my camera the size of an iphone.... which is because it uses an iphone for the teleprompter and the headphones remote as a text scroll control.
  Works great even in bright sunlight and are common for use by many TV stations in the field now.
  
  --
  Do not look at laser with remaining good eye.
5. Re:Sometimes the easy way is the better way by Lumpy · 2013-09-03 08:34 · Score: 1
  
  It doesnt make or break a deal. I sell high end AV systems (They cost more than your house and your parents house) that use $15,000 or more Pro Videoconference systems. and they dont maintain eye contact. the camera is either mounted above or below the screen. and when you are using a 80" LCD the camera is WAY up there.
  
  --
  Do not look at laser with remaining good eye.
6. Re:Sometimes the easy way is the better way by Lumpy · 2013-09-03 08:46 · Score: 1
  
  on an android phone? yes. but why do that. Use the iphone as a display and pipe the video from the laptop to the phone, then the video from the camera to the laptop. two wires and all is done. but has limited use. nobody outside of podcast recording care about eye line and eye contact.
  
  --
  Do not look at laser with remaining good eye.
7. Re:Sometimes the easy way is the better way by Platinumrat · 2013-09-03 09:32 · Score: 2
  
  Why not put 4 cameras in each corner and use software to combine the image in such a way that the senders image is in the centre?
Creepy by A10Mechanic · 2013-09-03 07:16 · Score: 1

Hey! Eyes up here!
1. Re:Creepy by Lumpy · 2013-09-03 08:37 · Score: 1
  
  Bourne Supremecy cam effects? That would rock.... well actually easy, just install the camera on a paint shaker.
  
  --
  Do not look at laser with remaining good eye.
Don't mess with the eyes by Russ1642 · 2013-09-03 07:24 · Score: 5, Insightful

They're putting out software that changes what your eyes look like so that it looks like you're not looking up a little bit. This will not work. It will make people's eyes look wrong and creepy. We are perfectly attuned to looking at eyes and anything that's a bit off will get noticed immediately. Start by fixing people's teeth or something but not the eyes.
1. Re:Don't mess with the eyes by Anubis+IV · 2013-09-03 07:36 · Score: 1
  
  Apparently they're actually moving the whole head to look like it's aiming at the camera, not just making the eyes look like they're aiming at the camera.
  In most of the images, it looks reasonable, but in a few it just looks a bit off somehow.
2. Re:Don't mess with the eyes by Russ1642 · 2013-09-03 07:53 · Score: 1
  
  Whatever. Tilting the face, changing angles and trying to make the eyes look like they're pointed a different direction will just look wrong. It sounds like wonderful technology but everyone's going to try it for five minutes and then turn it off because it looks creepy.
3. Re:Don't mess with the eyes by Anubis+IV · 2013-09-03 08:01 · Score: 1
  
  I'm completely with you. I was just pointing out what some people might see as a big difference between what you were talking about and what was actually happening. As I said, a couple of their sample images looked a bit off already, so I imagine it would look creepy as hell in practice.
4. Re:Don't mess with the eyes by naoursla · 2013-09-03 08:24 · Score: 1
  
  The Kinect gets 3D information. They can take the polygons mapped with the video image and apply small rotation from the picture of the other person onthe screen to the camera. You'll get some artifacts where you are exposing areas that the camera can't see, but maybe those can be filled in with surrounding pixels so you don't notice so much.
5. Re:Don't mess with the eyes by Ichijo · 2013-09-03 08:25 · Score: 1
  
  It will make people's eyes look wrong and creepy.
  
  Or maybe they'll look like Terry Gilliam's cutout animations.
  
  --
  Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
6. Re:Don't mess with the eyes by SpeedBump0619 · 2013-09-03 08:38 · Score: 1
  
  http://www.youtube.com/watch?v=7QrnwoO1-8A&list=TLtms-Tz7YPQk
  Given this demo from two years ago using a single hacked Kinect I have to believe that the technology is only going to improve. As long as the camera isn't occluded the 3D point data can be used to map sections of the 2D image onto a mesh created from the 3D point cloud. Then the camera can be virtually re-positioned and the scene rendered. Most of this is pretty easy using commodity hardware rendering engines.
7. Re:Don't mess with the eyes by dinfinity · 2013-09-03 11:34 · Score: 1
  
  In most of the images, it looks reasonable, but in a few it just looks a bit off somehow.
  I'd call the latter a bit of an understatement: http://www.gizmag.com/skype-eye-contact/28843/pictures#3
  The left guy in the right picture looks like he's changed race or even species.
  Considering that these are undoubtedly the cherrypicked best results and that the example images are low resolution, I'd say that the technology needs some more work. Having said that, there is nothing wrong with the idea in itself.
Re:NO WAR FOR BIG OIL by Anonymous Coward · 2013-09-03 07:33 · Score: 3, Insightful

Calling Obama "the left" is a joke.
Signed,
a Canadian.
300 million users by korbulon · 2013-09-03 07:35 · Score: 1

And still no decent alternative. Well how about it, science?!
That isn't eye contact by bkr1_2k · 2013-09-03 07:38 · Score: 1

Altering the image doesn't provide eye contact. Eye contact is a palpable connection between two people, not just me staring into the eyes of an image. Unless it communicates the "connection" (for lack of a better word) created when you actually look someone in the eye, it's just a gimmick.

--
"Growing old is inevitable; growing up is optional."
1. Re:That isn't eye contact by Zontar+The+Mindless · 2013-09-03 17:33 · Score: 1
  
  I use Skype all the time, and even pay for SkypeOut in spite of the money now going to one of my least favourite corporations on the planet.
  That being said, this whole "eye contact" business sounds like a solution in search of a problem, brought on by someone's petty obsession with something that we other 600 million users don't seem to be bothered by.
  
  --
  Il n'y a pas de Planet B.
2. Re:That isn't eye contact by bkr1_2k · 2013-09-06 04:37 · Score: 1
  
  Exactly. People who use video to talk regularly (like me) simply don't care. Like I said, a gimmick.
  
  --
  "Growing old is inevitable; growing up is optional."
Looking someone straight in the eye by BetaDays · 2013-09-03 07:39 · Score: 1

I don't think it's natural to always look someone straight in the eye. When I do it people get squirmish.

--
Paul: Father... father, the sleeper has awakened! - Dune
1. Re:Looking someone straight in the eye by Jmc23 · 2013-09-03 08:36 · Score: 1
  
  You might want to try looking at them in the eyes and not the eye. Hint, use both your eyes and look at them in 2.5d instead of 2d.
  
  --
  Don't complain about syntax, grammar, or spelling. There is no.hell like input on android.
Tiny Embedded Cameras by jomama717 · 2013-09-03 07:50 · Score: 1

The only way to solve this is to embed the screen with a grid of cameras clustered around the center, and allow the software to decide which camera to active by detecting where the other party's eyes are on the screen. Redrawing people's eyes just seems like the wrong way to go about it... even if it looks perfect 99% of the time, the remaining 1% will freak the sh*t out of everybody.

--
while [ 1 ]; do echo -n -e "\xe2\x95\xb$((($RANDOM&1)+1))"; done
1. Re:Tiny Embedded Cameras by oag2 · 2013-09-03 13:25 · Score: 1
  
  Errol Morris used almost exactly that setup to interview Robert McNamara in The Fog of War: http://www.fastcodesign.com/1663105/errol-morriss-secret-weapon-for-unsettling-interviews-the-interrotron
Re:Looking someone straight in the eye: eye roll by optikos · 2013-09-03 07:52 · Score: 2

It is quite natural for me to roll my eyes upward, when I must suffer fools. This software will inhibit my far-end image from doing so.
Look into the camera by PPH · 2013-09-03 08:11 · Score: 1

Paste a photo of tits next to the lens.

--
Have gnu, will travel.
Which is false reality.. by Lumpy · 2013-09-03 08:27 · Score: 4, Interesting

Most people do not make Eye contact when talking, people look at the other persons mouth mostly aiming the eyes at the center of the face. direct eye contact is seen as agressive even in the human species.

--
Do not look at laser with remaining good eye.
1. Re:Which is false reality.. by Khashishi · 2013-09-03 09:33 · Score: 1
  
  I suspect that watching the mouth move improves the comprehension of speech, even if you can't lip read.
2. Re:Which is false reality.. by dinfinity · 2013-09-03 11:50 · Score: 1
  
  The mouth and nose area are also important indicators of (micro)emotion ( http://www.apa.org/science/about/psa/2011/05/facial-expressions.aspx ), which could explain why many people (and women especially, I've noticed) regularly glance at that area when they are talking themselves. I believe it is to gauge the emotional reaction of the conversation partner to what they are saying.
  See also:
  http://theconversation.com/face-value-where-to-look-when-you-want-to-read-someone-11219
3. Re:Which is false reality.. by locketine · 2013-09-04 17:09 · Score: 1
  
  Only antisocial people are afraid of eye contact. It's normal for non-nerds to look eachother in the eyes, especially if they're friends. If they're enemies then yeah, eye contact is intimidating but not as intimidating as looking at the club you're going to beat them with. See wikipedia for more information: http://en.wikipedia.org/wiki/Eye_contact
  
  --
  Think globally but act within local variable scope.
Re:NO WAR FOR BIG OIL by Lumpy · 2013-09-03 08:35 · Score: 1

What is funnier is the moron thinks that GW bush was a peace loving rights fighting for president. The REpubs here have short term memory problems as they forget that that scumbag president was the single most unamerican president ever. He signed the PATRIOT act.

--
Do not look at laser with remaining good eye.
Might work if people look at screen not camera by Big_Breaker · 2013-09-03 08:38 · Score: 1

Didn't RTFA of course but the smart way to do this is to start with video of someone actually looking at the person on the screen rather than the camera. That way camera sees an off angle but "correct" picture. that picture is then rotated so the remote video looks "right".
Re:Occultation problem by Guspaz · 2013-09-03 08:39 · Score: 1

The Kinect does not use stereoscopy. It projects a IR laser grid (well, randomized) and then uses a single monochrome IR camera to see the grid and calculates depth from that. There are two cameras in the Kinect, an RGB camera and a monochrome IR camera, but only one is used for depth.
Link to paper and video by Nova77 · 2013-09-03 08:42 · Score: 2

It's a 2012 siggraph (Asia) paper. Here's the link with the video.
http://graphics.ethz.ch/publications/papers/paperKus12.php
Related: gaze-tracking hot at 2013 SIGGRAPH by peter303 · 2013-09-03 09:56 · Score: 1

There were several frontier demos of how to use gaze tracking for video games and variable resolution rendering . I think this is facilitated by turnkey table top boxes that can track your gaze.
Re:Uncanny valley by Zontar+The+Mindless · 2013-09-03 17:28 · Score: 1

I had to look it up, so it wasn't well-known to me, not being a specialist in human aesthetics and all like you, apparently.

--
Il n'y a pas de Planet B.
why this technical overkill? by NoZart · 2013-09-03 19:51 · Score: 1

a few years back, i have seen a demo where they used 2 cameras at the sides of the screen, made some interpolation vodoo and voila, you could look at the screen AND have eye contact. Worked like a charm. I always wondered why nobody included this in their notebooks, as it also worked "one-sided" where the user on the other side did not have this setup.
Did anyone read the article? by essbase_nerd · 2013-09-04 01:55 · Score: 1

The tech shifts the angle of the person, not just the eyes.
So, eye roll at will.
Re:NO WAR FOR BIG OIL by Lumpy · 2013-09-04 02:30 · Score: 1

Wah,... Republican doesnt want to admit his blessed leader is the scumbag that started it all... wah....

--
Do not look at laser with remaining good eye.
I have eyes! by myth24601 · 2013-09-04 03:39 · Score: 1

Can it be set to give me a better view of women's cleavage while still making me look like I am making eye contact?

--
No matter where you go, there you are.
That could easily be engineered... by gwolf · 2013-09-04 05:13 · Score: 1

Using an, umm, extension to the world-famous FUFME of yesterdecade.
(Oh, are you a newcomer? Well, their site is long gone, but you can start by reading http://www.dooyoo.co.uk/internet-sites/fufme-com/377859/ to see what it was all about)
Re:Uncanny valley by jimshatt · 2013-09-04 08:52 · Score: 1

And why is everyone looking up in the pictures? Who has the camera below their monitor?
Yes, I noticed that too. I guess they're exaggerating the problem to make their solution look better?