Slashdot Mirror
Eye Tracking Coming To Video Games
An anonymous reader writes "Over the past several years, we've had a spate of new input methods for playing video games. Instead of just pushing buttons, now we can wave body parts around, yell at the screen, or even (weakly) control things with our thoughts. Now, we're adding an eye-tracking sensor bar. It's being created by SteelSeries, but it's based on tech from a Swedish company called Tobii, who built similar tech for cars. 'Inside the device there are two cameras and an infrared light source. The infrared light reflects off your pupil and cornea, which is then captured by the two camera sensors. Throw in a healthy serving of Tobii's proprietary image processing algorithms, and a physiological 3D model of the eye, and you can work out the position of the eye and the direction of the gaze with high accuracy. Tobii doesn't seem to put an exact figure on the resolution/accuracy, merely saying that "within less than a centimeter" is possible.' Of course, the biggest question will be how well it works, but it seems like it could be a useful supplement to normal control schemes. I can see how it would be nice to simply flick your eyes to an icon to do something, or to make it easier to dig through your in-game inventory."
I can see how it would be nice to simply flick your eyes to an icon to do something, or to make it easier to dig through your in-game inventory."
I can't see that being nice at all.
I may look at an icon, then decide not to use it, when considering my options. Or what If I want to activate an icon but need to activate it at a precise moment... I have to avoid looking at it until the right moment? That's not user friendly.
On the flipside, presumably there will be some method of preventing spurious activations, will I have to do some sort of obnoxious eye waggle every time to "confirm" yes I really do want to activate it?
All in all it sounds downright awful and useless for most scenarios. I can't really think of a scenario where it would be more useful than obnoxious.
Even a simple 'pause' when I look away from the screen would be obnoxious.
I'm not against it though, maybe it will let one handed people play more games, or some paraplegic will be able to play doom by blinking at the screen and it betters his life...
But I don't think I want this rammed down our throats as something we'll actually need to do to activate something in a game.
I remember watching a show 20 years ago about gauging the accuracy of advertising by using eye tracking. They took people off the street into a special truck (it was bulky equipment), and they could very precisely see how the men's gazes never quite reached the lingerie brand name in the ad's corner.
There is no way anyone would ever want to use an updated version to track your ad viewing while browsing. Nope. Not gonna happen. Totally not running to the patent office (if Samsung doesn't have that one yet, someone needs to be fired).
Eye-tracking is a big honking window into the subject's subconscious mind. I will be extremely reticent about releasing that data into an information system.
It's creepy enough seeing the amount Google knows about me already just from searches and cookies. I sure as hell don't want advertisers to get fine-grained feedback about which ads attract my attention, never mind cranking up the distracting peripheral-vision movement to force my gaze.
And don't even get me started on the evil tricks you can play by keeping things just outside the user's central vision, no matter how hard you try to look directly at them...
It would be cool to use the eyetracker to determine what objects the player is looking at. Measure distance from the player-camera to the object being looked at and adjust the focal dept of the camera in real-time to that constantly changing distance. The result would be a 2D projection which achieves the illusion of continuously variable focal plane based on where you look and what you look at. Combine THAT with stereoscopic equipment like Oculus-rift and you have one hell of a VR setup.
-ubuntu others as you would have others ubuntu you.
I don't care about games, but I would love it when they can adapt this technology to give X magic focus powers.
I already use 'Focus follows mouse' on fluxbox, but too often I am looking at a differnt terminal window when I start typing away, and I had forgotten to move the mouse.
[Paranoid Mode] imagine how much more advertising money Google could make if they tracked WHERE you were looking. [/mode]
"The price good men pay for indifference to public affairs is to be ruled by evil men." ~Plato (427-347 BC)
So, in short, Tomb Raider X will know exactly how much time us perv gamers spend staring at her um muscles.
I know that there's this, but not sure if it's the same technology. Not that I care. But the military has had this in operation for some time now. ...Like everything else, I guess.
Politics; n. : A religion whereby man is god.
Context is always missing from computer tracking. Without context there is always a shadow of a doubt.
The dangers of knowledge trigger emotional distress in human beings.
More usefully, tracking eye movement could be used to control the camera. Tilting and panning to bring anything the user concentrates on to the centre of screen would be useful in a lot of games, and would get around the poor camera placement algorithms we've all been annoyed by.
It would have not to be too sensitive - the user glancing at a status display or an incoming message should not move the camera - but I could imagine this being really useful.
I'm old enough to remember when discussions on Slashdot were well informed.
game pads are crap compared to the keyboard and mouse.
All these new interfaces are just means by which an inferior input system can compete.
Its frustrating for game devs to be limited by the gamepad.
Strategy games are totally impossible. FPS games are awkward. Even RPGs need to be simplified and MMOs which are already pretty dumbed down have to be dumbed down farther.
So here is my suggestion for how to improve the situation.
Allow console users to plug in a keyboard and mouse. By all means, streamline it a bit if you must. But that in and of itself would resolve most of the problems.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
... "I" am doing no such nonsense. Looks more like a blatant slashvertisement for yet another worse-than-useless gadget than anything useful.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
The best games that I have played required a d-pad and no more than 8 buttons (most often no more than four). That includes several RPGs.
Maybe gamepads are crap. Maybe games built by devs who have poor UI design fundamentals and discipline are crap without keyboards.
The brain rarely lets the eyes stand still. Eyes are actually quite poor resolution anywhere except for the very centre of their visual space. The eyes will dart around often more than 3 times a second when you're not paying attention to anything specific and the brain puts the pictures together to build an image of what the surrounding environment looks like. The bigger the screen the worse the movement.
The idea of using the eyes focus point for selection criteria is borderline absurd. You open a screen and the first thing your eye does is dart around attempting to look at everything. It continues this process even as you click on something. Every case of sight based selection I've seen has required either immense concentration and a long time to confirm, or it involved using a display so small that you can't meaningfully discern what is being looked at (think Google Glass style HUDs).
Why do I want this again?
especially gamers who looked like this. http://www.imdb.com/name/nm0001204/
I got to the chocolate box before you, that's why the hard ones have teeth marks.
Not 30 minutes ago I finished building a point model (sort of like this one), which I'll use for head tracking in combat flight sims. I tried using face tracking, but it wasn't reliable enough.
IMHO, head tracking is better than eye tracking. For head tracking, a multiplier is configured so that e.g. turning your head 20 degrees turns the game POV all the way around. Once you're used to holding a controller with your face, it's pretty easy / natural to use. On the other hand, it takes a lot of effort to keep your eyeballs from constantly saccading around the room.
.: Semper Absurda
Tobii has been using it in adaptive communications devices for year. My late wife suffered from ALS, and she used a Tobii C17 with Ceye for speech, since she was no longer able to talk.
General Relativity: Space-time tells matter where to go; Matter tells space-time what shape to be.
You could increase polygon tessellation at an area of focus and remove detail from the peripheral. It would be very useful for triple 120Hz 4K 3D. ;)
But man, would it ever be useful for real work. Simple things, like making whatever windown I'm focusing on become active.
A good eye-tracking system could replace a mouse, with maybe a pair of buttons right below the space bar on the keyboard or something.
I would think blink or wink tracking would be more annoying than useful, though...
If the masses can keep you down, you're not the Ubermensch.
I want to be able to look at the terminal prompt I'm interested in, type, and have it show up there without ever touching the mouse or the tab key.
For those asking "what is this good for?" or complaining "why would I want this?", immersive virtual reality is in desperate need of new input methods. A convincing, immersive 3D world is wildly incompatible with our favorite pointing device, the venerable mouse. And "air mouse" is both obnoxiously inaccurate and obnoxiously fatiguing to use.
Gaze tracking integrated inside a VR head mounted display helps address the new user interface problems Rift users are already suffering. A 2D mouse cursor in your 3D world is just no help at all, but there are a great many abstractions that games just have to have (because no one wants the Inventory Box Interface Device plugged into their PC) and those abstractions depend rather heavily on traditional desktop user interface elements overlayed on top of the 3D world, represented as buttons and icons in windows. Trying to use such interfaces inside an HMD is difficult enough that alternatives are desirable. Gaze tracking could be that alternative. It's guaranteed to be less fatiguing than air mouse. If it's more accurate than air mouse, it could be very useful.
Eye tracking in the usability world has been in the 4- and 5-figure range for a while. At least one cheap ($1200) option required that you wear heavyish headgear.
Tobii makes some of this usability eyetracking equipment, and it's my hope that this device will be adapted to work with the popular Morae usability recording software.
Eyetracking will be part of nearly everyone's usability toolkit, vs. an expensive luxury, if so.
It might hold possibilities for eyetracking studies on mobile software as well.
Yes, my eyes followed the woman in the red dress, so what?
-- Tigger warning: This post may contain tiggers! --
What about when I am checking FaceBook with my Google Glass? How will the eyetracker know which display I am concentrating on?
This can be used for simply concentrating the graphics resources in the area that the eyes are focused. Microsoft Research was showing something like this off at last years SIGGRAPH conference. They are claiming an acceleration factor of 5-6 when using the eye tracking.
Their paper can be found here: http://research.microsoft.com/apps/pubs/default.aspx?id=176610
Having stood by and watched while someone was using the system, it was interesting to see their eye dart around the screen and the little high-quality circle that (presumably) was synchronized with the movements.
I want a "focus-follows-gaze" option for the X Window System. When I start typing I almost always want the input to be directed to the window I'm looking at.
Wait until Facebook adds this to their already overburdened user interface. Ugh.
"Facebook, still free to use, but now we require you to view it through our new FaceEyes® anti ad-avoidance technology."
Reduce those megawatts of power being sucked up by video cards: do high-resolution rendering only on the bit being looked at. Of course, it won't work that way; we'll use the same power to render better graphics.
Know how some games have timers on the grinding stuff like death penalties, etc, that only tick down while the game's your active window?
Now imagine they don't even let you look away either.
Dedicated game controllers all the way. None of this creepy clutter shit.
http://gamehacking.org/vb/threads/12747-nensondubois-codes http://twitter.com/nensondubois_
Standard mouse control remains. Some alternate keypress is used for "interact with what I am looking at."
I welcome our new 99% overlords.
Eye-tracking is a big honking window into the subject's subconscious mind. I will be extremely reticent about releasing that data into an information system.
It's creepy enough seeing the amount Google knows about me already just from searches and cookies. I sure as hell don't want advertisers to get fine-grained feedback about which ads attract my attention...
And the NSA wonders why I wear sunglasses indoors at night while using my computer these days... :)
Because soon the children will be too fat and lazy for joysticks and buttons.
That I'm right, and you don't like it, doesn't mean I'm a troll.
Inside the device there are two cameras and an infrared light source. Throw in a healthy serving of Tobii's proprietary image processing algorithms, and a physiological 3D model of the eye, and you can work out the position of the eye and the direction of the gaze with high accuracy.
Throw a healthy dosage of high intensity laser and get an even higher accuracy.
Hasn't this tech already proven useless for general usage due to eyestrain and such from using your eyes to communicate far more? Sure it'd be useful for those who lack other options like Steven Hawkings.
> I can see how it would be nice
Pphft... You think that game makers are going to use this technology to be nice?
The game can use this to great advantage of the player, by specifically creating danger that originates where the player is not looking.
Creative players may be able to trick the game by looking off to the left... at a mirror (or set of mirrors, so things are reversed), so the game thinks the player is looking in one spot when really the player is looking at another.
I also wonder how else the hardware may be fooled, e.g. wearing a pair of glasses with some fake eyes (perhaps just a cartoon drawing; maybe something more elaborate if the hardware is better).
For other players, I hate to think about how gamers may start to damage eye sight, by doing weird things with their eyes in order to (probably successfully) try to trick a game into thinking that they are looking in one spot, when they are really concentrating/looking in another spot.
seems to have already been shown last fall: http://www.prnewswire.com/news-releases/smi-eye-tracking-creates-thrill-in-survival-horror-game-223107871.html
http://en.wikipedia.org/wiki/Nystagmus
Ill just leave this here.
GENERATION 24: The first time you see this, copy it into your sig on any forum and add 1 to the generation. Social exper
The magic bullet for eye tracking is using it for adaptive HDR. Monitors for good reason can't show bright lights. This makes it difficult to fit all the luminance information on the screen at once.
By tracking the eye we can adjust luminance information locally. This can have a huge impact on image quality.
We'll be seeing a lot more of this, marketed as a cool new feature. Everyone makes about 250,000 saccades per day and the vast majority are unconsciously generated. By logging saccade targets one can easily map a person's interests, tastes, fears, sexual orientation, etc. Instead of page views ad companies will be selling saccade targets. That's bad enough, but the NSA will be collecting information about you that you don't even know yourself. With enough effort it would be possible to generate user profiles and identify users soley by their saccade fingerprints.
Just think. Now they'll really know if you read the full EULA!
Most glass (and possibly plastic lenses for glasses) bounce IR. So if I wear glasses, will this technology actually work, and be accurate?
once someone figures out a cost-effective sensor, clever people will figure our new ways of using it