Slashdot Mirror
What Will Apple Do With Swedish Eye-Tracking Technology?
andylim writes "An article on recombu.com explores the possibility that Apple is gearing up to launch eye-tracking technology soon. Citing a patent filed in 2008 that mentions 'gaze vectors' and a recent purchase of units from a Swedish eye-tracking company, the author suggests that the inclusion of eye-tracking tech in the company's forthcoming tablet would be Jobs's magnum opus. 'What better flourish to a career that began with the popularization of windows, icons, mouse and pointer than to usurp them all?'"
Too soon?
Sent from your iPad.
I'm not Swedish. Am I immune to this technology?
Dewey, you fool! Your decimal system has played right into my hands!
I can see how Apple's marketing department would be interested in "gays tracking".
Give me Classic Slashdot or give me death!
Not sure if this technology would apply to it but I've always wanted a computer with main unit, say the size of a cell phone, wired to a set of LCD glasses (preferably transparent so you could see whats going on around you while using it). Then you could navigate with voice commands, gestures and eye movements.
Though with multi-touch coming these days you could have multiple mouse icons and use eye movement and mouse movement on the same computer or instead eliminate the mouse and never have to take your hands off the keyboard to navigate (yes some of us use computers for more than porn).
Just my $0.02
'What better flourish to a career that began with the popularisation of windows, icons, mouse and pointer than to usurp them all?'"
Eye tracking technology doesn't usurp ANY of that. If anything, eyetracking technology makes windows and icons more useful, since those are designed to hold your attention for the short span that you need them.
And don't think that this technology would ever replace the mouse. You need a mouse for gaming, amongst many things. One such annoying technology around today is rollover ads. Our eyes often make tiny glances at colours and items that grab our attention.
Point is, they aren't changing the existing system, merely adding onto it.
I can't imagine this technology working well with glasses.
Bikini Team.
Eloi are stupid, throw morlocks at them!
I don't really think that Apple will use eye tracking... yet. Why? Because there aren't enough existing products out there. The vast majority of Apple's products show up when there are 1 or 2 other early products out there that Apple can improve on. Eye-tracking isn't used in any major way yet and so I don't think Apple will use it quite yet.
Taxation is legalized theft, no more, no less.
Having invented the home computer, the mouse, the GUI, the portable music player, the multitouch screen, the Unix kernel and the mobile telephone, Apple now invents eye tracking. Pay no attention to those Swedish guys in the corner and put your hands together for yet another amazing Apple innovation, personally created by Steve Jobs, the smartest man in the world. Thank you Steve!! I love you!! Please, take some more of my money.
Left to Google, you would have a subtle ad show up around the point of vision on your browser.
Microsoft - sorry i cant think evil today.
Yahoo would just keep the patent on a shelf and implement some unwanted feature three years after the patent expires.
Apple - i just dont know but I know that i will love it !!
There are two main problems with eye tracking. First, your eyes are always moving. Second, they’re attracted to motion.
Eye tracking, done correctly, would have to avoid both of these pitfalls. It would be possible, but tricky. It would have to differentiate between the constant motion of your eyes and deliberate motions that you wanted to make, or at least not be adversely affected by all of their unconscious movements. It would also need to avoid causing movement or changes on the screen that would draw your attention away from what you intended to look at.
For instance, if a normal cursor was displayed at the detected position of your gaze, it would (A) obscure, (B) distract, and (C) float irritatingly away from your gaze if its positioning was even slightly miscalibrated.
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
It always seemed to me that the greatest benefit of tracking the position / geometry of a user's eyes would be for determining their focal point. The user will look at your information directly if it's needed, but if it was always in focus they will be less fatigued by constantly changing focal points. In handheld devices this would allow you to glance at your phone by bringing into your field of view without having to take your focus off the road / sidewalk. Refining the technology enough that people could use computers at work without having to have corrective lenses on / in would do wonders.
I hate it when I look at a field and start typing only to find out that something else has focus. This happens to me in every GUI I've ever used and if a webcam with gaze vectoring can fix that I'd really like it.
They'll get rid of the track pad and you'll use your eyes to point at stuff on the screen. It will be intense.
I mean, what else would you do with Swedish Eye-Tracking technology? Track Swedish Eyes, obviously...
Bow-ties are cool.
If they eye tracking tech interoperats with the built-in web camera so the device sees what you see (not just want part of the device screen you are looking at) this could be used to delivery data about whatever your are seeing. It could also be used to deliver targeted advertising.
They're going to build an ebook reader app for their rumored tablet to kill the kindle & dominate the market. as Ramanujan once said for a famous one line proof, "Behold."
~dijjnn
One of the big challenges in videoconferencing is the illusion that the subject, who is looking at your face on the screen, appears as if he or she isn't making eye contact with you, as the camera is not located in the middle of the screen. While this may seem minor at first glance (ha ha), it's actually a pretty important issue in videoconferencing, with significant demand for software that corrects it.
A "gaze vector" is exactly the kind of information software would need to "correct" the illusion, to make it seem like the subject does have eye contact. I bet Apple is going to incorporate eye contact correction tech for videoconferencing in its products.
Oh wait, you said Swedish Eye-Tracking Technology? I thought you said Swedish Eye-Candy Technology.
Never mind.
They're going to have flashy, blinkey, animated ads that follow your eye movements, making sure you can't take your eyes off of them without looking away from the screen.
I hope the make it so you can shut the eye tracking down. As to the mouse, I can see an eye-controlled cursor in the future. I, for one, would be happy to have the mouse replaced; I get "mouse elbow" if I'm at the computer too long.
Free Martian Whores!
http://en.wikipedia.org/wiki/Autostereoscopy
The webcam already on top of most apple products will be used to track the eye of the user and adjust the display accordingly.
MacOS 3D anybody?
if they're tracking Swedish eyes, it won't be long until privacy advocates have to go get non-Swedish eyes from some skeevy drunk "doctor" like Tom Cruise had to in "The Minority Report".
Eye-eye, sir!
"My strength is as the strength of ten men, for I am wired to the eyeballs on espresso."
Tablet with built-in iSight camera + OS supported eye-tracking = accessible computing/media/network tool for people with little or no manual abilities. If it actually uses the iSight and not a head-mounted tracker, that's a large benefit in terms of ease of use & maintenance.
Full disclosure: I am a vision researcher who has worked with most of the available eyetracking systems on the market.
I had not heard of the company, and a quick look at their product line tells me why not: their standalone systems are limited to 60hz/120hz depending on the model - this was good several years ago, but has been considerably eclipsed by other companies' designs (e.g. S-R Research's Eyelink hardware, which happily does 2000hz monocular tracking). It looks on par with ASL's Eye-trac gear, which has similar limitations.
I would bet that Apple just bought a few of their systems to use in internal testing - I sincerely doubt that anyone there is using such slow gear for major research.
You can track where the eyes are looking (i.e. the intersection of the eye gaze direction with the surface of the screen), but this typically involves a lot of calibration, and often more than one camera, just to make it reliable. Even if it's reliable, it's horrible to use - they have these on display at conventions all the time, and they're generally used for checking if advertising works. The other is to track the position of eyes in an image and a rough direction of where they're looking. You can then use this information to augment the image captured from the camera to make it look like the eyes are looking at the camera (when in fact they were looking at the screen, as in video-conferencing). I suspect this would be the more likely technology to make it into a tablet/laptop.
The research lab I work at bought a Tobii system recently. During our pilot gaze study we discovered it had a strong tendency to only track white men. In fact, it only worked on white men. Our theory goes, white men were the individuals programming and testing the unit.
Canon had eye tracking in their A2E Film SLR camera 18 years ago - how is this different? Would the A2E count as prior art?
I had a Canon Elan with eye tracking. Worst camera I ever owned. The problem with it was my eye wouldn't be lined up the same each time I used the camera so it was always off. I found myself trying to reposition my whole head to line it up for focusing. What a joke. I'm sure it's better after all these years but it's likely to be far more trouble than it's worth and little more than a gimmick at best.
An eye for an i.
Maybe this technology can be used to let me finally catch up with those little floaty things in my eyes that keep getting away every time I try to look at them.
There was a story some time ago about Apple patenting small, "hidden" in the screen cameras as a means of correcting eye contact issue that exists currently in videoconferences.
Which really strikes me as another example of why patent system is badly broken in the US. Even I toyed some time ago with an idea of using small sensor / optical arrangement that minimizes size of the "camera", visible obstruction, so it can be placed in front of the screen without being too irritating. Hiding it between the pixels of LCD screen, when you have good enough manufacturing, seems to be just...a straightforward progression.
One that hath name thou can not otter
'What better flourish to a career that began with the popularisation of windows, icons, mouse and pointer than to usurp them all?'
So we're to assume from this that Steve Jobs (and by extension Apple, the company he co-founded) didn't do anything worth speaking of before the Lisa?
I have a lazy eye =(
Will be the shout of everyone trying to read a document and having the cursor automatically and constantly position itself right in the center of your vision. You be begging for the mouse to get that out of your way.
jsut athnoer menagiensls ltitle psrhae for you to dcoede. Why do we wtsae our tmie dnoig tihs?
I can imagine how the pointer moving ever so slightly away from where you are looking causes you to try to move your focus to where the cursor now is causing a cascading effect of chasing the cursor that is just out of focus and moving. It will eventually cause us all to have spastic eye movements constantly circling the page. That will be fun!
jsut athnoer menagiensls ltitle psrhae for you to dcoede. Why do we wtsae our tmie dnoig tihs?
The iPhone has multi-gesture apps because Apple first came to the broad market with such a device. People will build software to what hardware manufacturers make popular, ont the other way around.
jsut athnoer menagiensls ltitle psrhae for you to dcoede. Why do we wtsae our tmie dnoig tihs?
What the fuck are YOU looking at?
So these days, in order for Steve Jobs to to market effectively, he needs to. . .
1. Sit down and really think before jumping.
2. Repackage old technology with sensible user interfaces and thus change the world.
3. Do nothing new for a few years.
4. Announce nothing.
5. Let the world speculate with wet-dream anticipation until it infects even a decidedly biased anti-Mac forum like Slashdot.
6. Do nothing.
7. Do more nothing.
8. Release some more old technology with a sensible user interface and make everybody orgasm. Again.
9. And yes, Profit.
The man is either brilliant or the rest of us are just really slow.
And while I admire Jobs for being able to see, I can't stand Apple stuff. It's all designed for pod people. The part I can't reconcile is that he sees that people really ARE from pods and rather than swim against that tide, instead makes baby toys and rakes in the 'Wow'.
-FL
Maybe if your definition of pod-people means non-technologists, or even technologists who stick within a realm of expertise.
The "sensible user interface" of your post is what allows people to see what the technology can do; those people who don't have the time and/or inclination to spend so much time with technology that could be made to do something cool if you first (a) figured out what that is and (b) made it a reality.
What took phones so long? If the iPhone simply represents a "sensible user interface" (and you'll not get an argument to the contrary from me), that means that older phones simply represented insensible user interfaces. Would not having figured out how to do conference calling from an older cell phone make you a pod person?
I'm not playing down invention. Invention is vital, and more fundamental than popularization. For some technologies, popularization is unnecessary - their domain is limited. However, where appropriate, popularization is incredibly important. Enabling the average person to use new capabilities is what Jobs sells, and it is valuable.
He doesn't make things for pod-people, he makes them for programmers who are sick and tired of crappy user interfaces. Even Tivo, which is the best DVR by far, is a crap product compared to most of the things Apple makes. I am constantly thinking to myself "why did anyone make something like this, I could do way better" when I use non-Apple products. I think that the software designers at Scientifica Atlantica, for example, need to kill themselves out of shame. They make products that are an embarrassment to our entire profession. At least if the DVRs that they make for Time Warner are any indication of things.
With eye-tracking, you can do away with the painful auto-scrolling on mobile/PDA eBook reader software and just have the page advance when (or a few seconds after) your eye gets to the last line of the page...
"They've canceled the show but we're still here. What does that make us?" "Big Damn Junkies, Sir!" "Ain't we just"
I have nystagmus, which means that my eyes constantly wiggle though I perceive an unmoving image. I bet that an onscreen pointer calculated by my gaze vector would be in constant motion and therefore unusable or at least very difficult for me.
I say this not in complaint, merely in observation. I'm sure this won't be adopted in the near future, and who knows if it will catch on once it's developed? But as someone who never before had to concern himself with being physically able to access things, it is an interesting shift in perspective.
Here's hoping that mice or touch will remain supported for the new disabled =)
Your brain is not a computer.
eye-tracking linked with auto scrolling would be nice. as well as zooming (which would finally rid us of this multi-touch nonsense!)
Just that a. Apple doesn't really know how to do it right (they may have an idea, but will fail), and the hardware is still not there...
Really? do you consider iTunes to be a non-crappy user interface?
Humans are only able to perceive fine detail directly along the gaze vector: peripheral vision is mostly restricted to general shapes, colors, and motion. So a rendering system, like a FPS game, that tracked the gaze vector could determine where in the 3D scene a user's gaze was directed, and render the geometry and textures enclosed by the cone centered around that vector to a very fine degree of precision while leaving the rest of the scene very coarse. The user would perceive that high degree of detail everywhere as the eye scans the display frame by frame while the system is actually rendering many fewer polygons and pixels.
Wouldn't it be possible to make 3D Images with eye-tracking technology.
What will Apple do with it? "Not a damn thing" would be my guess, at least not in this decade. They just bought the technology so nobody else would, just in case.
I can't wait for the pop-up ads that follow your eyes around.
Apple already uses the "fish eye effect" to magnify icons in the Macintosh dock as you mouse over them. I expect that they'll use eye tracking to magnify whatever you're looking at. (That should greatly please they guy who said "Two words: Bikini Team.", immediately above.)
In fact, I'll go so far as to predict that they'll add it to the Universal Access system preference (assistive technologies for the impaired) as an alternative under the Zoom feature. That way, you can turn it on and off with a keystroke, for when you want the visual representation not to be distorted (drawing, for example). Or there may be a keystroke to switch between eye-centered zoom (fish eye) to the current mouse-based zooming of everything with truncation (flat). That way, a visually-impaired person who needs zooming on all the time would be able to switch easily between fish eye (to see where they are in the window better) and flat (to draw better).
Also, I expect 3rd party game developers will use it for heads-up displays in first person shooters. That might mean extending extending the graphics port metaphor to include two focal points, one for the mouse cursor and one for the visual focus. (I'm assuming that game developers would want the guns to fire where the mouse cursor resides.) Once that paradigm has shifted, there'll be demand for two mouse cursors to allow gamers to carry two gun-like game controllers and fire them independently at different targets.
So no, I don't think that visual tracking is "not Apple-like" at all. It's the sort of cool stuff we've come to expect of them, actually.
telescreens anyone?
No, I don't.
Clearly, the context in which my comment was made specified that a sensible user interface was a good thing, not to be casually dismissed. I'm unsure where you got the impression this meant every Apple product met that standard.
If Apple was seriously interested in this company's tech, they'd have purchased the company and not a couple of units. I'd say a much more reasonable explanation is that Apple has has a Human Factors Engineering lab where they'd like to track where people are looking when using their products, so they can better place things like menus, icons, and buttons.
/sigh, sorry, misread who you replying to as the other comment was hidden. Carry on.
I'm sure we all remember this demonstration from a couple of years ago using Wiimote hacking:
http://www.youtube.com/watch?v=Jd3-eiid-Uw
Get rid of the clunky IR hardware, track eye movement directly, and you've got the kind of potential for desktop sexiness that only The Steve could bring us.
Jobs' was selling personal computers for 6-7 years before WIMP, for 5-6 years before the IBM PC. And since WIMP the object-oriented NeXT tools were used by Tim Berners-Lee to create the World Wide Web, and there was this thing called the iPod which had a whole generation named after it. And after that came a multitouch phone you might have heard of, with both an iPod and a Mac in it. Jobs doesn't need to look for a follow-up to bringing WIMP to consumers.
And the Apple tablet doesn't need eye-tracking to be interesting. It has the potential to make the IBM/Microsoft PC look like as antique as the typewriter it replaced.
Even if the technology is perfected, there are some interesting privacy issues. People don't have complete conscious control of their eyes, and where someone looks at an image can reveal information that they might wish kept private. Are you looking at the cute girl in the picture- or the cute guy? Are you looking at the image of the fancy car - maybe you should get a targeted add. Related technology may be able to read something about your facial expression.
I'm not necessarily opposed to the technology, but I think there need to be some limits on how the input data can be used. So far what you input to a computer or phone is completely under your own control. This would provide some input that you did not control.
Sorry, andylim, the acronym WIMP stands for 'windows, icons, menus and pointer', not 'windows, icons, mouse and pointer'. A common misconception, but wrong - take it from someone who was there.
Eye tracking is not just a potential user interface tool, it is also an advertising tool. Advertising may even be eye tracking's primary use. Verifying that you looked at an ad and recording how long it held your attention is useful information for an advertiser. Also consider how ads are typically priced, either by some large number of impressions or by actual clicks. For impressions these are unconfirmed impressions. They may have been inserted into a web page but there is no confirmation that they were actually noticed. With eye tracking a third pricing option may be introduced, one for confirmed impressions where the ad held the user's focus for some minimum time duration.
--
Perpenso Calc for iPhone and iPod touch, scientific and bill/tip calculator, fractions, complex numbers, RPN
I'll get my coat...
"Be nice, veer left, and never stop thinking" Iain Banks - Walking On Glass
Can't you solve this just with an optics dealigned wrt the camera CCD? /. )
This is the way used, for instance, to get buildings straight while they are imaged from a point where obviously the perspetive would deform them entierely...
I think it is called 'axial correction' or something alike in ordinary photo, see for instance http://www.mir.com.my/rb/photography/hardwares/classics/olympusom1n2/shared/zuiko/htmls/35mmSHIFT.htm , by the middle of the page you get impressive examples...
(of course this is an hardware solution, so even as a simple one it should probably banned on
Herve S.
I can see this being useful in addition to a mouse/trackpad. Quite often I'll be working with a lot of windows or clickable content on the screen and I can look at the widget I want to click on faster than I can get my cursor over to it. I'd want to be able to turn it off quickly though.
So we took an Eye and put an i innit, iEye, ay caramba.