Attention Sensitive User Interface

griffjon writes "The NYT (lame free reg blah blah) is running an article on Microsoft research into an attention-oriented UI that will use cameras and mics as well as software to monitor where a user's attention is focused and query other software (like e-mail notification, IM, etc.) to keep it from interrupting their chain of thought." This strikes me as being a really cool idea if properly implemented. Even simple things like not letting your biff update until you change focus out of a word processor. (mind you the anti-MS block on Slashdot will of course equate Microsoft's involvement with the project to mean that this is really about mind control or the corporately financed return of the plague, but what are ya gonna do?)

Comments

Just try and implment this

[dashmaul]

To come up with something that works well, would take enormous amouts of time and resources, as well as adding huge over head to the system. Hmm, wonder if intel is gonna push this.

This kind of technology would us be seriously in danger of doing nothing but annoying the end user. Ever gotten into a fight with Microsoft Word over some formating issues? It can be dang near impossible to get it to do what you want because it is being so helpfull.
Still if ever got impletmed correctly, and wasn't annoying, it would be nice.

Re:Just try and implment this

[fiziko]

Ever gotten into a fight with Microsoft Word over some formating issues? It can be dang near impossible to get it to do what you want because it is being so helpful.

That's why I switched to LaTeX. Well, first I switched to StarOffice for the equation editor, and then I switched to LaTeX for the excellent cross-referencing, table of contents generation, and damn near everything else Word wouldn't get right the first half a dozen times I tried. God, Word's automatic outline generator? I still haven't figured out how to keep the numbering accurate automatically when I change the document. Maybe there's a way, but I never found it. LaTeX is, in my opinion, a far superior product, with much lower system requirements. It's basically a markup language, but anyone with a modest combination of IQ and HTML (eg) experience should be able to pick it up in under half a day.

Full monitoring?

[Saib0t]

Well, this sounds to me very much like we're going to be monitored more and more.

We have rich decision software called Bayesian Inference Software that we can build down into the system that can track your usage and adjust in an automatic fashion

As someone mentioned already, this can be used for many another thing than what this is intended.

A couple of these uses c(w)ould be:
- Permanent monitoring of the users with the camera. It can already be done right now actually, but a boss deciding to put a webcam on every machine for supervision purposes will make everyone feel 'spied'. This system would provide an 'excuse' for having webcams on every machine.
- Advertisement banners can now position themselves where you're looking.
- Since the thing would monitor the user's activities in order to determine what to give the focus to (or what to prevent being given focus to actually), it'll be easy to keep trace of the activities of the user: slashdot reading 2 hours, coding 4 hours, speaking jokes with colleages 30 minutes...

This thing really raises a couple of disturbing issues. I may be paranoid but I don't like monitoring systems. At least are they aware of that: And Horvitz and his researchers themselves acknowledge that the information collected by the notification manager software -- potentially, information on the personal activities and movements of millions of people that would be stored on the Internet -- raises privacy and security issues that have yet to be resolved. But I doubt those issues will be resolved.

Ignoring the system requirements to run the thing - that certainly aren't trivial - the system could be useful, if the user is given to set the "disturbance value" (or "worth") of possibly disturbing events. But that would be a hell to configure, imagine every morning having to say to the program "I'm waiting for urgent messages from person X and Y" and changing that everyday. I doubt very much that a program will be able to determine what I think important and what not.
A simple example of this would be that a message I receive from a colleage might very well be the information I've been waiting for 2 days but also an email to, for example, notify me of a colleague's birthday party.

Another thing I'm skeptical about in the article is: He expects that the system will be able to greet and converse with new visitors. The conversation, he says, will be on par with speaking to a person who is hard-of-hearing.

AFAIK (As Far As I Know [I realised that acronysm's meaning only very recently...]) no current software is able to converse with a human being. Answer a couple of pre defined questions maybe, but certainly not converse.

Stare ....

[HiQ]

So now the system will actually *know* when I'm staring that stupid paperclip to death; I hope they will implement a new feature: when you stare madly at the paperclip, it will catch fire, and will be reduced to a pile of ashes! Kewl!
How to make a sig
without having an idea

Heh

[FascDot Killed My Pr]

So kids with Attention Deficit Disorder AND Automatic Power Management are going to have their computers shutdown on them every 10 seconds...

Seriously, this sounds neat--if it works. But I can imagine new programs trying to compete for my attention by flashing, show nudie photos, or whatever in an attempt to boost a Nielsen-style rating.
--

But I WANT to be distracted!

[(void*)]

I want to know when email comes, even in the heavy midst of coding, becuase it could be email from my wife. I want IM to tell me when the boss logs on, becuase I have Important Stuff I wish to report to him.

Why is MS always thinking about "how cool this ..." or "how cool that ...". Don't they realize that many of this is just straitjacketing people into one set of actions or options? Perhaps a droid might like it, but I am not a droid. I am a human being with priorities that cannot be turned into a well-ordered list.

What about Outlook viruses?

[FascDot Killed My Pr]

So what if I'm using Outlook? A mail comes in, I glance at Outlook--it opens the mail. It has an attachment, I look at the filename--it launches. Oh no! A virus! Don't infect Word (glance at Word). Crap! Don't send to the people in my address book (glance). Dammit!

There are times when you want to study something passively...
--

Let Me Tell You Where to Go Today. . .

[larsal]

Novice users are already distracted from the work they're desperately trying to figure out by every blink on their DSL modems, every whirr of their hard drives, and every change in the "helpful" indicators, telling them that they're on line 8.6", no, wait, 9.4" of their document.

Now try to imagine the same people believing, thanks to a new, ritalin-demanding UI, that they're supposed to be dealing with all the random odds and ends of software and background apps [already needlessly numerous] the UI decides they've been paying attention to!

"Am I supposed to deal with the 'Task . . . scheduler' now?"

"No, you're writing an essay."

"But it came up and. . .look! The calculator just started! Oh! 'Help'! That must be useful. . ."

"It's the 'Help' function for the calculator. . ."

"I wonder if it can help me write the essay?"

Larsal [It's Worse than an Animated Einstein]

They already do this

[cs668]

and.... It sucks.

The menus in office 2000 apps already try to guess what you want to do. They are always moving menu entries arround based on what you are doing.

The end result is that you always have to read the pull-down menu -- you can never learn the position of a selection.

So by trying to anticipate what you want they make you less effecient. Wouldn't this just be more of the same?

IBM as well

[jonathanclark]

I went by the IBM software research labs here in San Jose and got see some neat demo of exactly this (attention sensitive UI).

The nice thing about this is that eye tracking is very cheap. The eye reflects IR very well so all you need is an IR strobe and a cheap IR CCD. An end product could cost less than $50.

One demo allowed you to speed up mouse click on things by automatically moving the mouse to an approximate location on the screen where you are looking.

They had one demo that would track your eye and blur the screen except for where the eye was focused. Everyone else sees a blurry screen, but you (the person being tracked) can't see a difference. Could be very cool in 3d games if the game could render the areas of the screen you were looking at in more detail and those you weren't in less detail. The military has been experimenting with this on high-end flight sims that do this with good success. But if your playing on a 13" monitor then pretty much everything is in focus. :)

Checkout their project page for a little for info.

http://www.almaden.ibm.com/cs/blueeyes/

--

CmdrTaco has it wrong.

[Anonymous Coward]

I don't think this is stupid because Microsoft is involved; I think it's stupid because it's stupid.

This is nothing more than Bob with hardware. It's that idiot paperclip on steroids. It's just one more way that Microsoft insults the intelligence of their customers. I don't need an idiot paperclip popping up to tell me that it "looks like" I'm writing a letter. I know whether I'm writing a letter, thanks. If I were functionally illiterate then perhaps it might be useful to have dopey software "help" me write a letter. But Microsoft makes the blanket assumption that ALL its customers are functionally illiterate.

I certainly don't need my computer deciding for me whether I should be notified that I have mail. I think I'm capable of making that decision for myself.

No thanks, Mr. "Chief Software Architect" Gates.

Two weeks of running Windows 2000. Two Blue Screens of Death. Thank goodness for Linux!

Badly-behaved software: Attentions & distractions

[tjwhaynes]

The problem with a lot of the software I see around today is that in the desire to make software more open and friendly, it has got a lot more distracting to use. It's difficult to Zen-out when using a piece of software when every minor adjustment triggers an animated effect, be it a spinning hour glass, back illuminated button or piece of paper flying across the screen. In an attempt to give the user more feedback about what is active and what is not, software designers have taken away the "quiet" interface and have jazzed it up.

And this has not been restricted to just the application itself. The applications often demand attention like some spoilt brat - the "HELLO? YOU HAVE MAIL!!!" syndrome. While in some cases, such as Lotus Notes, the default is to rise to the top of the window stack and bang a modal window up to get your input everytime there is new mail, you can tone this down to an audible bell only. Or ICQ clients which reappear on the top at a new message coming in. And there are others - visual alarms on calendaring tools and probably more that I have forgotten.

When I have the option, these programs are pushed into the bit bucket as fast as possible. Using them is a dire waste of productivity. Where there is no choice about using that software, I try and tone down the alarms to be just audible effects which I can acknowledge without having to press a key, move the mouse or otherwise stir from whatever I'm doing.

So really, this research sounds like a patch for the problem, rather than a cure. The problem is with the UI design - programs are increasingly "rude" in their attempts to get attention. At least if I hold the source, annoying habits in essential software can be trimmed to a minimum. But rarely in the Unix side of the world do I have to worry about annoying software - 95% of the stuff which irks me is Windows-ware. Maybe the art of Zen is dead on the MS platform...

Cheers,

Toby Haynes