Slashdot Mirror
Attention Sensitive User Interface
griffjon writes "The NYT (lame free reg blah blah) is running an article on Microsoft research into an attention-oriented UI that will use cameras and mics as well as software to monitor where a user's attention is focused and query other software (like e-mail notification, IM, etc.) to keep it from interrupting their chain of thought." This strikes me as being a really cool idea if properly implemented. Even simple things like not letting your biff update until you change focus out of a word processor. (mind you the anti-MS block on Slashdot will of course equate Microsoft's involvement with the project to mean that this is really about mind control or the corporately financed return of the plague, but what are ya gonna do?)
Their involvement with the project is really about mind control or the corporately financed return of...
Never mind.
-- You see, there would be these conclusions that you could jump to
To come up with something that works well, would take enormous amouts of time and resources, as well as adding huge over head to the system. Hmm, wonder if intel is gonna push this.
This kind of technology would us be seriously in danger of doing nothing but annoying the end user. Ever gotten into a fight with Microsoft Word over some formating issues? It can be dang near impossible to get it to do what you want because it is being so helpfull.
Still if ever got impletmed correctly, and wasn't annoying, it would be nice.
guvf vf zl fvt
Personally, I'm inclined to think that the most useful tools are the highly customizable ones - while I don't go near the Microsoft paperclip, tools like Outlook or procmail give me fine-tuned control over how I receive and organize my mail and that's good. So a big thing that I look for in new software is how easily I can customize it to work with me. Microsoft Word nearly always seems to work against me.
I can't imagine trying to set up rules for something that was responding to my hand and/or eye movements. They're often way too subconscious and I don't understand them well enough to formulate rules that are as useful as "Beep when I get mail from my boss." Also, what if I want to wear mirrored sunglasses while I code? (I just got a new desk at which I face big windows with no blinds - around 3 or 4 pm, I'm very tempted to put on dark glasses). Is that going to break the UI? There are also a lot of variables in a person's behavior - sitting in a different chair, not getting enough sleep, and drinking too much coffee can all change one's movements. Although I've got to say that a UI which could detect when I hadn't had enough coffee and brew me a fresh cup would be a huge improvement.
So, I think this idea is barking up the wrong tree. The things I'd rather see in a new UI paradigm are some integration of voice commands, easier methods for customization (so that it's not just for geeks anymore. Outlook's Rules Wizard is actually moderately good at this.), and an interface with some sort of ability to learn from interactions with a user (while maintaining enough consistency so the guy doesn't feel like his computer is schizophrenic).
But just think of the boon this could represent to script kiddies everwhere....
M$ AI: It looks like you are trying to crack a system, can I offer you some help from my knowledge base? There an excellent source of exploits covering most M$ products located here.
"The first time I got drunk, I got married. The second time I bought a chimpanzee, after that I stayed sober" Arian Seid
Eye trackers have been around for about two decades; the only new thing about this one is how simple it is. It depends on the fact that human eyes have quite different reflective properties in IR than the rest of the body or clothing. So they take a video frame with IR lights on, then one with it off, and subtract. Eyes show up as dots.
We have rich decision software called Bayesian Inference Software that we can build down into the system that can track your usage and adjust in an automatic fashion
As someone mentioned already, this can be used for many another thing than what this is intended.
A couple of these uses c(w)ould be:
- Permanent monitoring of the users with the camera. It can already be done right now actually, but a boss deciding to put a webcam on every machine for supervision purposes will make everyone feel 'spied'. This system would provide an 'excuse' for having webcams on every machine.
- Advertisement banners can now position themselves where you're looking.
- Since the thing would monitor the user's activities in order to determine what to give the focus to (or what to prevent being given focus to actually), it'll be easy to keep trace of the activities of the user: slashdot reading 2 hours, coding 4 hours, speaking jokes with colleages 30 minutes...
This thing really raises a couple of disturbing issues. I may be paranoid but I don't like monitoring systems. At least are they aware of that: And Horvitz and his researchers themselves acknowledge that the information collected by the notification manager software -- potentially, information on the personal activities and movements of millions of people that would be stored on the Internet -- raises privacy and security issues that have yet to be resolved. But I doubt those issues will be resolved.
Ignoring the system requirements to run the thing - that certainly aren't trivial - the system could be useful, if the user is given to set the "disturbance value" (or "worth") of possibly disturbing events. But that would be a hell to configure, imagine every morning having to say to the program "I'm waiting for urgent messages from person X and Y" and changing that everyday. I doubt very much that a program will be able to determine what I think important and what not.
A simple example of this would be that a message I receive from a colleage might very well be the information I've been waiting for 2 days but also an email to, for example, notify me of a colleague's birthday party.
Another thing I'm skeptical about in the article is: He expects that the system will be able to greet and converse with new visitors. The conversation, he says, will be on par with speaking to a person who is hard-of-hearing.
AFAIK (As Far As I Know [I realised that acronysm's meaning only very recently...]) no current software is able to converse with a human being. Answer a couple of pre defined questions maybe, but certainly not converse.
One shall speak only if what one has to say is more beautiful than silence
So now the system will actually *know* when I'm staring that stupid paperclip to death; I hope they will implement a new feature: when you stare madly at the paperclip, it will catch fire, and will be reduced to a pile of ashes! Kewl!
How to make a sig
without having an idea
So kids with Attention Deficit Disorder AND Automatic Power Management are going to have their computers shutdown on them every 10 seconds...
Seriously, this sounds neat--if it works. But I can imagine new programs trying to compete for my attention by flashing, show nudie photos, or whatever in an attempt to boost a Nielsen-style rating.
--
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
Why is MS always thinking about "how cool this ..." or "how cool that ...". Don't they realize that many of this is just straitjacketing people into one set of actions or options? Perhaps a droid might like it, but I am not a droid. I am a human being with priorities that cannot be turned into a well-ordered list.
I have read and agree to the EULA...>Click
Paperclip: "You have read nothing. No software will be installed until you've read everything, including the procedures for sacrificing the chicken. Since you lied to me you must also read all the marketing brochures for Win 2K."
Me: "Hey is that an error message?"
Paperclip: "Here, read your e-mail. No wait here's some spreadsheets. Urgent database coming up!"
The BSOD will be replaced by an urgent e-mail from your boss "get in here or you're fired" Once you left it would reboot. Paperclip: "You did not write 1200 lines of code this morning. Really, this is where you were when you left to see your boss."
This sentence no verb.
Wait a second... I wonder how much they're being paid by the ad-mongers to develop this...
you don't want it to, it will also come up WHERE you don't want it to. I can see some serious abuse of this kind of technology.
Lots of windows applications are already very presumtious about their importance in your life. Many an email package dines themselves to be of such import that, upon receiving new mail, they bring themselves to the foreground and open up a dialogue box informing you of your urgent life-or-death mail from XYZ-Spamco asking you to buy their product. Of course, the fact that you were mid-command, telneted into a unix box trying to stop circuit board etcher before it starts because you just realized there was a fault in your layout. But no, that spam mail is more important than your carefully preped sheet of copper laminated fiberglass.
Now, not only can applications bother you with this sort of thing, they can make sure the dialog box comes up right where you're looking, intentionally breaking your train of thought.
Apparently the software will even read your email and try to schedule appointments for you, etc. Of course this is somewhat beyond the state of the art - but who'd want it anyway? I haven't even let human secretaries make appointments for me; let's face it, our time is one of the few things we can't increase, and I take issue with anyone who'd (inadvertently) waste it.
Chris @ chrisworth.com
- Read fiction at www.espressostories.com
So what if I'm using Outlook? A mail comes in, I glance at Outlook--it opens the mail. It has an attachment, I look at the filename--it launches. Oh no! A virus! Don't infect Word (glance at Word). Crap! Don't send to the people in my address book (glance). Dammit!
There are times when you want to study something passively...
--
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
Novice users are already distracted from the work they're desperately trying to figure out by every blink on their DSL modems, every whirr of their hard drives, and every change in the "helpful" indicators, telling them that they're on line 8.6", no, wait, 9.4" of their document.
Now try to imagine the same people believing, thanks to a new, ritalin-demanding UI, that they're supposed to be dealing with all the random odds and ends of software and background apps [already needlessly numerous] the UI decides they've been paying attention to!
"Am I supposed to deal with the 'Task . . . scheduler' now?"
"No, you're writing an essay."
"But it came up and. . .look! The calculator just started! Oh! 'Help'! That must be useful. . ."
"It's the 'Help' function for the calculator. . ."
"I wonder if it can help me write the essay?"
Larsal [It's Worse than an Animated Einstein]
The social interface BOB, the paperclip's accursed ancestor, tried to introduce a virtual valet that did this sort of thing- and flopped utterly. Like to know what happened to the MS executive dealing with the project?
Bill Gates married her.
Chris @ chrisworth.com
- Read fiction at www.espressostories.com
and.... It sucks.
The menus in office 2000 apps already try to guess what you want to do. They are always moving menu entries arround based on what you are doing.
The end result is that you always have to read the pull-down menu -- you can never learn the position of a selection.
So by trying to anticipate what you want they make you less effecient. Wouldn't this just be more of the same?
I went by the IBM software research labs here in San Jose and got see some neat demo of exactly this (attention sensitive UI).
:)
The nice thing about this is that eye tracking is very cheap. The eye reflects IR very well so all you need is an IR strobe and a cheap IR CCD. An end product could cost less than $50.
One demo allowed you to speed up mouse click on things by automatically moving the mouse to an approximate location on the screen where you are looking.
They had one demo that would track your eye and blur the screen except for where the eye was focused. Everyone else sees a blurry screen, but you (the person being tracked) can't see a difference. Could be very cool in 3d games if the game could render the areas of the screen you were looking at in more detail and those you weren't in less detail. The military has been experimenting with this on high-end flight sims that do this with good success. But if your playing on a 13" monitor then pretty much everything is in focus.
Checkout their project page for a little for info.
http://www.almaden.ibm.com/cs/blueeyes/
--
-- Virtual Windows Project
The problem with attempts at "smart" behavior in software is that its behavior becomes less predictable. I think this is an attempt to patch up previous attempts at smart behavior -- like the paper clip's interruptions based on smart analysis of what I'm doing. I see why they're trying to fix it, but now if I look away from the computer for a second, its state will be different when I look back. I can no longer easily control what input I give the computer; testers can't reproduce bugs because they don't remember exactly when they looked away from the screen; the msdn knowledge base fills up with bugs related to what noises and body movements you shouldn't make while FoxPro is saving a large file. Awareness of a user's drifting attention levels could be interesting in a game or something, but I don't want it in the OS until it's *really* mature.
This is nothing more than Bob with hardware. It's that idiot paperclip on steroids. It's just one more way that Microsoft insults the intelligence of their customers. I don't need an idiot paperclip popping up to tell me that it "looks like" I'm writing a letter. I know whether I'm writing a letter, thanks. If I were functionally illiterate then perhaps it might be useful to have dopey software "help" me write a letter. But Microsoft makes the blanket assumption that ALL its customers are functionally illiterate.
I certainly don't need my computer deciding for me whether I should be notified that I have mail. I think I'm capable of making that decision for myself.
No thanks, Mr. "Chief Software Architect" Gates.
Two weeks of running Windows 2000. Two Blue Screens of Death. Thank goodness for Linux!
The problem with a lot of the software I see around today is that in the desire to make software more open and friendly, it has got a lot more distracting to use. It's difficult to Zen-out when using a piece of software when every minor adjustment triggers an animated effect, be it a spinning hour glass, back illuminated button or piece of paper flying across the screen. In an attempt to give the user more feedback about what is active and what is not, software designers have taken away the "quiet" interface and have jazzed it up.
And this has not been restricted to just the application itself. The applications often demand attention like some spoilt brat - the "HELLO? YOU HAVE MAIL!!!" syndrome. While in some cases, such as Lotus Notes, the default is to rise to the top of the window stack and bang a modal window up to get your input everytime there is new mail, you can tone this down to an audible bell only. Or ICQ clients which reappear on the top at a new message coming in. And there are others - visual alarms on calendaring tools and probably more that I have forgotten.
When I have the option, these programs are pushed into the bit bucket as fast as possible. Using them is a dire waste of productivity. Where there is no choice about using that software, I try and tone down the alarms to be just audible effects which I can acknowledge without having to press a key, move the mouse or otherwise stir from whatever I'm doing.
So really, this research sounds like a patch for the problem, rather than a cure. The problem is with the UI design - programs are increasingly "rude" in their attempts to get attention. At least if I hold the source, annoying habits in essential software can be trimmed to a minimum. But rarely in the Unix side of the world do I have to worry about annoying software - 95% of the stuff which irks me is Windows-ware. Maybe the art of Zen is dead on the MS platform...
Cheers,
Toby Haynes
Anything I post is strictly my own thoughts and doesn't necessarily have anything to do with the opinions of IBM.