Researchers Track Mouse Movements and Hesitations
lpctstr writes "Researchers from the University of Washington and Microsoft Research have found that cursor movements and cursor hovers can detect the relevance of a search result and whether a user may abandon the search. They use an efficient algorithm written in Javascript to silently record movements and clicks on Bing and find that computing relevance using movements + clicks works better than just clicks (the current state-of-the-art). They explain some of this due to cursor and gaze being closely aligned on the web, and especially so on search result pages. Is this the future of innovation in search ranking — Google and Bing tracking your every twitch and pause?"
Asians or Latinas, it's so hard to choose!
How would this work for people like me who don't move their mouse unless there's something they've decided to click on?
My mouse is not an extension of my eyes. It's a tool I use if and only if there's a task to be accomplished with it.
I hope Google takes the high ground here and -doesn't- try to track this -- Otherwise, I shall be forced to use Greasemonkey to block it. And it won't be the only thing I'll block if it comes to that.
They explain some of this due to [mouse] cursor and gaze being closely aligned on the web, and especially so on search result pages. Is this the future of innovation in search ranking — Google and Bing tracking your every twitch and pause?"
...Just in time for Web use to go mobile and touch-based.
Microsoft is going to patent hesitation
Open Source Network Inventory for the masses! Kuwaiba
I would subpoena all search engines in advance to provide all "hesitations on mouse movements" of their users.
Based on that, it would be possible to track potentially dangerous net users with bad intentions and immediately storm in their houses.
This is news?
Even my university notes make reference to using mouse gestures or gaze tracking to determine which search results are relevant, whether the user is hesitating or uninterested in the paragraph you're offering him.
How is this news?
They've already started doing a very hacky thing to their search results in order to monitor us. The links are changing to a redirect url when clicked. Had to use YesScript to block it.
Shame the idea won't last long with the proliferation of touch screen devices where you get no cursor position feedback at all. Maybe they can look at this again when we are able to upgrade eyeballs to a version that supports javascript!
I already tend to shy away from web pages that use mouseovers; nothing more annoying than having stuff pop up in your face when you're essentially just scanning the content. If I know they're actively tracking this trivia, I WILL find a way to block or spoof it.
the method will make spamming and gaming the system easier. well, not easier, but opens up a new route.
and of course, throw this out of the window on touchscreens.
world was created 5 seconds before this post as it is.
Is suddenly not a cult thing any more, but a quiet protest...
!
Indeed
I do not believe in karma. "Funny"=-6. Do good and forbid evil. Yours, Oft-Offtopic Flamebaiting Troll.
And they could also extend it with facial recognition with your webcam / phone camera, and why not also record sound, in case you're always complaining loudly when getting bad results,....
Exactly. I typically stick with the keyboard: either page up/down, or cursor up/down. I sometimes use the mouse to adjust the scrollbar, but seldom move within the window until I'm ready to click a link.
Luke, help me take this mask off
So I walk away to, oh, I don't know take a piss or something, and when I come back wherever I bumped my mouse getting up is the most relevant thing in my search? Riiight.
From TFA:
They use an efficient algorithm written in Javascript.
Is it me or they are using two incompatible words in the same phrase?
Thought I'ld comment on "It's not a cursor, it's called a pointer." but then WikiPedia says that MS tends to use carets to call cursors and cursors as pointers, and that some people use text cursor and mouse pointer to call the two respectively, to avoid ambiguities. So, that is new to me, as I've always called the blinking vertical line where text will be entered as cursor, and the arrow-like thing that moves when you move the mouse as pointer. Which is also what I teach my students. Now, I question why they would call it WIMP and not WIMC...
5ryn
I hate javascript. I knew from the very start it would be a source of abuse more than use. I knew it and avoided using it in my code whenever possible.
I turn it off at every chance and only allow it sparingly on some sites.
and now they want to meta-watch us. just wonderful. as if working around the 'helpful autocompletes' all over the place (I'm talking to you, google) isn't enough hassle to get the data we want.
typing into text boxes is a huge pita. SO much other processing goes not (even if you disable spellchecking) that I lose characters, find repeated characters and the typing lag is WORSE than it was 20 yrs ago with an 8 or 16bit cpu and less than a meg of ram. I'm not kidding - the more js that runs 'in the background' the laggier shit gets as you type. my 3ghz dualcore 'drops characters' like its no one business; and I know that I'm not alone since I see so many posts on so many forums with dropped and repeated chars. the web SHIT has taken over our computers and only gives us tiny slices of time to do OUR work in.
the solution is to go back to a batch oriented web again, for at least some things. I do NOT want 'journalling - saving!' happening WHILE I am typing in a text box. the only thing happening should be cursor moves, chars entered and cursor blinking. while in emacs or vi, my text speed is very fast; so why is it that web-based text areas are SLOW AS SHIT ?
its the javascript. the language for advertisers and webfuckers (what I call webmasters who fuck wtih your browser, thinking that THEY own the formating and content display on your system and not you).
do a view source and see all the crud that comes thru. how much is really need to query and display results? I could do that in simple forms/cgi's and bypass all the crud.
so, we need more 'submit' style front ends that sit there and do NOTHING until you hit submit. no animation, no character counting, no 'journalling - saved!' bullshit. no copying of my data to you 'in case'. just fucking sit there, take my text and when *I* hit submit, THEN you can bulk upload it to the main server.
javascript annoys the hell out of me. it has ruined what was once a nice responsive web. now, I drop characters as the background jscript tasks own more cpu than the foreground ones do. ;(
--
"It is now safe to switch off your computer."
I find I am using the search engines as a spell-checker. It is pretty handy to start typing a word and have it present the correct spelling. In these cases I never had any intention on clicking anything.
This issue is a bit more complicated than you think.
Queue the automated search that sends back fake interaction data.
no-flash, no-java allowed on search page including google, bing or whatever. Thanks for the head up.
Yup, device equipped with touch screens are rather different beasts than desktops.
And browsing happens to be among the top activities for which tablets are designed. Keyboard-less, light devices are better suited to consume content, big machines with lots of inputs are better suited to create content.
Lots of browsing will occur on the current and future generation of tablets, and that's what is going to get mined through newer techniques (webcam secretely analysing gaze direction ?!?)
Meanwhile we will continue to use desktops and big laptops to write code (we /.ers) or write TPS reports (the non-/. crowd).
"Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
I have Tourettes Syndrome (srsly), and have a habit of randomly clicking my mouse buttons on empty areas of the screen while reading. Let's see you read into THAT, bitches!
What Microsoft REALLY needs to do a better job of is tracking mouse movement (specifically, acceleration, ballistics, etc) to do a better job of discerning intent when you go to grab something... and a better job of adaptively figuring out over time whether its assumptions about your intent are right or wrong. It really seems like every new version of WIndows leaves me fighting and frustrated with it a tiny bit more.
Here's a real-world concrete example. Suppose the mouse pointer is approaching the right edge of a window that's maximized to the left panel of a multi-monitor setup. The mouse pointer slows down, and seems to also be approaching the scrollbar. The left button gets pressed, and the mouse moves in a direction that's mostly upwards. Well, except the pointer overshot the edge a bit, and the left click technically occurred 2 pixels into the window on the monitor adjacent to the right. Taken in isolation, Windows has no real choice but to assume the user meant to click the pixel on the other monitor even though it contextually makes no sense. But combined with the observed ballistics (slowing down, slight arc towards the scrollbar, motion after left-click that makes more sense as a scroll-gesture than a... well... meaningless gesture), it's obvious what the user meant to click. And for the most part, Windows, seems to be completely oblivious to it.
Now, for a counter-example: trying to select text without adjacent whitespace. For me, Windows (Word and Outlook in particular) NEVER seems to get this right. I'll click at the right starting point, letting go and starting over if I'm not happy with it. Then I'll start highlighting. But way too often, it'll stop selecting a character or two short of where I want. If I keep moving the mouse, it'll grudgingly select the remaining characters... but feels compelled to ALSO ignore my hard initial-selection work and expand the other end of the selection too. Dammit. I'll then spend the next 10 seconds fighting with it trying to select the text I REALLY want. Half the time, I'm forced to give up, let it select the damn adjacent whitespace, and edit it away after I paste. It annoys me to no end.
In the end, it feels like Windows has simultaneously gotten worse in two directions. It forces its opinion on me without learning from its mistakes or giving me the option to beat it into submission so it quits interfering, and simultaneously forces selection with almost single-pixel precision to make increasingly-dense window gadgets work. I'll admit that Java is even worse in this regard, and Linux (or at least Gnome/Compiz) doesn't seem to be any better, but it's still annoying as hell.
ClickTale is already doing this sort of mouse movement tracking as part of its web analytics service.
... when I switch my mouse over to my left hand? That's the best indication that I've found an interesting page I can think of.
Have gnu, will travel.
Fine by me. My mouse cursor resides in white space until I'm ready to click. Nervous habit.
Leela: "Is all the work done by children?" Alien: "No, not the whipping."
Microsoft Research is found to be copying Google.
Roughly a year or two ago Google made it known they've long been searching the subject and have long had plans to bring this to market.
Suddenly both Apple and Microsoft are constantly trailing Google. You can say what you want about the three companies, but clearly both Apple and Microsoft have fallen behind Google as so far as real tech companies go. I'll leave it as a reader exercise to determine if its Apple or Microsoft which claims position two or three.
I prefer a computer to do what I want to do. However, given that the computer cannot read my mind, it will probably do what it THINKS I want to do...
...before they start building heartbeat and moisture sensors into the mouse so they can gauge emotional responses to what's on the screen. Then they can really target specific demographics. Or at least sell the information so others can.
thought i should post this
http://tech.slashdot.org/story/10/07/27/1624251/Google-Nabs-Patent-To-Monitor-Your-Cursor-Movement
http://slashdot.org/comments.pl?sid=1734136&cid=33049918
...before they start building heartbeat and moisture sensors into the mouse so they can gauge emotional responses to what's on the screen. Then they can really target specific demographics. Or at least sell the information so others can. (The AC post before was mine...I wasn't logged in)
Loading...
Researcher 1: Why does his mouse cursor keep going up, then down, then up, then down...
Researcher 2: Look at his search parameters. He's masturbating.
Researcher 1: Gross.
What surprised me more about this story is the fact that they found enough Bing users to get a decent dataset for their research.
~Syberz
If they got feedback that the first time it was this slow, but then subsequently it took 2 seconds every time, great - we can chock it up to learning curve. However, I suspect they would find it consistently takes longer to poke around the ribbon bar and would come to the conclusion it sucks and we could go back to the perfectly useful menu system.
Conversely, I watch my 3 year old nephew pick up an iphone, swipe through two pages of icons, pick the one he wants, and start playing the game. No pauses, no hesitation. That seems to me they captured how (at least a 3 year old) intuitively works.
I do focus groups all the time, but this only lets me capture users at a specific point in time. If we could get this level of feedback from a built in usage library (without affecting performance), I think we could all do a better job of UI design that is intuitive.
The sun is the same in a relative way, but you are shorter of breath and one day closer to death
LOL While reading this my cursor was on the story 2 above this one, does that mean I was reading it? :-)
Kosh: "Understanding is a 3 edged sword, your side, their side, the Truth."
but feels compelled to ALSO ignore my hard initial-selection work and expand the other end of the selection too.
Yeah, I remember that shit too, before the ribbon made me give up Office completely. There's actually an option somewhere called 'intelligent selection' or some stupid crap like that. Remove it and it will select from where you clic.
Non-Linux Penguins ?
Do you have a smart phone? You'd love Swype. It's basically what you describe but for the soft-keyboard on a touchscreen. You put your finger down somewhere near the letter your word starts with, drag it around in the general direction and neighborhood of the rest of the letters in the word, and then lift it up. Swype calculates a probability of what word you were trying to type based on the shape, even if you didn't quite touch all the letters in the word. If it's confident enough, it inserts the word into the text field... otherwise it displays a list of words in order of confidence for you to pick from. (If you wanted the top word in the list you can just start typing your next word and it will insert it.) And of course it automatically learns any new words you type.
At first I thought it was just a gimmick but I've been VERY impressed by it on my Galaxy S phone... I specifically got a phone with a slide-out keyboard because I didn't like the idea of a soft-keyboard, but now I use both in different situations. The hardware keyboard is still nicer for entering symbols and other non-words conveniently (e.g. an SSH session or a web password), but for regular writing Swype is actually faster!
Sometimes I wonder.
The mouse movement tracking has been in the news recently, see: http://www.newscientist.com/article/dn19156-innovation-shrewd-search-engines-know-what-you-want.html
This would be scary.. if it wasn't potentially true.
How long, exactly, until major websites "require" for a flash / silverlight bug to play.. the sole purpose of which is to enable the camera to track your eyeballs. Resolution? Not that great... but matched with mouse movements and matched up against clicks.. it could well be data worth having.
Of course, we all use flashblock, so it won't be a problem for us. Right?
You have a sick, twisted mind. Please subscribe me to your newsletter.
If I understand correctly, YOU are talking about common sense and a bit of respect for users... from Microsoft.. ;)
Sorry Sir we are the usability authority not you, If we say you can press shift and select with the arrow keys you better..
I've been working in usability for several years and have noticed that gaze and pointer movement are usually aligned. This has helped me limit the way hover is used as an event in interfaces to let users get to deeper levels of information in web applications. To illustrate: When Google implemented hover on their image searches to reveal intermediately larger thumbs in search listings, I thought they'd made a mistake; it's largely irritating to have larger images appearing and covering other things one is still viewing as potential targets, as one may not want that information at that point. Users may be simply browsing with their eye and pointer together and pre-empting a click on any one of a number of potential targets.