Slashdot Mirror
Opera Adds Gesture Navigation
Trepidity writes "The Opera web browser appears to be the first to add gesture-based navigation (made popular recently in the game Black&White) as a standard feature. You can perform a bunch of common actions with simple gestures, such as holding down the right mouse button moving left and releasing to go back, or moving up then down while holding the button to reload the current page. A list of the various implemented commands can be found on their site." I've been playing a fair amount of B&W lately - the interface took a bit to learn, but once you['ve got it done, it's actually a very efficent system of getting around - the use within the Web might finally take the Web beyond just a point and click interface. Maybe. Probably not. CT: Just don't try it with a thinkpad style nipple mouse. My wrist lost feeling. Update: 04/18 02:55 PM by T : Read more below for a software project that promises to spread some gestural goodness even further.
Mike Bennett writes with news of his "free software project. It's called wayV, and provides gesture recognition for X. Version 0.1 was released a while ago and let you start applications with gestures, version 0.2 will be released this week and also includes the ability to send keypress, e.g. make a gesture to change desktops, etc." This looks like a modestly conservative 0.1, too;)
I remember back when gesturing the right way to get a computer to do something was considered a joke. You only had to stand on one foot and wave the right way when Win 3.1 wouldn't boot. The "hold button down and slide left" gesture was your way of fingering the computer. On a good day "moving up and then down while holding the button" was a high five. Then again why else would we be in a recession if 1001 things to do with E-Commerce sites wasn't the invention of the day?
No, it's not new at all. In fact, it's been used all over the place in the gaming world for some time. Good examples are Hybris and Battle Squadron, two Amiga games that used circular motions with the joystick to launch a smart bomb, or separate the wings from the ship. Of course, that was back in the days when gameplay was important, so making the user interface efficient was more of a priority than it is in today's world of flashy graphics and sound but no gameplay.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
...and the new patent on "single gesture response to Amazon's patents"...
"A gesture in which the third digit of either hand is extended while the rest remain closed, and the hand then placed in such a manner that the back of the hand faces towards the recipient, indicating displeasure"
=)
Ok how many weeks till we read a story on slashdot along those lines:
"Foobar Inc. has threatedned to sue Opera because Opera browser is using their patented NaturalNav(tm) technology without permission."
Will all the patent insanities happening lately, this will not surprise me.
There's enough of a learning curve involved with sedond and third mouse buttons and shift- and control-click operations to where much of it is used only by "power users". This is interesting, again, for power users. But honestly, the mouse is only an "intuitive" tool when it has one button and no chording or other such behavior modifiers.
Mice have been in the mainstream since 1984. That's 17 years. It's shocking how little innovation there has been in interface design since the Apple Lisa. We're still dealing with mice, overlapping windows with raise/close widgets, modal dialogs and trashcan icons. I look back and remember that Radio Shack had rudimentary speech recogniton and speech synthesis peripherals available in 1981. Mass-produced flat touchscreens go back at least to the GRiDpad. Moving pointers with eye movements was done years ago.
Technologies like handwriting and speech recognition made strides right up to 1984 or so, and then stagnated until a few years ago. Human-language command parsing was coming along well, too, in the mid-1980s and has barely been heard from since. The WIMP (Windows, Icons, Mouse, Pointer) UI killed off interest in a lot of things.
Things like this gesture stuff--wildly non-intuitive extensions to mouse functionality--are indicative of the myopia and stagnation among interface designers. Mouse "gestures" will be useful to small, technical audiences like engineers, providing a shorthand for dealing with complex visulaization applications. But it's much too reliant on training and so completely non-intuitive that it can never be a viable direction for mainstream UIs.
Why, in the year 2001, some 14 years after GRiD's touchscreen-and-stylus tablets, can't people reading a document onscreen simply touch the corner of a doument and flick it to the left to turn a page? Why can't I pick up a pen and scribble recognizable corrections directly on a spreadsheet? Why, if I do want to use speech dictation software, can't I make corrections with a pen at the same time using standard proofreading marks? And why isn't my speech recognition profile stored on a central sever or on a pocket-sized dongle so I can dictate text from any computer or telephone anywhere? Why are keyboards still being used by people besides programmers, paralegals and data-entry clerks?
Why do command-line OSes not offer plain-English (or French or Mandarin) command recognition? Surely if the parser used in Zork worked as well as it did on a lowly 6502 back in 1981, and I used BBSes with plain-English command recognition ("Go to the library and download 'bluebox.txt'") in 1985, you'd think natural-language access to basic (and not-so-basic) file and system management operations, among other things, would be a piece of cake by now.
This isn't innovation. It's just sort of sad.
It'd be great if Lionhead released their source, including the gesture recognition. But why wait, then there's...
LibStroke - a stroke translation library
Implemented in C, and with a transliterated Java version included as well.
strokes-mode.el - a strokes recognition minor-mode for emacs
Go easier on your wrists, take a break from C-M-A-|, and make vague mouse wavings at emacs to make it do your bidding.
IMHO, the algorithm used in strokes-mode seems much nicer than that in Black & White, or even libstroke. It could be just a matter of parameters, since for all I know B&W and libstroke could use pretty much the same algorithm as strokes-mode.el.
I'm already looking at tweaking the Java libstroke class to play around with it in a few Java apps I'm poking at.
Hemos...just yesterday there was this post:
:)
Posted by Hemos on Tuesday April 17, @11:29AM
from the i-wish-this-was-more-like-this dept. "It's a Windows app, so I'm not able to run it. Neither do I have a CueCat -- but apps like this make me smile. "
With regards to the cuehack story. Now you tell us "I've been playing a fair amount of B&W lately "....
Ok....are you playing it on Linux or do you just talk out your butt alot? I am not trolling here...I am serious! Don't give us the "I can't run a windows app" crap anymore...we know you have a windows machine in there.
... on one gesture shopping.
Oops. My mouse slipped.
There was a windows program called Pointix that allowed you to navigate the web with what they called "glicks". A counter-clockwise circular motion would mean "back" for instance. I forget what it was called, but I was using it in, well, late 1998. It had lots of different gestures that were all programmable.
The web site and domain, pointix.com, are junk now, but this was a really cool program. Along with GetRight it's one of the two programs I ever registered.
B&W doesn't have gesture based navigation. You gesture for common commands (pick up leash, drop leash, change leash, perform miracle, perform special move). Which is really more what Opera appears to be doing - the most common browser commands happen to be navigation.
PalmOS did this earlier, the most common PDA commands are "input character", "delete", "select", and "scroll". In turn, this comes from writing and proofer's marks (you know, omit, insert, new paragraph. These have a proper name?), which is just recording gestures (the written alphabet, the marks) in pen and ink for later use. You might argue that drag-and-drop, especially in the context of cut-and-paste, is gestural input.
I'm not discounting the usefulness, or the novelty of incorporating gestures into a standard WIMP interface application. I'm just putting a little perspective on the above quote.
I think you're onto something here - why not go all the way - take that "natural" (split) keyboard and turn it into two mice, each with 50-odd buttons?
Wanna keep it still and type? Lean your wrist back a notch. Wanna move around, lean your wrist forward. (Or vice versa, some ergonomics expert has no doubt figured this out :)
If the problem is that users don't want to switch between a movement controller with no buttons and a keyboard with lots of buttons, perhaps the solution is probably to move the buttons to the movement-controller, not, as has been done up until now, the other way around.
I use gestures to start X applications.
See http://freshmeat.net/projects/wayv/
Anyone who's played Black and White will know that gesture navigation is actually a major pain in the butt to use. Sure it's novel, but frantically waving the mouse around for several seconds in some pattern does not make for a quick and useful interface, especially when it fails a good portion of the time.
Shouldn't there be a sort of consistency among the different applications used within one GUI. Before you know it, we are back in the 'goold' old DOS day, where every application was setting it's own standards (remember WordPerfect?). These gestures would only be nice if they were an integral part of the GUI, and thus useable for all applications.
Actually, this concept is really not that new. Check out the Strokes extension in Emacs and XEmacs. It does this very thing. You can teach it various patterns, and then you can use those patterns to execute commands. Heck, now that I think about it, you could integrate Strokes and Emacs W3 to do exactly what Opera is doing...
This is the (real) innovation we've been waiting for. It's just sad that it came out of a game house instead of application programmers.
Think about it: The one reason that many people think that command prompt shells are superior to 'gruntnclick' is that the ability to use written language and commands is infinitely more flexible than typical WIMP operations. Despite the fact that it's slower than Grandma before she's had her prunes, most of the developers I know eventually drop down to csh or bash to get 'any real work' done.
Gesture systems, provided in combo with typical WIMP operations, have the potential to change that. If there is a gesture for every non-destructive command, and gestures can be stacked so that you can direct the output of one gesture command into another, you've created a truly flexible and intuitive command interface.
I've been playing B&W since it came out, and in only a couple weeks, I can shoot fireballs and sheild spells around like no one's business. I suspect that this will be true for a great majority of computer users. Not all, but enough to make the project worth it.
Zoom into an image in photoshop, select a square capture to clipboard, paste into new image. If I can do that with a few gestures rather than 8 different menu commands, I will have sped up my image processing dramatically without having to write a complex script or plugin to do it for me.
Now Lionhead has talked a little bit about releasing their source code if the game becomes popular enough. What I would like to see is source for their gesture recognition systems so that it can be integrated into KDE and Gnome, and OS plugins for Win32 and MacOS. With the level of interest in this new system, that may not even be necessary.
This *will* work. Get behind it, guys!
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
villager: "We need more poooorn."
whisper: "Pooooooooooooorrrrrrnnn"
--