Slashdot Mirror
Opera Adds Gesture Navigation
Trepidity writes "The Opera web browser appears to be the first to add gesture-based navigation (made popular recently in the game Black&White) as a standard feature. You can perform a bunch of common actions with simple gestures, such as holding down the right mouse button moving left and releasing to go back, or moving up then down while holding the button to reload the current page. A list of the various implemented commands can be found on their site." I've been playing a fair amount of B&W lately - the interface took a bit to learn, but once you['ve got it done, it's actually a very efficent system of getting around - the use within the Web might finally take the Web beyond just a point and click interface. Maybe. Probably not. CT: Just don't try it with a thinkpad style nipple mouse. My wrist lost feeling. Update: 04/18 02:55 PM by T : Read more below for a software project that promises to spread some gestural goodness even further.
Mike Bennett writes with news of his "free software project. It's called wayV, and provides gesture recognition for X. Version 0.1 was released a while ago and let you start applications with gestures, version 0.2 will be released this week and also includes the ability to send keypress, e.g. make a gesture to change desktops, etc." This looks like a modestly conservative 0.1, too;)
I've filed an RFE bug report for mouse gestures in Mozilla. If you'd like to see mouse gestures in Mozilla, please vote for this bug. Of course, you need a Mozilla account to vote on bugs, but you can easily create an account if you don't have one.
Alex Bischoff
---
Alex Bischoff
HTML/CSS coder for hire
I remember back when gesturing the right way to get a computer to do something was considered a joke. You only had to stand on one foot and wave the right way when Win 3.1 wouldn't boot. The "hold button down and slide left" gesture was your way of fingering the computer. On a good day "moving up and then down while holding the button" was a high five. Then again why else would we be in a recession if 1001 things to do with E-Commerce sites wasn't the invention of the day?
Does Opera crash as frequently as B&W?
Do your window decorations morph to reflect the kind of sites you visit?
--
No, it's not new at all. In fact, it's been used all over the place in the gaming world for some time. Good examples are Hybris and Battle Squadron, two Amiga games that used circular motions with the joystick to launch a smart bomb, or separate the wings from the ship. Of course, that was back in the days when gameplay was important, so making the user interface efficient was more of a priority than it is in today's world of flashy graphics and sound but no gameplay.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
...and the new patent on "single gesture response to Amazon's patents"...
"A gesture in which the third digit of either hand is extended while the rest remain closed, and the hand then placed in such a manner that the back of the hand faces towards the recipient, indicating displeasure"
=)
I remember gestural interfaces having a brief heyday of popularity on the Mac back around 1988-89 when Hypercard came out. They made alot of sense at the time, and I'm not sure why they never caught on with the desktop computing crowd. Other posters are astute in noting that any decent pen based interface such as PalmOS or NewtonOS has relied on gestures for along time now. I definately think NewtonOS had made excellent use of gestures, such as scrubs to erase text.
right. what those games needed was a way to issue complex commands quickly, without taking your hands off the input device (or eyes away from the screen).
and for the very same reason, gestures have become an integral part of Alias|Wavefront's Maya (the successor of the infamous PowerAnimator). the gestures take a bit to learn, but once you've got them wired you'll never look back. i know a few artists who have completely removed all of Maya's menu bars and buttons - no need to clutter your screen real estate when everything is available by hitting blank and a few strokes with your pen/mouse. needless to say, their productivity has improved dramatically, and they keep complaining about having to use 'traditional' menus in all those other applications.
Ok how many weeks till we read a story on slashdot along those lines:
"Foobar Inc. has threatedned to sue Opera because Opera browser is using their patented NaturalNav(tm) technology without permission."
Will all the patent insanities happening lately, this will not surprise me.
I wonder if Microsoft had an emergency meeting yet to try to get this into the next version of Windows so they can claim to be innovative.
They probably decided against it because it would be too confusing to the average Windows user.
Sounds like a good opportunity for Apple to innovative too. Increase that market share from 0.1% to 0.2%.
...but its not something I could ever get on with. I always found it quicker to hit the G key for grabbing than I did moving the mouse around in some arcane squiggle. Still, I guess it has its uses.
"And people are just figgering out that this widget is in there?"
Ah, no. I've been using it since it first appeared in the beta releases several months ago.
It's just *Hemos* that's finally figured out that it's there.
For me, I find it frustrating using other applications. I keep expecting to use gestures in wordprocessing...
--
--
Don't like it? Respond with words, not karma.
There's enough of a learning curve involved with sedond and third mouse buttons and shift- and control-click operations to where much of it is used only by "power users". This is interesting, again, for power users. But honestly, the mouse is only an "intuitive" tool when it has one button and no chording or other such behavior modifiers.
Mice have been in the mainstream since 1984. That's 17 years. It's shocking how little innovation there has been in interface design since the Apple Lisa. We're still dealing with mice, overlapping windows with raise/close widgets, modal dialogs and trashcan icons. I look back and remember that Radio Shack had rudimentary speech recogniton and speech synthesis peripherals available in 1981. Mass-produced flat touchscreens go back at least to the GRiDpad. Moving pointers with eye movements was done years ago.
Technologies like handwriting and speech recognition made strides right up to 1984 or so, and then stagnated until a few years ago. Human-language command parsing was coming along well, too, in the mid-1980s and has barely been heard from since. The WIMP (Windows, Icons, Mouse, Pointer) UI killed off interest in a lot of things.
Things like this gesture stuff--wildly non-intuitive extensions to mouse functionality--are indicative of the myopia and stagnation among interface designers. Mouse "gestures" will be useful to small, technical audiences like engineers, providing a shorthand for dealing with complex visulaization applications. But it's much too reliant on training and so completely non-intuitive that it can never be a viable direction for mainstream UIs.
Why, in the year 2001, some 14 years after GRiD's touchscreen-and-stylus tablets, can't people reading a document onscreen simply touch the corner of a doument and flick it to the left to turn a page? Why can't I pick up a pen and scribble recognizable corrections directly on a spreadsheet? Why, if I do want to use speech dictation software, can't I make corrections with a pen at the same time using standard proofreading marks? And why isn't my speech recognition profile stored on a central sever or on a pocket-sized dongle so I can dictate text from any computer or telephone anywhere? Why are keyboards still being used by people besides programmers, paralegals and data-entry clerks?
Why do command-line OSes not offer plain-English (or French or Mandarin) command recognition? Surely if the parser used in Zork worked as well as it did on a lowly 6502 back in 1981, and I used BBSes with plain-English command recognition ("Go to the library and download 'bluebox.txt'") in 1985, you'd think natural-language access to basic (and not-so-basic) file and system management operations, among other things, would be a piece of cake by now.
This isn't innovation. It's just sort of sad.
It'd be great if Lionhead released their source, including the gesture recognition. But why wait, then there's...
LibStroke - a stroke translation library
Implemented in C, and with a transliterated Java version included as well.
strokes-mode.el - a strokes recognition minor-mode for emacs
Go easier on your wrists, take a break from C-M-A-|, and make vague mouse wavings at emacs to make it do your bidding.
IMHO, the algorithm used in strokes-mode seems much nicer than that in Black & White, or even libstroke. It could be just a matter of parameters, since for all I know B&W and libstroke could use pretty much the same algorithm as strokes-mode.el.
I'm already looking at tweaking the Java libstroke class to play around with it in a few Java apps I'm poking at.
I'll bet this is so they can sell the browser to webpad manufacturers, especially those who aren't going to put Windows on them.
I seem to remember news of a webpad that was going to use BeOS, for example...
I bet if you search through the /. archives for "popmouse" you'll find a few posts by me...
--
It's nice to have multiple independant systems, however. If one could do everything with ..either.. the keyboard or the mouse, then when things were working well, one would have options, and when something was broken, one would have possibilities.
Of course, there's always the pickaxe through the screen...
But, within reasonable limits, duplications are useful.
Caution: Now approaching the (technological) singularity.
I think we've pushed this "anyone can grow up to be president" thing too far.
No, that's Up, Down, Left, Right, A, B, Start.
:)
-- Veni, vidi, dormivi
We used an interface like this for VLSI circuit design back when I was at Purdue. The package was called Mentor Graphics, and their term for this type of interface was "strokes". You could even add your own, and it was extremely intuitive after you spent a half-hour fiddling with it. It saved a TON of time-- the interface to Mentor is pretty heavy without it. Lots of deep, nonintuitive menus.
Although I haven't played B&W, we used a gesture interface in the Mentor Graphics circuit design package at Purdue. The problem you mention with mouse sensitivity (and dodgy mice, too) was a serious problem in the labs, since those machines took a lot of abuse, and had huge monitors with tiny little arcane menus. But the gestures are size-independent and work anywhere on the screen, and so as long as you make approximately the same shape the gesture works. This is an excellent alternative to menus when the menus are a pain in the butt.
Hemos...just yesterday there was this post:
:)
Posted by Hemos on Tuesday April 17, @11:29AM
from the i-wish-this-was-more-like-this dept. "It's a Windows app, so I'm not able to run it. Neither do I have a CueCat -- but apps like this make me smile. "
With regards to the cuehack story. Now you tell us "I've been playing a fair amount of B&W lately "....
Ok....are you playing it on Linux or do you just talk out your butt alot? I am not trolling here...I am serious! Don't give us the "I can't run a windows app" crap anymore...we know you have a windows machine in there.
--
--
Mod up a post Rob doesn't like and you'll never mod again
... on one gesture shopping.
Oops. My mouse slipped.
There was a windows program called Pointix that allowed you to navigate the web with what they called "glicks". A counter-clockwise circular motion would mean "back" for instance. I forget what it was called, but I was using it in, well, late 1998. It had lots of different gestures that were all programmable.
The web site and domain, pointix.com, are junk now, but this was a really cool program. Along with GetRight it's one of the two programs I ever registered.
Boffoonery - downloadable Comedy Benefit for Bletchley Park
B&W doesn't have gesture based navigation. You gesture for common commands (pick up leash, drop leash, change leash, perform miracle, perform special move). Which is really more what Opera appears to be doing - the most common browser commands happen to be navigation.
PalmOS did this earlier, the most common PDA commands are "input character", "delete", "select", and "scroll". In turn, this comes from writing and proofer's marks (you know, omit, insert, new paragraph. These have a proper name?), which is just recording gestures (the written alphabet, the marks) in pen and ink for later use. You might argue that drag-and-drop, especially in the context of cut-and-paste, is gestural input.
I'm not discounting the usefulness, or the novelty of incorporating gestures into a standard WIMP interface application. I'm just putting a little perspective on the above quote.
The software we (http://www.5dt.com/) used to distribute with our Data Glove product had a demo that mapped gestures to actions way back in 1995 already. Some of our applications use it too, not unlike the way it appears in B&W. Our current glove driver has some simple gesture recognition built into it. Granted, this is "real" hand gestures, not gestures using the mouse, so it probably isn't quite the same thing.
-----
Just add voice recognition, and then you can move on to grunt navigation. It has taken years, but now my wife know that when she is helping me work on the car, "unhh" means "hand me that rachet". "SHIT" means "get me a bandaid".
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
Mentor Graphics had this same thing a long time ago - you could use various strokes (I believe that is what they were called) to perform various actions in their VLSI program. Most people I knew relied on this heavily - once you got the hang of it, it was very efficient.
Someone I knew at Purdue had started incorporating something like this in FVWM, I believe, but I don't know how far he got (this was several years ago)
Hmmm... I'd like to see some features like this incorporated in KDE/GNOME (maybe even Windows XP.. er, scratch that thought).
This must be how Dust Puppy navigates the net.
Best Slashdot Co
Yes, and if it has lots of "option" keys, say 3 called "control", "meta" and "shift", we could get hundreds of commands with just 3 or 4 keys held down simultaneously. Commands like "control+meta+shift+!". And we could call the editor "editor macros" or "emacs" for short.
Best Slashdot Co
Great, now I can do Tai Chi and use my computer at the same time.
It's good to see people coming up with ways to allow mouse users to cahieve their goals more quickly.
I wonder though whether it will end here. We already have 3 button mice, 5 buttin mice (yes, the "wheel" is actually buttons 4 & 5) Adding any more buttons would have been silly to operate with one hand, so they invented "gesture" control.
Maybe the next step could be to make it two handed. Give it a more rigid design (say a static board shaped design) and that would allow the hands to more more independently of it thus making the addition of many more buttons a better option.
They could call it a "button board" or even a "keyboard" that would be cool, and allow VERY fast control of apps once learned. Now if only all the applications were optimised for this kind of input. Maybe we could have an editor that was optimised for this so called "keyboard"! we could name it "vi"
-- MartinG To mail me: echo kewyjlcxyzvjfxbqwh | tr bcefhjklqvwxyz
Ever use wily? (I should say acme, I guess). It uses extensive chording of mouse buttons to do away with pretty much all menus and control commands.
It's been a while since I used it (and then only briefly, so I'll make it up), but typically, you select a bit of text with B1, and then B2B1 chord on a word to execute that word with the selection as stdin. B1B2 chord selects then cuts, B1B3 pastes. jsut B3 opens a file.
I was unable to live without font-lock and programming modes from emacs so I quickly reverted, but it was suprising how quickly the idioms became subconcious (jsut like ctrl-a and ctrl-e are impossible for me to unlearn when using windows, to my enless select-alling and centering annoyance).
Very possible, Lionhead have mentioned both a Linux port and the possibility of releasing the source code. Both made possible by Lionhead funding the project privately.
treke
Fame is a vapor; popularity an accident; the only earthly certainty is oblivion.
True GESTURES use no mouse button; they just watch the pattern of position over time.
In Black & White, and in other applications that use a mouse to detect gestures, those mouse actions that do not require the buttons to be GESTURES. Drag & Drop is not a GESTURE but a direct interaction with the objects involved. Tooltips use the simplest of GESTURES: hovering in place. No mouse button.
On PalmOS and other devices where the pen must actually touch the screen to have its position registered, I would still call them STROKES, not GESTURES. Most artists' pen tablets can register the position of the pen if the pen is merely close to the tablet, not even touching, and therefore can support GESTURES.
[
I use gestures to start X applications.
See http://freshmeat.net/projects/wayv/
My party was for Shift-Ctrl-Click 'New Window in Background'. That is without a doubt my favorite features (apart from the whole 'self contained' bit), and the feature I miss the most in EVERY other brower!
And it can be gestured with 'Down Up' on a link! Woop! :)
Just wondering, I've been pondering a Mac purchace just for OS X for a few months now, and it'd be nice to be able to use Sensiva with it...
This is a very good example of providing a good user interface.
For those not in the know, it's totally invisible.
For those in the know, it's available right there and then!
I just wish all software was this user friendly..
Stop the brainwash
Ah found it on the Aminet - it was called Stroke. So there ya go.
It works better with a direct pointing device like a pen, less well with an indirect pointing device like a mouse, and badly with a velocity pointing device, like a joystick or force-sensing button. Basically, if you can't handwrite with the input device, gestural input will be a pain.
Many web sites do not allow the user to right-click, in an attempt to "protect" their "content" by using EcmaScript to disable the contextual menu that accesses "Save Image As..." and "View Source." This is a Bad Thing, as if even the Back button is moved to the right mouse button, the webmaster has complete control over your browsing. It's also annoying for those with disabilities such as dial-up connection.
(Read More...)
Tip: IE doesn't completely disable right-click. When the dialog pops up, keep holding the right mouse button and press Enter. Release to get your contextual menu.
Will I retire or break 10K?
Personally I think gestures are great if you work with a stylus, but mice are too clumsy to precisely use them. Even with my Palm Pilot it is not uncommon for it to screw up a gesture I make.
Anyone who's played Black and White will know that gesture navigation is actually a major pain in the butt to use. Sure it's novel, but frantically waving the mouse around for several seconds in some pattern does not make for a quick and useful interface, especially when it fails a good portion of the time.
I do think it's good that unusual and innovative methods of software control are being tried out, especially in a respected piece of software like Opera. Maybe this will get the ball rolling, but to tell you the truth I can't see this sort of thing being standard in any M$ apps any time soon. Users who need the Paperclip to help them "save as" aren't going to like the mousing equivalent of waving one's hands in the air :-) Maybe it could be implemented as a "power user" setting.
Freedom: "I won't!"
There is piece of CAD software called ARRIS that has been doing this for years. I rather like it, but it's not that new.
I don't need a million points of light, just two points of multi-mode fiber and a 10 Gig-E router.
I also noticed problems with /. afterwards: I was unable to navigate pass a main article page, clicking on a comment link would bring me back to the start page, logging me out in the process. I was going to post a comment on the anomaly but the same thing happened when i clicked reply.
What's weird is that when I tried to post in Netscape and then IE, they both did the same thing. Didn't have a problem with any other pages, tho.
I enjoyed Opera so much that if I can get it to work at home, I'll very likely switch to it permanently.
with the new Theremin Interface! With just two metal rods mounted on your monitor you can control your PC by just waving your hands in front of the screen!!!
Actually, that would be rather cool.... Especially if it still made the oooEEEOoooouuuUU noises.
Hacker: A criminal who breaks into computer systems
"Information wants to be paid"
Shouldn't there be a sort of consistency among the different applications used within one GUI. Before you know it, we are back in the 'goold' old DOS day, where every application was setting it's own standards (remember WordPerfect?). These gestures would only be nice if they were an integral part of the GUI, and thus useable for all applications.
I only use gestures in Black and White to the extent that I have to due to lack of keyboard equivalents. I use the keyboard to move, and I'm just happy I realized that you can press "R" to repeat the last miracle...
If you don't want my koalas, baby, don't shake my eucalyptus tree.
It's just sad that it came out of a game house instead of application programmers.
Wrong. Mentor Graphics have been doing gesture-based input for some time. I have personally seen it in one of their products at my work a couple of years ago. Also, read comment #8 (sorry, can't get <a> hyperlink to work).
-----
-----
Obviousness is always the enemy of correctness. -- Bertrand Russell
Just how revolutionary is this gimmick if people have not even noticed it for weeks or months? And if people have not noticed this bell and whistle before, is it even news?
or does this fall into the category of if it is a major bug, it is news, but if it is an anti-bug (ie, a feature) then let the PR department deal with it.
on a separate front, I can see this when 3D interfaces become popular for computers. You'd have a widget in your hand for interfacing with the machine, and operate the unit by gestures.
The similarity of this to a magician with a magic wand is *purely* coincidental.
:)
Check out the Vinny the Vampire comic strip
"It is a greater offense to steal men's labor, than their clothes"
...read the various combinations and think to themselves, "Back Forward Punch... Sonic Boom!"
So are we going to see the Opera strategy guide with the full list of key/mouse combos?
Actually, this concept is really not that new. Check out the Strokes extension in Emacs and XEmacs. It does this very thing. You can teach it various patterns, and then you can use those patterns to execute commands. Heck, now that I think about it, you could integrate Strokes and Emacs W3 to do exactly what Opera is doing...
As soon as I started thinking about this and reading some of the other comments in this discussion I started thinking the same thing. It's enough work sorting out which buttons on button bars do what (especially if you have non-standard buttons-- and this only gets worse in Linux as opposed to Windows, where at least there seem to be some basic bars for the fundamentals like cut, paste, print, etc). Usually word-based menus are not too tough since they use language, which is highly expressive.
But to me the idea that I'm going to have to learn to right-button-left-drag, just to do the same thing as cmd-left-arrow or alt-left-arrow seems like a major waste of time. And is right-button-left-drag going to mean the same thing in Word, Excel, or Outlook? Can it even mean the same thing? If there is a page scroll gesture, will it be consistent? Do we really think the big software houses will accept the Opera standards across the board, thus making life easy for users?
I do not have a signature
The reason that CAD tools have had these features for longer is that they are inherently more complicated. For example, just to rotate and translate your 3-D model, you have 3 axii of rotation and 6 directions of tranlation. How do you do that without some form of Gesture based control?
Maybe the idea of 'gesture based control' as used in web browsers will become the norm of Office/productivity apps. Finally, companies such as Microsoft don't need to dumb down their products anymore, since 99.9% of the office/productivity force use computers. They can concentrate on making them more efficient and not easier to use. I'm still waiting for a webbrowser that can see my eyes and determine which link that I'm looking at.
Keeping
What's sad is not the fact that the game houses are innovating and creating great new concepts and ideas for interacting with one's computers.
What's sad *is* the fact that a lot of programmers refuse to think like non-programmers and design interfaces that are truly beautiful to see and use, like B&W's.
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
This is the (real) innovation we've been waiting for. It's just sad that it came out of a game house instead of application programmers.
Think about it: The one reason that many people think that command prompt shells are superior to 'gruntnclick' is that the ability to use written language and commands is infinitely more flexible than typical WIMP operations. Despite the fact that it's slower than Grandma before she's had her prunes, most of the developers I know eventually drop down to csh or bash to get 'any real work' done.
Gesture systems, provided in combo with typical WIMP operations, have the potential to change that. If there is a gesture for every non-destructive command, and gestures can be stacked so that you can direct the output of one gesture command into another, you've created a truly flexible and intuitive command interface.
I've been playing B&W since it came out, and in only a couple weeks, I can shoot fireballs and sheild spells around like no one's business. I suspect that this will be true for a great majority of computer users. Not all, but enough to make the project worth it.
Zoom into an image in photoshop, select a square capture to clipboard, paste into new image. If I can do that with a few gestures rather than 8 different menu commands, I will have sped up my image processing dramatically without having to write a complex script or plugin to do it for me.
Now Lionhead has talked a little bit about releasing their source code if the game becomes popular enough. What I would like to see is source for their gesture recognition systems so that it can be integrated into KDE and Gnome, and OS plugins for Win32 and MacOS. With the level of interest in this new system, that may not even be necessary.
This *will* work. Get behind it, guys!
The next Slashdot story will be ready soon, but subscribers can beat the rush and slashdot the links early!
It actually worked well in my opinion, I'm not sure if it isn't supported any more or just isn't enabled at my current company. Time to drag out the manuals to see and maybe relearn SKILL.
Chris Kuivenhoven is a thief, beware
No I won't have to sit up and click the next arrow!!!!!!!
what? what I thought we were in the trust tree in the nest, were we not?
Cops: ID please Jedi gesture Me: You do not need to see my id
villager: "We need more poooorn."
whisper: "Pooooooooooooorrrrrrnnn"
--
I read this paper titled Contextual Animation of Gestural Commands a while back, found it fascinating and wondered when we were going to see some practical applications. I'm glad to see Opera et. al. trying the gesture nav approach.
Your fantasies contain the seeds of important concepts.