Designing a Practical UI For a Gesture-Based Interface

An anonymous reader writes with a link to an intriguing account of the challenge of designing a close-range, hand and finger-based gesture recognition interface using 3D cameras. Things like this look good in science-fiction, but it's hard to create a gesture-based system that makes sense to the user and rejects gestures not meant for the computer.

Comments

Test it on Italians

[Overzeetop]

You're going to have to test it on Italians for gesture rejection. I can't imagine what kind of havoc could be created if the interface over-saw even the most mundane of conversations.

If Microsoft is involved...

[TWX]

...I can think of a particular gesture that will be very common, but most likely won't be recognized as it'll be provided after the system isn't working...

Re:If Microsoft is involved...

[vlm]

American Sign Language? Oh wait you mean something else.

Seriously whats wrong with ASL? Large installed base, lots of training, could be pretty darn useful as a translator when someone without ASL "talks" to someone with ASL.

My guess is MS would try to embrace extend and extinguish ASL... make it just like ASL but upside down and backwards, just to make it hard for everyone.

Take a look at the Leap

[gentryx]

I just got my development kit from Leapmotion. Take a look at their 1m video. Pretty impressive, I would say. Also, the resolution of their sensor (1mm) seems to be much higher than that of what cameras can do today.

more challenging...practicality.

[argStyopa]

The biggest challenge is explaining who/how/why anyone would NEED a gesture-based interface.

Sure, we all thought Minority Report looked cool with the flying window-thingy, but honestly, Mr Cruise could have 'flipped through' his data far, far more easily sitting at a desk with trivial motions of a mouse instead of giant arm-wavings and calisthenics.

Plus, then he'd get to SIT, which is a little more conducive to surviving the 10+ hour days most of us spend staring at a little computer screen.

Re:more challenging...practicality.

[hey]

I dislike gesture UIs. Not just because they are new. The main reason is that if you are a newbie there are no hints about what to do. With shortcut keys you could always use the menus instead if you don't know the keystrokes. But Windows, Blackberry, Android, iPhone, all have different gestures its easy to get lost if you use more than one platform.

As an ASL Interpreter...

[bigattichouse]

I interpreted ASL in educational settings (High School, Freelance, University, Public, and even elementary.) for something like 6 or 7 years.

My arms were ripped, and you could expect to burn several hundred calories (EASY) during a day of doing that. Also, I had learned the stretch properly thanks to some Aikido training.. and I still had some bad habits that caused me repetitive stress problems.

Gestures are a novelty, and a lot of work for the user... I think there will be many blind alleys before they become natural.

Some problems/ideas I see:
1. Exhaustion - you waste a lot of energy
2. "Namespaces" - you can make two gestures at once - geez... so you have a left hand gesture that tells the computer to listen (the ASL "Attention" one handed would work) + a command - maybe even "against" that hand. Its like a salute with your left hand vertical moving away from your face.
3. Facial expressions are a HUGE part of ASL, probably not even considered. "WH" questions get eyebrows scrunched, other queries eyebrows up, puffed cheeks and all kinds of things...
4. Security - I defy you to sign EXACTLY like someone else... It's possible, and easy in a mocking sense (High schoolers) - but I imagine a door that could see you carrying groceries and unlock combined with voice recog., or other simple things would be useful.