Slashdot Mirror
Designing a Practical UI For a Gesture-Based Interface
An anonymous reader writes with a link to an intriguing account of the challenge of designing a close-range, hand and finger-based gesture recognition interface using 3D cameras. Things like this look good in science-fiction, but it's hard to create a gesture-based system that makes sense to the user and rejects gestures not meant for the computer.
You're going to have to test it on Italians for gesture rejection. I can't imagine what kind of havoc could be created if the interface over-saw even the most mundane of conversations.
Is it just my observation, or are there way too many stupid people in the world?
The beauty and the promise of devices like Kinect, and the power of modern computers, is the potential for an interface with almost no gestures. I know skeuomorphism has taken a lot of hits here lately, but this is the worst yet. They've made a model of a bookshelf in beautiful 3D, and the first thing you'll want to do is pluck a book from the shelf and peel the covers apart. Instead, you'll have to poke, and jab, and pinch, and drag. If you want a 3D gesture-based interface, bang it out Kinect style, or iOS style, with beautiful-looking icons to double-tap, or double-punch, or whatever. Make it look like a traditional interface, because doing the opposite will only make me sad when I discover that it is still just a traditional interface.
...I can think of a particular gesture that will be very common, but most likely won't be recognized as it'll be provided after the system isn't working...
Do not look into laser with remaining eye.
I just got my development kit from Leapmotion. Take a look at their 1m video. Pretty impressive, I would say. Also, the resolution of their sensor (1mm) seems to be much higher than that of what cameras can do today.
Computer simulation made easy -- LibGeoDecomp
The exaggerated gestures looked completely silly in Minority Report. Who in his right mind would want a computer like that?
take a big guess
The biggest challenge is explaining who/how/why anyone would NEED a gesture-based interface.
Sure, we all thought Minority Report looked cool with the flying window-thingy, but honestly, Mr Cruise could have 'flipped through' his data far, far more easily sitting at a desk with trivial motions of a mouse instead of giant arm-wavings and calisthenics.
Plus, then he'd get to SIT, which is a little more conducive to surviving the 10+ hour days most of us spend staring at a little computer screen.
-Styopa
I interpreted ASL in educational settings (High School, Freelance, University, Public, and even elementary.) for something like 6 or 7 years.
My arms were ripped, and you could expect to burn several hundred calories (EASY) during a day of doing that. Also, I had learned the stretch properly thanks to some Aikido training.. and I still had some bad habits that caused me repetitive stress problems.
Gestures are a novelty, and a lot of work for the user... I think there will be many blind alleys before they become natural.
Some problems/ideas I see:
1. Exhaustion - you waste a lot of energy
2. "Namespaces" - you can make two gestures at once - geez... so you have a left hand gesture that tells the computer to listen (the ASL "Attention" one handed would work) + a command - maybe even "against" that hand. Its like a salute with your left hand vertical moving away from your face.
3. Facial expressions are a HUGE part of ASL, probably not even considered. "WH" questions get eyebrows scrunched, other queries eyebrows up, puffed cheeks and all kinds of things...
4. Security - I defy you to sign EXACTLY like someone else... It's possible, and easy in a mocking sense (High schoolers) - but I imagine a door that could see you carrying groceries and unlock combined with voice recog., or other simple things would be useful.
meh
Sorry, but whatever idea you have was patented already.
I would imagine the most intuitive interface is what you do to move objects when dreaming. For instance when I want to pull an object toward me, I focus on it, then tug an invisible rope quickly as to whip it back. I also push my hand outward holding my palm out as to say "stop" to push it away. Somewhere in our heads we already know what will work
"When life gives you lemons, don't make lemonade. Make life take the lemons back!" -- Cave Johnson
I'm wondering when the first device will appear which shows GUI elements even before the user has touched the screen. i.e. it senses the user moving their hand close to some portion of the screen and a hidden GUI associated with that area pops up in time to be there when they touch it. It'd have to be done carefully to avoid frustration but I think it'll come in time.
As extra, optional, and not intrusive input device could be good, games are a good example, information consuming activities could be other. But for most content creation (music, and paint excluded) probably would be bad.
It's the solution humans use in gesture-based communication. Eye contact. The computer needs an obvious eye, and the software needs to be able to tell when the user is looking at it. Bonus points if the eye can blink or move or otherwise express the computer's understanding of the gesture.
Need a Python, C++, Unix, Linux develop
That would probably work. Face tracking is doable now and should be a subset of the capabilities of the hardware needed for a gesture interface in general.
Have to quote this from _The Hitchhiker's Guide to the Galaxy_ :
A loud clatter of gunk music flooded through the Heart of Gold cabin as Zaphod searched the sub-etha radio wave bands for news of himself. The machine was rather difficult to operate. For years radios had been operated by means of pressing buttons and turning dials; then as the technology became more sophisticated the controls were made touch-sensitive--you merely had to brush the panels with your fingers; now all you had to do was wave your hand in the general direction of the components and hope. It saved a lot of muscular expenditure, of course, but meant that you had to sit infuriatingly still if you wanted to keep listening to the same program.
The fundamental problem of gesture recognition has been recognized for quite some time now.
The gesture stuff is nice, but we still need to type words to communicate and I'd like to see some kind of new creative keyboard interface that will make it more portable. Why not use this 3D gesture recognition to recognize how we move our fingers, to allow us to type virtually without a physical keyboard?