Slashdot Mirror

A much more informative page that explains the technology (called GRASP - Glove-based Recognition of Auslan using Simple Processing) briefed in the CNN article can be found on Waleed Kadous's website:

GRASP Site:
http://www.cse.unsw.edu.au/~waleed/thesis.htm l

More generalized Gesture & Sign Language Recognition Research:
http://www.cse.unsw.edu.au/~waleed/gsl-rec/

Also see the self-proclaimed Gesture Recognition Home Page (good resource, tons of links)
http://www.cybernet.com/~ccohen/

Or just search google like I did for 'Machine Gesture Sign Language' and get a wealth of links.
</whoring>

We've been doing optical gesture recognition tech for several years now. Funded by various grants from various organizations, we have one product (Use Your Head) that uses a normal USB camera (only on Windows though, sorry) and turns a person head motions into keystrokes. The intent of that product is to turn the normal head-bobbing and weaving seen by people playing FPSs into actual movements. It uses < 5% CPU and is completely customizable including sensitivity.

If you want, check it out at:

Main Gestures Site

Use Your Head Site


Ron Hay
Turing Machinist
Cybernet Systems Corp.

Coming from inside Cybernet, I have a bit more info on the product for those that are interested.

First, the product uses very little CPU (less than 5%) and given that it runs off of normal webcams (which typically have a framerate of less than 30fps) has little lag.

Second, the product is purely "optical" - you don't have to attach dots or sensors or anything to your head.

Third, the device isn't meant to replace the keyboard/mouse/joystick (I pounded mightily on the UseYourHead group in the beginning that gamers - specially FPS gamers - will never want to go away from their controller of choice) - but it basically gives you an unobtrusive tertiary device.

Fourth, it moves by tracking translational movement (left, right, up, down) not rotational movement of your head (twisting). It was obvious from the start that a device requiring you to take your eyes off the screen was a bad idea.

Finally, a bit more information about the tech. UseYourHead runs off of Cybernet's Gesture Technology which was developed on various Military R&D contracts and is capable of identifying complex gestures, where a "gesture" can be a series of positions by anything from your hand to your head or even your feet or something held in your hand. Think of slapping a camera on your TV and never having to search for your remote again because you can wave the channels up or down, or use simple signals to specify a channel.

UseYourHead is the first foray into commercial-land for this technology. We wanted something simple, something basic, something useful. Originally intended almost entirely for the first person shooter market, we recognized the basic motions most people make when playing those game is to weave, duck, and try to peer around corners, over ledges, ie. head tracking. Head tracking is *really* simple for the gesture tech, the hard part was getting it to work as fast as possible with the relatively slow web cams.

As a first pass, UseYourHead takes head movements and lets you map them to keystrokes. Its the simplest setup that allows UseYourHead to work well with almost all existing games. However, game developers can directly integrate support for UseYourHead (through a DirectInput wrapper) and have a more continous motion. Imagine your screen as a window into the virtual world, and as you try to peer around a corner, the game smoothly shifts to give you the correct perspective. Even more interesting, game developers can use our tech to access more complicated gestures. For instance, the game Black & White has a system for casting spells in it that requires you to make gestures with your mouse, imagine being able to use your hand directly to make those gestures (Somatic components from D&D :)

Oh, and while UseYourHead is meant for Windows, all the original tech runs on various versions of Unix (Cybernet puts out a Linux product called Netmax as well)

Feel free to email me if you have any questions, or visit Cybernet's web site for more company info)

Ron Hay
rhay@cybernet.com
Game Designer/Developer
Cybernet Systems Corp

Tracking head movement is so low-expectations when Cybernet also offers a high-speed real-time optical tracking system designed for full body motion capture.

Now, if some enterprising programmer out there can figure out how to interface this to Quake or to a PS/2. Then add some force-feedback equipment!!!

Can you GNU???

No, Cybernet is a very established company. They have been around for twelve years. They invented force feedback technology and continue work on government research on human-computer interaction. A trip to their website would be most helpful. www.cybernet.com

-f
http://www.peruano.org

OK I took a small visit to google and found a rather good link collection about gesture recognition either using cameras or gloves. This things would make an interesting supplemet, especially if they could be permanently attached. (Maybe using bio-energy like those tracking devices mentioned a few days earlier).
Ciao, Peter

Peak Performance
VNSIII
Whole Body Kinesthetic Displays

Ken