Slashdot Mirror
Using Your Head As A Joystick
Ant sent in linkage to an article about Cybernet Systems and their new Use Your Head gaming peripheral that tracks head movements and uses them as input for games. Works using a USB Cam, and obviously its not gonna be running under Linux any time soon, but this is pretty sweet. When
they have the version that can detect me cursing and use that to signal a retreat, I'll be happy ;)
You can see screenshots and download an alpha demo HERE. Good luck.
It makes Street Fighter games realistic right down to the concussion!
Go ahead, try doing 12 dragon punches in rapid succession.
--------
Think about it for a second. If you had a VR helmet or some equivalent (visors, etc), and this, you would be able to have full immersion in the graphical environment.
Now, yes it would certainly make for a kick-ass game of quake. But what I want this for is to be able to extend my desktop from one screen to a full 360 wraparound. Maybe even an virtual globe centering on me.
Wouldn't that be great for programming? My personal experience has been that the more information you can have readily available, the better you do (How many of us have bought 21" monitors, just so we can fit more windows on the screen). I can just picture having my main development environment in front of me, a second window for code I'm re-using to my left, my web browser at about 45 degrees up, etc. Only problem I see is getting the mouse to work well under those circumstances, since it would have much farther to travel, relatively speaking.
Anyway, once someone writes the drivers to make my computer do what I just described, I'm getting one of these for home and probably another for work
Every response here has been under the assumption that the head will be the sole input device. But as both the linked article and the vendor web page states it is an additional input device used to augment your mouse/keyboard, so you'd use it like a third hand.
less keystrokes are always good.
Coming from inside Cybernet, I have a bit more info on the product for those that are interested.
:)
First, the product uses very little CPU (less than 5%) and given that it runs off of normal webcams (which typically have a framerate of less than 30fps) has little lag.
Second, the product is purely "optical" - you don't have to attach dots or sensors or anything to your head.
Third, the device isn't meant to replace the keyboard/mouse/joystick (I pounded mightily on the UseYourHead group in the beginning that gamers - specially FPS gamers - will never want to go away from their controller of choice) - but it basically gives you an unobtrusive tertiary device.
Fourth, it moves by tracking translational movement (left, right, up, down) not rotational movement of your head (twisting). It was obvious from the start that a device requiring you to take your eyes off the screen was a bad idea.
Finally, a bit more information about the tech. UseYourHead runs off of Cybernet's Gesture Technology which was developed on various Military R&D contracts and is capable of identifying complex gestures, where a "gesture" can be a series of positions by anything from your hand to your head or even your feet or something held in your hand. Think of slapping a camera on your TV and never having to search for your remote again because you can wave the channels up or down, or use simple signals to specify a channel.
UseYourHead is the first foray into commercial-land for this technology. We wanted something simple, something basic, something useful. Originally intended almost entirely for the first person shooter market, we recognized the basic motions most people make when playing those game is to weave, duck, and try to peer around corners, over ledges, ie. head tracking. Head tracking is *really* simple for the gesture tech, the hard part was getting it to work as fast as possible with the relatively slow web cams.
As a first pass, UseYourHead takes head movements and lets you map them to keystrokes. Its the simplest setup that allows UseYourHead to work well with almost all existing games. However, game developers can directly integrate support for UseYourHead (through a DirectInput wrapper) and have a more continous motion. Imagine your screen as a window into the virtual world, and as you try to peer around a corner, the game smoothly shifts to give you the correct perspective. Even more interesting, game developers can use our tech to access more complicated gestures. For instance, the game Black & White has a system for casting spells in it that requires you to make gestures with your mouse, imagine being able to use your hand directly to make those gestures (Somatic components from D&D
Oh, and while UseYourHead is meant for Windows, all the original tech runs on various versions of Unix (Cybernet puts out a Linux product called Netmax as well)
Feel free to email me if you have any questions, or visit Cybernet's web site for more company info)
Ron Hay
rhay@cybernet.com
Game Designer/Developer
Cybernet Systems Corp
Besides that; this will be fun too see when you're in the heat of the game and the phone starts ringing of something else is drawing your attention. Personally I don't think these products will do good at all. They sound like nice, short termed, toys which will also soon dissapear. Take for example MS's joystick (I forgot the name) which reacts to movement; is anyone actually using one of these I wonder.. In the local stores they seem to become obsolete. The same with the previous "head movement tool". You hardly see that around anymore as well.
I knew this rang a bell - here's a 1995 game for the SGI Indy that uses the indycam as a 3d input device. From what I remember it was kind-of-usable but not exactly pinpoint accuracy. Got it at home somewhere.
"don't fall into the fallacy of believing that Perl can solve social problems. Maybe Perl 6 can, but that's a ways off"
In the VR industry, stuff like this has been done for years ranging from cameras detecting light reflective points placed on the body,
magnetic field systems, muscle current detection, eye movement tracking, suits and gloves and helmets with multi-position switches, spring or memory metal tension measurements, infrared distance measurements, etc. There is a plethora of systems of various price and quality, and the leaders in this industry are Polhemus and Fakespace, with a few new folks who are consolidating the remains of failed companies and producing new stuff.
The VR industry was too early, like the AI industry, and failed to deliver on all its promise - and thus failed commercially. Also like AI, a resurgence seems to be building. However, I hope this time we can separate hype from reality (and dumb ideas from useful ones).
As you may have noticed when you move around real 3D space, you don't just use one part of your body. So, controlling movement through a 3D simulation with just your head seems a bit ridiculous. Why is it more ridiculous than using your hands to control a joystick? When you use a joystick, you are using your hands for their purpose: to manipulate small physical objects. This is then converted into movement in the simulation algorithmically, and (primarily since we're used to driving motorized vehicles) we're pretty good at mentally interpreting our hand movements into planar movement (and even some 3D movement) in 3D space.
Now, if you look at the VR and telerobotics community, what use is head tracking? It is usually used in conjunction with hand (or whole body) control of motion, in order to control field of view. That is: your body or hands control movement, and you use your head to look around. Sounds familiar. That's the point.
So, if you combine decent head tracking (and the technology they use does not seem decent) with joysticks or gloves or suits, you're on your way to something (Fakespace has a nice system called the Boom which is fairly good for both moving and looking around). The problem is, can you get the response time necessary for your mind to associate the movement of your head with what your eyes are getting back from the screen.
First off, the bezel on the CRT may interfere sufficiently with your field of vision that your mind loses track of the association between head tracking and "looking around" in the simulation (or game). Serious VR and telerobotic apps with lots of funding use periscopes (like on the Boom), goggles, or helmets to ensure that association by immersion into the environment.
Experienced gamers may have overcome the motion sickness from non-immersive motion sufficiently to not have this problem, but the differing nature of the head-tracking vs. joystick (or keyboard) game movement may be such that you still have a period where you have to adjust by playing A LOT and getting used to the sensation - that is, training your mind to perceive this weird world you now inhabit.
So, it may be fun to use such a hybrid system, if the technology is there. However, camera-based tracking technology has the worst tracking resolution and response time so far. Worse, this system is going to be highly succeptable to image noise - a problem Biovision and others got around by using reflectors you place on your body and associate with points of movement that the software maps each point on the body to. What their system seems to be doing is trying to handle this with motion vector extraction of the whole image received, and/or diffing the image and using quadrants or octants to translate changes into on-screen motions (unfortunately, I could not try this out to verify).
Such a methodology is (a) slow (especially since the tracking computations are being done on the machine that is already bogged-down rendering the game - VR apps are often multi-CPU) and (b) not terribly accurate... So, this system, in my analysis, is just hype. It is no new direction in gaming, rather an old one poorly rehashed.
I suggest that you stick with joysticks and keyboards until the good stuff becomes more affordable...
o/~ we are pissed, we are pissed, we have to resist... o/~ - ec8or
Thanks for the book reference! I think I'll be buying a copy today. (Although I prefer Amazon. :+)
Please mod this post only if you think others should/n't read this. I have enough ego^H^H^Hkarma. Thanks!
This thing could be dangerous! In some of the battles we've had at work would have given me whiplash for #(*$ sake!
Seriously, they are just asking for trouble.
why don't you use your head for thinking? That's what it was designed for...
Don't post on slashdot. Get back to work.
(Ok, it's only $30, people will expect it to be not-very-good, and no, I haven't tried it).
We have some expensive VR kit, including head trackers. Guess what, they have noticable lag - this is custom hardware, and not cheap. How fast is a software program that tries to analyse the position of your head from a webcam going to be? My guess - not very.
I wonder how much CPU it needs to do it too (whilst you're playing Quake III)...
Mike.
ps) Would still be fun to get for Christmas though, I guess they are releasing at the right time.
Tales from behind the Lagom Curtain
Why are you knoking your head over the table?
I'm out of ammunition and now I'm using my bare hands...
(Interesting note, I believe that Marathon did it the reverse way, instead of strafe, you turned your head left or right while your body kept forward. You still were limited to 8 directions from this point, of course.)
Anyone remember Robotron? In this game, your head facing was different than your body facing (you had two joysticks). I think that this head joystick would be applicable to games like this, only limiting your head movements to approx 135 deg in both left/right and up/down axis. Your mouse and keyboard now control your body's movements save for up/down looking, while your headpiece now controls the direction your head (and therefore your weapon) is facing. Yes, that would cause a bit of relearning, but it increases the freedom of movement of the character (you can now move, while facing north, NNE directly). As for whiplash, instead of turning your head around fast, you'd have to turn your body around fast, which is limited to the keyboard/mouse (since you can't get your head 180 degrees by itself!)
"Pinky, you've left the lens cap of your mind on again." - P&TB
"I can see my house from here!" - ST:
Yeah, I'm a nerd. But that's the whole point.
I would GLADLY stick a bleepin dot to my forehead and wiggle my head around to be done with my mouse. I would at least like to try it.
I used to be a right hand mouser. RSI.
I am currently a left hand mouser. RSI.
If they don't come out with something for some other body part where I can move the pointer around without a mouse, I'll be doing surgery someday.
Oh well....
Please mod this post only if you think others should/n't read this. I have enough ego^H^H^Hkarma. Thanks!
Getting distracted by the cat, and watching my avatar pirouette into a vat of lava...
Five tons of flax.
These guys aren't the first, or even the second company to propose this sort of pointing system.
The Nod was a pointing device developed by Sage/Stride in the pre-ibmpc dawn of computing that worked by sticking a reflective dot to your forehead and standing a sensor on your monitor.
It didn't sell well, because people felt silly sticking a reflective dot on their forehead and wiggling their head around to move the pointer.
This is just like television, only you can see much further.
*Sneeze* .. 'Bugger, I fell off that cliff again'
http://twitter.com/onion2k
Ming for president!
Seems to me that somebody tried this a couple years ago for PC games and the general consensus was that the thing sucked for two reasons. 1. It tracked twists of your head, which seems cool for flight sims and stuff until you realize your monitor isn't bolted to your face and it's very difficult to play a game without looking at it. 2. Your neck gets tired a lot faster than your wrist, and your eyes get tired even faster than that when you're forcing them to stay locked on a monitor while you twist and tilt your head.
A Transmission From PlanetJIM.[end trans]
Try to imagine playing Track and Field (or any other Joystick Waggler) with this. You'll look like you're having a fit!
So now I have a choice between this and a wireless joystick on a headband.
Vaguely reminiscent of the old Mindlink controller for the Atari 2600. That had a band you stuck on your head which would detect facial muscle movements. I think it was probably more fun to watch someone else playing with it than to do so yourself...
The low frame rate problem would probably be exacerbated when you consider you would probably have to move your head a fairly large distance (at least an inch ? ) for the software to be able to reliably detect movement.
Also, it seems like a camera pointed directly at your head is only going to be able to discriminate between four directions (head up, down, left, right), and wouldn't be able to easily detect (head forward, head back), which might be important directions if you were playing a FPS.
Finally, I wonder how sophisticated the motion detection scheme is ? How would they deal with, e.g. the rotation of your head, which might look to a simple-minded scheme like side-to-side movement ?
Sure, this sounds cool, but the usefulness of it is pretty minimal. For quite some time now, since the advent of Doom, the most popular input device combination for computer gamers (or at least the professional ones), has been keyboard+mouse, because of the number of buttons and quickness of response. It'd be nigh impossible to perform a rocket jump (aim straight down + fire rocket + jump to blast self into air) and then aim at an enemy on the ground using your head. Try (but carefully!) to move your head one way and then reverse direction and go the other. Now do the same with a mouse. And I don't believe this can be used to click your mouse buttons, either, unless it's got a real good way to pick up on left/right eye blinks as clicks :)
But yes, I will concede that this is "a-cool-thing," and might find a small niche in the gaming market. It reminds me of the arcade gimmick machines where you (and optionally a friend) stand in front of a blue screen and play Tekken by really punching, kicking, and jumping. It was almost impossible to do anything in the game correctly and quickly (ie: forget combos), but it was still damn fun.
I am interested in this - my main question is what do you guys use for tracking? Accelerometers? It didn't appear that there was any form of "full 6DOF tracking", based on the video - but rather a limited yaw/pitch/roll tracking (in other words, it looks like you are tracking orientation of the hand, and not position).
How are you guys "getting around" the VPL/Lanier patents (or are you licensing them from Thompson)?
What is the price target? And will there be Linux support in some manner?
I support the EFF - do you?
Reason is the Path to God - Anon
Tracking head movement is so low-expectations when Cybernet also offers a high-speed real-time optical tracking system designed for full body motion capture.
Now, if some enterprising programmer out there can figure out how to interface this to Quake or to a PS/2. Then add some force-feedback equipment!!!
Can you GNU???
I rarely read replies, it's my opinion and if you thought about your opinion a little more, I'm OK with that.
I have broken an old (almost) 10 lb. Northgate Keyboard over games of Quake before. I can't imagine banging my head against the wall as hard as I can ;) It may actually save me money in the long run ;)