Slashdot Mirror
Head Tracking w/ the Wiimote
mrneutron2003 writes "This guy just doesn't know when to stop. Johnny Chung Lee graces us with yet another one of his inventive Wiimote projects. This time it involves using the Wiimote and a pair of inexpensive LED safety goggles (with the standard LED's replaced with InfraRed ones) to allow positional head tracking , achieving an effect similar to what is experienced with three dimensional displays and CAVE systems. The video dramatically illustrates the effect. Game developers take note. This simple little variation on infrared tracking could allow for some seriously immersive gameplay in the future." This guy deserves a medal.
Surely he's sent in his resume. That's some really cool concepting, and not that Nintendo doesn't have their own cool concepts, but this is just incredible. The best part is, it's really simple and appears to be mass producible for cheap - two things Nintendo does well already.
No sig for you. YOU GET NO SIG!
The youtube video on the linked site comes up as unavailable, but the one actually on the youtube site seems to work. http://www.youtube.com/watch?v=Jd3-eiid-Uw Cool stuff.
That is freakin COOL.
I can only imagine what something that that, coupled with a graphics engine like Assassin's Creed has would do for immersive gaming.
Is it just me (and my girlfriend), or does this guy sound a lot like Kermit the Frog?
Also, the head tracking is awesome.
adventure-today.com
The Wiimote is truly the ultimate hackable peripheral...
The Wiimote can only track four IR sources at the same time, which seems a fairly arbitrary limitation that should be trivial to lift. With the IR light sources uniquely identified through signals encoded in the light stream, many more could be tracked. Shouldn't Nintendo upgrade the Wiimote quicky to allow for tons of new applications?
blow your mind already
Headtracking for games has been around for a long time but this solution really takes the cake for using inexpensive, off the shelf technology..
The TrackIR solution linked above costs around as much as a Wii itself.
Unless nintendo (being the intuitive ((not a fan boy)) people they are) have already thought about this and are just waiting a while to release a title that you can use this functionality in.
Of course, having a 42" plasma or larger is going to be the optimal thing here - and since the wii is priced to sell, then a lot of people are going to be using them on their old 1980's 27inch CRT's.
I think this is cool as can be and all. I even think writing this software is a pretty awesome accomplishment. But we have had head tracking systems for a while now. It is only now that the components have become a commodity. Nintendo's Wiimote is just the beginning of decades old technology making it out of the lab and into Walmart.
Between those things and multi-touch, I am literally waiting for a revolution of computer input design. 10 years ago, there was the movement, but not the technology. Today we have the technology. Please, give us some games that use this, give us multitouch tablet Macs (sorry windos fanboys, microsoft could pull it off technologically, but it wouldn't be useable), give me a VR multitouch table! Now! The flying car can wait until next year...
Assorted stuff I do sometimes: Lemuria.org
Give this man a consulting job!!!
Nintendo, are you listening?
from the this-is-just-to-cool dept.
You spelled 'too' wrong.
Honesty may be the best policy, but by process of elimination, dishonesty is the second best policy.
Does the Wii have enough horsepower to pull this off on its own? The demo was running on his PC, and I'm curious how processor intensive something like this would be.
Why the Wii isn't for "serious" gamers? Who needs 1080p when you've got this?
Combine this with the weight-shifting capability of the Fit, and you've got an immersive gaming experience that's second only to the holodeck.
So. Freaking. Cool.
"Only two things are infinite, the universe and human stupidity, and I'm not sure about the former."
and I can't watch this video but I have never before seen slashdot so unanimously agree that something was cool... I think the first thing I will be doing after getting out of this connection hell is watch this video...
When are game developping companies or even Nintendo going to give this guy a contract? He is the master of the innovative ideas.
when you turn your head to look at something in an appache, the machine gun follows. When you turn your head and the Wii follows....You're not looking at the screen anymore. I tried the wii at a best buy and I almost threw it through a wall. I don't know if it was broken or what, but it sure was a pita to get it to do anything.
Oh Crap, I'm an optimist.....
It's been done. Remember the Nintendo PowerGlove?
Incidentally, if you've never tried gloves-and-goggles VR, it's cool for about ten minutes. Trying to do things by making gestures in the air is a huge pain. Without tactile feedback, it's tiring and inaccurate. I tried most of the VR systems in the first round, including Jaron Lainer's original system. No good.
It might not suck if the system had an end to end lag of under 10ms. "Turn head, wait for view to catch up" systems drive the user nuts. That problem was solved by "cave" type systems, where the user is surrounded by screens. Bulky, but tolerable.
http://www.naturalpoint.com/trackir/
What is interesting is that he's coming up with some very creative ideas, and giving them away for free.
This will likely spur an avalanche of Wii hacks, and could easily cause wiimote sales to go thru the roof..
I'm totally enjoying the adventure Johnny!
Bavarian Purity Law of Rice Krispie Squares: Rice Krispies, Marshmallows, Butter, Vanilla.
This guy deserves a medal.
I guess he'll have to settle for a PhD from Carnegie Mellon instead.
Because there aren't any "serious" games for it.
May the source be with you.
How about three WII remotes together... you'd have a virtual room you could write on and move things around with your fingers?
As I understand it, the infrared detector on the Wii remote is basically a camera with an IR filter in front of it.
Potentially you could just use a webcam with an IR filter in front of it instead of a Wii remote.
Note: 1) there is usually a filter to filter out IR inside most webcams, so that would have to be removed. 2) IR emitter tracking would have to be done on the PC instead of inside the Wii remote.
Maybe they meant that it was not to be used for heating?
When our name is on the back of your car, we're behind you all the way!
Alas, shutter glasses(a polarized screen with an LCD shutter) and cross-polarized glasses don't play with LCD displays, because LCD uses polarization to turn the pixels on and off. LCD latencies are also a little high for shutter glasses.
... an no-one is going to build the latter until there are plenty of mainstream 3D apps to support the market.
They only work with DLP projectors (uses little mirrors), CRTs, plasma, and upcoming display technologies like Field Effect Displays and LED displays. Obviously there are a lot of display technologies that do work there, but LCD is a very popular technology for widescreen TV and of course, for PC monitors.
Either way you do it, you also have to double the grunt of your rendering system (or half your graphical complexity), and you need specific software support to get it right (you can go a long way with a driver that knows it's rendering for stereoscopy and just produces the correct eye POVs, but the glitches you get in the foreground and HUD are only tolerated by enthusiasts.). With shuttering you need glasses. With cross polarization you need to double the number of display elements (by having two displays or a special display with double the horizontal resolutions). Used in POV applications, all of these technologies are a one-user gig.
Stereo "Wii-D" will probably never happen ; half the audience have an incompatible display device, the system does not have an enormous excess of GPU grunt. Stereo3D would only be common with one of the following display devices...
* Personal head-mounted 3D display (probably VRD goggles)
* Large area wide aspect flatpanel displays with inherent stereo 3D support built in at the factory (which means basically doubling the vertical rez and making a special polarized filter for the screen).
The parallax effect that Johnny Lee demonstrates conveniently exploits the tendency of the human brain to "fill in the gaps" ; I'd be intrigued to see how convincing it really is.
As another poster points out, head tracking really isn't very well received for the PC, because the PC is an inherently static device. You can move your head, but your hands have to remain fixated on the keyboard / mouse. The Wii has an advantage here because the input device moves around with you. Several times during Zelda I got up from my chair and started moving almost involuntarily, my whole body was immersed in the game. I would never have tried that on the PC ; when I feel the urge there it probably just contributes to my neck tension.
If the static, 3rd person POV of Zelda can make this gamer rise up and move, a game armed with a head tracking linked POV would be compulsively immersive, even without stereoscopic 3D.
Not Linux? Not useful. (But very cool.)
An I the only one who thought of this when I read the title:
http://www.thinkgeek.com/stuff/41/wiihelm.shtml
Ramen
Holy shit that was awesome, why is this guy not employed somewhere they can give hive lots of money? If I were in a gaming department for the next XBOX360 flight game or something, I would hire this dude and give him as much money as he needed to make potential customers feel as if they were inside a frigging airplane lol man that was sweet looking.
If you don't vote, you don't matter, so don't waste your time telling me your opinion
Just thought of this. This would be a wicked interface, and much more natural, for someone piloting a drone helicopter or even robotic vehicle.
If you had several monitors, this could be used to make them feel as if they were an actual "pilots seat" of a vehicle giving perfect perspective to the "pilot" because they know where the head is oriented and each monitor could produce the proper peripheral and image views for the "pilot"
It would take a little tricky camera work for the robotic vehicle, but I am sure gratuitous funding could solve those problems.
If you don't vote, you don't matter, so don't waste your time telling me your opinion
Why does he always use a wiimote for this? A couple of years ago, i build a basic minority-report-like-wave-your-hand-in-front-of-the-screen UI with a webcam, an IR-LED and a infrared filter in front of the webcam (i simply used the dark-red looking plastic that usually covers the transmitting LED in a regular remote control). I then simply pulled the images from the webcam using some v4l-utility, and ran some (extremly basic) pattern-recognition in perl to detect the white 'dot' that represents the IR-LED that was stuck to my hand, and used it to move the cursor around on the screen - really not too difficult.
If someone would write a nice library for this (using intel's obencv library for the pattern-recognition, for example), almost all the 'hacks' from this guy could be implemented without the wiimote, and this would seriously rock.
apart from that - AFAIK, face-detection (just detecting a face and the position of he eyes in an image) usually works well enough, so this particular hack should even be possible without using any infrared LEDs, and this would be really cool if implemented correctly. Any takers? Becasue it really looks cool in the demo.
He says this is only good for one player at a time.. if the Wii remote was able to map other colors to other players, it'd be able to handle different players for each window in a split-screen game, no?
I would love to have this capability for flying the aircraft in Battlefield 1942 (and Desert Combat). My biggest challenge in dogfighting is that I can't crane my neck around to follow the other aircraft off the screen!
"Slow down, Cowboy! It has been 3 years, 7 months and 26 days since you last successfully posted a comment."
He has great projects/ideas, but why only windows! Nothing hes made supports OS X or Linux! :(
He should at least open the source!
All these things are awesome. I can only imagine how amazing this particular hack would be if you modified Metroid Prime 3 to track the player's head. It would just be too amazing. And the next time I have to give a presentation on something I'm definantly using the lightpen/wiimote touch display thing.
--The universe will not be altered by forum threads, even those which are very wry. --Tycho Brahe (Penny Arcade)
Homeboy sounds like Ray Romano or someone maybe doing an impression of Ray Romano. I bet that could be worked into a marketing strategy, considering who's playing Wii these days :)
When the axe came to the forest, the trees said, "Look out - the handle was once one of us."
Anyone who's been to an arcade in the last few years has seen atleast either Police 911, or MoCap Boxing. both games use similar tech with out the need for special sensors to detect the body(mocap boxing uses special gloves to detect the location of your fists though).
It's neat, but, big deal.
Non impediti ratione cogitationus.
Old news: http://local.wasp.uwa.edu.au/~pbourke/projection/Wii/index.html, the wii remote is not accurate enough.
I was just thinking about how all this worked, and i'm confident what i'm thinking won't work with wii hardware, but i realized how to make fully functional 2 person VR with this technique. it would require two pairs of shutter glasses, and a system capable of pushing over 144fps, simply put, it broadcasts data set A for user 1 and datas et B for user 2, and closes A's shutters when B is on screen, and B's shutters when A's is on screen. at 144+ fps, each user gets 72fps, which is more than enough to make the dual broadcast un-noticeable. Only hard part is the hardware to do it. now i know there are lots of problems with shutter glasses and all that, what i just described would be the shortest path i can think of to achieve 2 person function on one screen.
I've decided to Diversify my Holdings. I've divided my cash between my left and right pockets, instead of all in one.
That way 2 players could have split screen action as the IR signals/readings would still work... (well, not use the Wiimotes, but another form of sensor such as glasses ?)
I just released some of my code I was playing around with this summer
:/) was tracking color LED's. This way I was able to create a two player pong game by using a regular webcam and green/red LEDs.
it allows for very similar things.
you can find the code here http://snippets.dzone.com/posts/show/4920
be warned, it has no documentation and its really messy.
the included demo is able to track and plot the movement of multiple IR emitters. I have other demos as well like "swing" detections (ie. swinging motions). A more elaborate hack involved using a wireless mouse+LED to simulate a wiimote experience (moving around the mousepointer and clicking).
To recreate the hack in this story all you need to do is to attach the IR emitter to some glasses.
The last thing I was working on when I had to quit (school started
Obviously using an IR-webcam (simple hack) with IR emmitters is a lot easier because it removes a whole lot of "noise" (ie. useless motion).
"The majority is always sane, Louis." -- Nessus
http://slashdot.jp
While much of this guy's latest stuff has been cool in a fun toy sort of way, it doesn't really do anything to cover any actual interaction beyond cursor movement. What this still needs, is an easily configurable method of generating different types of mouse clicks and scrolling to accompany the IR tracking. While he could just go back to using the Wii Remote normally, it just wouldn't be hacker worthy.
So, I've been thinking... how do you address the click/scroll stuff and the position tracking, all using only one hand?
My solution... modify an optical bluetooth mouse. Rip out the built-in LED used for optical tracking and use the available leads to power an external IR LED positioned at the top/front of the mouse body. Once you have this much in place, the mouse can be tracked externally by the Wii-Remote, but would still be perfectly capable of broadcasting mouse click data directly to the same machine. All you'd need to do is find a mouse that can be comfortably used in an upright position.
Sure, it's not quite Minority Report style control, but it would offer a lot of flexibility while requiring only a minimal set of custom drivers.
8==8 Bones 8==8
here's a link to some demos. demos are buggy cos I don't have my good webcam around
http://www.youtube.com/watch?v=UGImvHEtUpE
"The majority is always sane, Louis." -- Nessus
http://slashdot.jp
"There are eye-tracking systems that are not nearly as fatiguing, but if you've seen one, you'll understand why they haven't taken off in popularity."
They seem to be popular with this guy.
SPINNING DROP KICK~!!!!
When was the MIG-29 introduced again?
IIRC they made many more Cobras then Foxbats. So your mass application claim falls kind of flat as well. Not that it matters.
John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
Not in this case though.