Slashdot Mirror


Details On Natal's Motion Capture Technology

An anonymous reader writes "Following yesterday's announcement of a late 2010 launch date for Natal, more details are emerging on exactly how Natal works. Alex Kipman, the project's lead developer, explains that Natal uses only 10-15% of the Xbox's resources to calibrate to a new player inside 160 milliseconds, track one or two players simultaneously, and use rudimentary knowledge of body anatomy to estimate where hands or other body parts are even when they can't be seen by Natal — for instance when they are held behind the back."

25 of 121 comments (clear)

  1. How obfuscated protocol? by sznupi · · Score: 2, Interesting

    The aspect of how Natal communicates with X360 is most interesting to me, and surely many others.

    It seems like it could be mighty fun peripheral for robotics projects... (and who knows, perhaps MS could even promote it, seeing that they have their robotics suite? Certainly small number of Natals not used for gaming wouldn't harm MS financially in any significant way; but they might like good publicity)

    --
    One that hath name thou can not otter
    1. Re:How obfuscated protocol? by AndrewNeo · · Score: 3, Interesting

      I believe there was talk of the hardware working on the PC, too, though the hardware is pretty much just a few cameras and other sensors, leaving it up to the libraries on the Xbox or PC to do the processing. Now, if Microsoft releases an SDK (which is in their best interest, if they want PC integration) then it shouldn't be an issue.

  2. Anyone Know How Accurate Natal Is? by eldavojohn · · Score: 2, Interesting

    Accuracy aberrations plague a lot the new free motion input devices. Does anyone know to what degree of accuracy this works? Down to the inch? Centimeter?

    My friend was lamenting how in Rainbow Six he could hug up to a corner or object and only expose a small part of his body when he poked out to shoot. The problem being that in Modern Warfare 2, this is not the case. You have to expose your whole body. He was hoping/speculating that with Natal, a first person shooter might be able to lock into a stationary mode (much like when one is shooting through a scope) and be able to move his head, shoulders, arms, etc as he pleases to peak (and/or) shoot around a corner when he's pinned down. He wants these games to get as close to real life as possible and I told him that it's a great idea and would increase the reality of pray and spray suppressive fire tactics ... but I was skeptical of the devices ability for this accuracy. So, anyone know any concrete numbers yet?

    --
    My work here is dung.
    1. Re:Anyone Know How Accurate Natal Is? by TheRealMindChild · · Score: 3, Funny

      So, anyone know any concrete numbers yet?

      I don't have any concrete numbers, but I did just pick up this foam number one a few days ago...

      --

      "When life gives you lemons, don't make lemonade. Make life take the lemons back!" -- Cave Johnson
    2. Re:Anyone Know How Accurate Natal Is? by flitty · · Score: 3, Informative

      The article says that 31 limbs are tracked to within a 4 cm box. So move your hand 1.5". It's that accurate (which isn't great).

      --
      Whether or not there is some sort of god, I'm not supposed to say/god is a word and the argument ends there-Smog
    3. Re:Anyone Know How Accurate Natal Is? by MagicM · · Score: 2, Funny

      Fixed that for you.

      He wants war games to get as close to real life as possible without any actual work, exertion, exercise, social interaction, or dirt.

  3. Natal Demo by rehtonAesoohC · · Score: 4, Informative

    Here is a tech demo showing a Peter Molyneux demonstration of a game using Natal.

    The ramifications of a system like this are pretty important I think. Honestly, this is what I had hoped the Wii would become... Not to start a Wii controller vs. Natal debate or anything, but I feel like games using full body motion capture would be much more interesting to me. At the end of the day, when I tried to play Wii tennis using proper motions and what not, I felt like an idiot after looking over and seeing someone else accomplish the same thing by flicking their wrist.

    That said, I am interested to see how finely grained the sensitivity will be tuned to Natal recognizing people and motions. IE: if person A is playing a single player game and person B walks up and makes a hand motion, will Natal disregard that motion, or will it do whatever person B did? If so, it could seriously dent the efficacy of games being played by one person... Part of the benefit of having a physical controller is that someone else has to take it from you when they want to play, haha!

    1. Re:Natal Demo by Anonymous Coward · · Score: 3, Interesting

      I'm sorry, Molyneux is the LAST person I'd trust to give a balanced review of anything.

      Especially Natal.

      You could also see for yourself

    2. Re:Natal Demo by __aapspi39 · · Score: 2, Insightful

      I call shenanigans on this demo. once the footage of milo actually starts, you may notice that the avatar is looking out from the screen directly towards the girl at points.

      Granted, she is rather easy on the eyes, but this tends to suggest the thing has been staged. Either that or the tv used has a 3d capability more advanced than anything in existence.

    3. Re:Natal Demo by sammyF70 · · Score: 4, Informative

      yeah ... the milo demo is a semi-fake. there was apparently someone behind a curtain controlling the character see : http://www.gametrailers.com/video/episode-60-invisible-walls/51228?type=wmv (somewhere in the third quarter they talk about it)

      --
      "DRM is like the Ford Pinto: it's a smooth ride, right up the point at which it explodes and ruins your day."-C.Doctorow
    4. Re:Natal Demo by DrXym · · Score: 4, Interesting
      Here is a tech demo showing a Peter Molyneux demonstration of a game using Natal.

      Tech demo? More like total fabrication. Microsoft are merely insinuating they have perfected natural language processing, natural speech synthesis, perfect motion recognition and facial recognition amongst other things.

      I'm sure Milo will turn up at some point but it will be a pale imitation of this. We'll realise that you can grunt and howl at Milo and get the same reactions.

    5. Re:Natal Demo by Xest · · Score: 2, Insightful

      Microsoft never made any such insinuation whatsoever. I don't blame you for having this mistaken understanding of the situation as there's so much FUD going around about the Milo demo created by fanboys of other consoles but it's still completely and utterly wrong so rather silly to keep repeating.

      The Milo demo does one thing and does it well, it demonstrates how Natal can be used as a CONTROLLER to get PRE-SCRIPTED RESPONSES (whilst I hate caps, these points are fundamental to what Natal is and how it can be used). This is no different that when you walk upto a character in an RPG with a classic console controller and the NPC speaks, the NPC isn't recognising you as a person has turned up and speaking specifically to you, it's detected you are in range through basic math and played back a specific response. The Milo demo is no different other than the fact the controller is different- they have demonstrated that Natal as a controller allows for different inputs just as a classic controller usually has two directional sticks and a multitude of buttons. It can scan images, it can respond to gestures and it can respond to voice recognition (something that has worked for years now- see Endwar for a previous successful example of it's use in games).

      It's far from a fabrication and all the conspiracy theories that fail to actually show anything worthwhile are utterly stupid. One example was some dumb Youtube video pointing out Milo is not looking at the woman- well duh, it's pre-scripted, it's designed to look good for the camera. The real issue with the Milo demo is this, fanboys saw it as a serious threat to their favoured console because they made the false assumption in their fanboy rage that it was demonstrating strong or near strong AI which is clearly absurd to anyone thinking rationally.

      Natal is what it is, it's a controller, it will allow more immersive interactions, but no one's really pretending that alongside Natal Microsoft have managed something that might not even be possible, and certainly isn't possible with today's technology- strong AI. Microsoft made it quite clear the Milo demo was to show off the potential of Natal, they never once tried to pretend that alongside Natal they'd made a massive leap in AI. If you made that assumption either mistakenly or maliciously because of fanboyism then that's your insinuation not Microsoft's. Interestingly what this also means is that because Milo was pre-scripted, the whole thing is also perfectly possible to have in your living room but the caveat is that it wont be as exciting as your insinuation suggests, because like the actor, you'll also have to perform the pre-set actions that get pre-scripted reactions. Watch the presentation and tech demo again- do they even once suggest that Milo is dynamically figuring out how to respond to the users actions? Not in the slightest.

      What Natal will be able to do however is allow players to use more than just their thumbs in games- there's no reason you can't play Call of Duty like you always have with the controller but also now be able to physically lean left or right to peak round corners of buildings and so on. In games like Fable there's no reason you wont be able to make gestures like pretending to throw a ball to make your character throw a ball for your dog. There's no reason you wont be able to make gestures to draw things which can be turned into objects ala Scribblenauts and Crayon Physics. It does one thing and does it well- opens up the console for much more immersive controls just as the Wii did and just as Sony's new motion control system also will, the only difference being that Microsoft have done it in a different way to Sony and Nintendo.

      All that said, I've no idea why you would think they haven't perfected motion and facial recognition for use in games, these have been solved problems for a fair while now to a level where they're useful in many games and applications and as pointed above, the same goes for voice recognition. Natural language processing and natural speech

  4. Sounds pretty cool by Reapman · · Score: 4, Interesting

    Natal sounds cool, and might just be what i'm looking for for a reason to buy a 360 to compliment my PS3. I'm still concerned about the complete lack of buttons, and am wondering if MS is going to be releasing a controller you can strap to your arm or hold with 1 hand for when you need a button to say change weapons. Also the lag issue remains to be seen. This is one possible advantage Sony's motion MIGHT have.

    But in the end it's all about the games. Both techs have the potential I think to be fun, just need to see the actual implementation. And I wonder what Nintendo has in their R&D to combat this (other then that Vitality Sensor) All very interesting, although I don't see this gameplay replacing old fashioned "sitting down with a controller in your hand" gaming, just another form of entertainment.

    All of this talk makes me pretty excited for what comes after this current gen...

    1. Re:Sounds pretty cool by Dalambertian · · Score: 2, Interesting

      Despite what they keep showing in the press releases, I don't think the gesture control is what the big titles will be using it for. Remember Johnny Lee's wiimote hack at TED? http://blog.ted.com/2008/04/wii_remote_hack.php According to his blog, he's been working on Natal and seems very impressed with the specs. http://procrastineering.blogspot.com/2009/06/project-natal.html Anyway, I'm guessing it's new camera controls like his that would really sell this thing for the "serious gamers" and not necessarily the motion sensing.

  5. Re:I just want to try it by flitty · · Score: 2, Interesting

    The article states the latency for the camera to recognize a new position is 10 miliseconds, which IIRC is close to what wireless controller lag is. The better complaint is that it is only accurate to about 1.5", which is fine for large gesture based gaming, but minor refined movements won't be picked up.

    --
    Whether or not there is some sort of god, I'm not supposed to say/god is a word and the argument ends there-Smog
  6. Accuracy by chelberg · · Score: 2, Insightful

    In my mind what will affect user experience the most is its accuracy, and latency. It uses a infrared camera, so I'm wondering if anyone knows the camera's specs, especially regarding resolution and latency. Without knowing these, it is difficult to evaluate what it is good for. The article quotes a several cm. accuracy, and 10 ms recognition time, but it not clear what the sensor's limits are, and how much latency there is in the processing.

  7. Play with the spider woman by Quiet_Desperation · · Score: 4, Funny

    The article says that 31 limbs are tracked

    So even many-limbed deities like Vishnu and various spider gods can play? That's so thoughtful of Microsoft to include the divine.

    1. Re:Play with the spider woman by __aapspi39 · · Score: 2, Informative

      Vishnu didn't actually have more than the usual complement of 4 limbs; it's just Indian depictions of their deities and gods often use more than one pair of arms to symbolise different aspects of their being.

      On the subject of Natal, I'm very skeptical of anything that has Peter Molyneux involved - even before the Milo videos turned out to be fraudulent i was very suspicious.

  8. Re:Debate? I gots yer debate! by phigmeta · · Score: 3, Insightful

    Considering the Wii gang is bunch of 7 year olds, girls who don't want to break their nails, and some gay guys who just finished a sweaty game of wii cheer 2..... I think we will be fine.

  9. What I want to see is room lighting requirements. by gmezero · · Score: 5, Interesting

    Given how every other video game system camera fails to work correctly in typical home lighting environments, this is the thing I'm most curious about.

    For instance in my home, when we want to use the Eye Toy to play something like Kinetic, we have to drag out a shop-light and a couple of reflectors to stage the room and assist the contrast detection. Otherwise, the accuracy is garbage. If this thing can't work in a dark room then this also pretty much kills playing games in a room with the lights out (which is how I generally play racing games and FPS games at night).

    My expectations are less than low and I'm just waiting to hear about how ever single player to use the system will need to have a Live account (with your avatar adjusted to R/L body mass).

  10. Sony also has better accuracy by SuperKendall · · Score: 3, Informative

    Sony's motion tracking technology also reportedly has sub-millimeter accuracy, which could be better for games (along with the fact that controllers could have real buttons which would give you finer action control while moving).

    I'm still dubious if either system really ends up being better for games than the Wii though. It's fun to flail around for a while, but the Wii is nice in that for really long gaming sessions you don't have to exhaust yourself with full-body motion.

    --
    "There is more worth loving than we have strength to love." - Brian Jay Stanley
  11. Multiple people by KingSkippus · · Score: 4, Funny

    31 limbs isn't out of the question if it's something like a multiplayer party game. That could be, for example, seven people, three of whom are men.

  12. OBPennyArcade by schon · · Score: 5, Funny

    I think Natal is amazing

    I dunno.. I'm kinda scared of this: http://www.penny-arcade.com/comic/2009/6/5/

  13. Re:awesome by Toonol · · Score: 2, Insightful

    There are a lot of Wii motion-based games that work well. The (numerous) ones that DO suck, would probably have been just AS terrible with a controller, because they were cheap cash-in titles.

    I agree that motion sensing won't kill controllers; but it will become an essential feature. Would you like to go back to using your computer with no mouse?

  14. Amputees by Anonymous Coward · · Score: 2, Interesting

    What is someone was missing from an elbow down? Would the system see that as someone with it pointing directly at the device?