Slashdot Mirror

← Back to Stories (view on slashdot.org)

Open Source Gesture Recognition For Kinect SDK

Posted by timothy on from the everybody-will-have-been-kung-fu-fighting dept.

Rymix writes "I have been working with a new internet-friend of mine to produce an open source gesture recording and recognition engine for Kinect SDK. It's based on the Dynamic Time Warping technique and allows developers to record their own gestures and reliably recognise them. It's currently 2D but 3D is an easy development, coming soon. We're looking for community take-up and contribution to this project — it could help a lot of people with rapid prototyping and could even be used in production solutions (within the Kinect SDK's terms of use, of course!)."

25 comments

  1. If this gets a 'hardhack' tag... by Anonymous Coward · · Score: 0

    ..you will all die. Nuclear-style.

  2. Open Source and can only be used with the SDK EULA by h4rr4r · · Score: 0

    Open Source and can only be used with the SDK terms, seems pretty pointless. Probably would be better to work on supporting the device on an actual Open stack.

  3. Needs more work by PCM2 · · Score: 2

    Initial testers found only one gesture they could make at the Microsoft device that triggered any response.

    --
    Breakfast served all day!
  4. Re:Open Source and can only be used with the SDK E by Anonymous Coward · · Score: 0

    Yeah, how dare you release something in a manner that h4rr4r doesn't approve! This is a total affront!

  5. What's the big deal? by PCM2 · · Score: 1

    In all seriousness, why all the stories about the Kinect? I don't play videogames, so that aspect doesn't interest me. Getting it to do stuff it wasn't made to do is a cool hack, and I can appreciate that. I can picture some important applications for the disabled, I guess. Is there anything more to it than that? Am I missing something?

    --
    Breakfast served all day!
    1. Re:What's the big deal? by Zerth · · Score: 1

      A similar device to do 3d vision for robots used to(still does) cost at least in the mid thousands if not low tens of thousands of dollars.

    2. Re:What's the big deal? by Anonymous Coward · · Score: 0

      porn?

    3. Re:What's the big deal? by AdmiralXyz · · Score: 4, Informative

      Every single time there's a Kinect story, someone has to ask this question. My cynical side wonders whether they'd still be doing this if the Kinect wasn't a Microsoft product, but here's a comment I posted a while back that explains why, even though it may be overrhyped as a video game peripheral, the Kinect is damned cool for robotics applications. The robotics community has, almost overnight, dropped work on many other kinds of input sensors to focus on the Kinect because it's so much more useful (see openni_kinect on ROS for the primary driver, although we'll see if this continues to see love now that an official SDK is out).

      --
      Dislike the Electoral College? Lobby your state to join the National Popular Vote Interstate Compact.
    4. Re:What's the big deal? by Lunix+Nutcase · · Score: 1

      Good point. We should have all the people currently hacking on the Kinect or using the SDK to just stop all work and abandon it all because PCM2 can't see any important applications for the hardware. With such definitive evidence on the uselessness of the Kinect, this SDK or any apps currently written or being worked on, I can't see how any could argue against these conclusions.

    5. Re:What's the big deal? by PCM2 · · Score: 2

      The robotics community has, almost overnight, dropped work on many other kinds of input sensors to focus on the Kinect because it's so much more useful

      First I've heard of it. The way the summaries are always worded, to the uninitiated it always sounds like the same story: Kinect hacker hacks Kinect!

      --
      Breakfast served all day!
    6. Re:What's the big deal? by PCM2 · · Score: 1

      Good point. We should have all the people currently hacking on the Kinect or using the SDK to just stop all work and abandon it all because PCM2 can't see any important applications for the hardware. With such definitive evidence on the uselessness of the Kinect, this SDK or any apps currently written or being worked on, I can't see how any could argue against these conclusions.

      My God, you have read my mind! You should find some scientists, I'm sure there is important research to be done on your uncanny abilities.

      --
      Breakfast served all day!
    7. Re:What's the big deal? by Dr+Herbert+West · · Score: 2

      Mod parent up, please. I'm getting sick of the flurry of "you made something with your own two hands? color me unimpressed" type comments on Slashdot anytime there's a story about kinect hacking or other Maker type stuff. I got into this field because I liked to break/take apart/repurpose stuff as a kid, and these experiments satisfy that urge today. Not everyone comes to this site to bitch about video drivers or start OS platform flame wars. /end rant.

      In response to AdmiralXyz's comment, I'm not really seeing so much work being done with the SDK that's not video game related-- openNI has been around for a while, and it's cross platform-- even runs on Linux I believe. The "weird kinect experiments" community seems to have embraced openNI as well as openKinect (a wrapper for Processing/Java), a trend that I would like to see continue.

    8. Re:What's the big deal? by Lunix+Nutcase · · Score: 1

      So then what exactly was the point of your post?

      Important News Bulletin: PCM2 has no idea for how the Kinect could be used for interesting or important applications! Film at 11.

    9. Re:What's the big deal? by PCM2 · · Score: 1

      Apparently, despite having the same punctuation, the point of your post was not the same as the point of my post, which was to ask a question.

      --
      Breakfast served all day!
  6. Lol, you guys are funny by Rymix · · Score: 2

    There are several reasons for this study: 1) It's a component of an academic study into the effects of 'natural' human-computer interaction on learning and teaching. There sure is a lot of buzz about Kinect at the moment, so studies like this will help to prove just how useful gesture controls could really be. Or not. Open mind... 2) Dynamic Time Warping is a viable but relatively under-explored method for vector-based gesture recording and recognition. It's interesting to see how well this can perform against more established methods. 3) There's not much out there for the Kinect SDK yet, so why not build something nice, like? If it proves to be popular I will refactor into an open-standards form and release for all sensor applications. 4) It's a very immature product and needs a lot of work before it's really useful - hence the open source. 5) Kinect SDK's license is currently research only, but I have it on good authority that this will change soon. I'm not quite sure what to, though. 6) Working on stuff like this is, to me, like playing with Lego. I just enjoy it. I enjoy creating something, investigating, exploring...don't like that? I'm cool with that. But don't bite me for it. 7) I just realised this numbered list could look rude or arrogant. Sorry about that. But I couldn't figure out how to get carriage returns into comments. 8) Another point of KinectDTW is that you can teach it you own gestures. Try it, and if they're unreliably recognised, try tweaking the parameters for your environment. A future development would see dynamic parameters per person/room/gesture for maximum reliability. 9) There is no 9) 10) If you like it, or even if you don't, please keep the feedback rolling in. I like to read you opinions :) And if you can, contribute. I'd really appreciate better minds than mine working on this too.

    1. Re:Lol, you guys are funny by sonamchauhan · · Score: 1

      Someone mod parent post up please ... its the original story poster.

      And my feedback. The Kinect is a potentially great piece of hardware but...
      Being protected from a potentially eyeblinding 60 mW infrared laser, with only crackable, diffraction gratings to disperse the beam across the room is a bit too risky for me.

      This video shows that accidentally bringing your eye one or two inches, even with the Kinect optics intact, can blind you.
      http://www.youtube.com/watch?v=7qLDzLYPG-w

      Think of when you may get your face accidentally too close... like when switching something on or off in AV equipment... or worse, children starting into the projector.

      Balanced discussion here: http://skeptics.stackexchange.com/questions/2625/is-the-kinect-ir-laser-safe
      (see the end of the page, where I found the YouTube link above)

    2. Re:Lol, you guys are funny by Rymix · · Score: 1

      Very interesting comment. I can't claim to know anything at all about this device's safety (or lack thereof) but I will certainly look into it. If you'll excuse the pun. Steve

    3. Re:Lol, you guys are funny by sonamchauhan · · Score: 1

      Heheh - good pun. :)

  7. Great... but... by Anonymous Coward · · Score: 0

    ... Can I do some Minority Report level shit yet on my Linux box?

  8. Re:*snore* by Rymix · · Score: 1

    You rock. Not much, though.

  9. Missing Gesture Lib by BSalita · · Score: 1

    This software unblocks my use of Kinect. Microsoft has not released a gesture library. Looks like this will enable it's creation.

    1. Re:Missing Gesture Lib by Rymix · · Score: 1

      Awesome! Glad to be of help. It would be great if you shared your gesture files and settings...this is all rather experimental after all :)