Slashdot Mirror


Controlling Games and Apps Through Muscle Sensors

A team with members from Microsoft, the University of Toronto, and the University of Washington have developed an interface that uses electrodes to monitor muscle signals and translate those into commands or button presses, allowing a user to bypass a physical input device and even control a game or application while their hands are full. The video demonstration shows somebody playing Guitar Hero by making strumming motions and tapping his fingers together, a jogger changing his music without having to touch the device, and a man flexing a muscle to open the trunk of his car while he carries objects in both hands. The academic paper (PDF) is available online.

47 comments

  1. Insensitive by kamapuaa · · Score: 5, Funny

    Slashdot readers don't have any muscles, you insensitive clod!

    --
    Slashdot: providing anti-social weirdos a soapbox, since 1997.
    1. Re:Insensitive by Anonymous Coward · · Score: 0, Troll

      i can get my girl to take off all her clothes when i flex my muscle, you insensitive clod!

    2. Re:Insensitive by gravos · · Score: 1

      i can get my girl to take off every zig when i flex my muscle, you insensitive clod!

      Fixed that for you.

    3. Re:Insensitive by Anonymous Coward · · Score: 2, Funny

      Have you looked at your right arm recently?

  2. Bypass a physical input device by Anonymous Coward · · Score: 0

    A muscle sensor is no less a physical input device than a button. Handsfree, perhaps, but still a physical input device.

  3. Finally. by Anonymous Coward · · Score: 0

    Now I can jack off and spawn camp at the same time, at least for 2 or 3 minutes.

  4. Sensitivity by girlintraining · · Score: 3, Insightful

    This works fine for off/on states, but not graduated ones where a range of input is needed. Muscles are binary -- they are off, or on. At least, at the cellular level. But when they're put in bunches, only some are activated while others are not, which leads to a range of possible force levels. Effectively monitoring neural activity here requires a large number of sensors to accurately determine how much force is being requested and then translate that into a digital representation. As well, do not forget that in the human body, motion is comprised of two separate inputs from the sympathetic and parasympathetic nervous system: And while complementary, these two are not always perfectly in balance. This is why prothetic limbs have to be computer-assisted and lack fine motor control: They simply can't get a good enough input resolution.

    So yes, it'll be great for mouse clicks (binary), but I'll still own your ass in a video game in anything that uses a vector (analog).

    --
    #fuckbeta #iamslashdot #dicemustdie
    1. Re:Sensitivity by Anonymous Coward · · Score: 0, Troll

      The saddest thing about Slashdot's mod system is that this piece of nonsense will almost certainly be moderated up to +5 either Insightful or Interesting. There really should be a -1 Misinformed mod.

    2. Re:Sensitivity by girlintraining · · Score: 0, Flamebait

      The sadder thing about slashdot's moderation system is it's made of a bunch of hateful asshats who mod me down because of who I am, not what I say. Everything I stated is correct, and if you want to disagree you can do it constructively by providing citations. But... I with an angry dyke on the loose, who has time to check their facts?

      --
      #fuckbeta #iamslashdot #dicemustdie
    3. Re:Sensitivity by Anonymous Coward · · Score: 0

      The sadder thing about slashdot's moderation system is it's made of a bunch of hateful asshats who mod me down because of who I am, not what I say. Everything I stated is correct, and if you want to disagree you can do it constructively by providing citations. But... I with an angry dyke on the loose, who has time to check their facts?

      wtf are you talking about.

    4. Re:Sensitivity by Anonymous Coward · · Score: 0

      The sadder thing about slashdot's moderation system is it's made of a bunch of hateful asshats who mod me down because of who I am, not what I say. Everything I stated is correct, and if you want to disagree you can do it constructively by providing citations. But... I with an angry dyke on the loose, who has time to check their facts?

      wtf are you talking about.

      wtf are you talking about?

    5. Re:Sensitivity by Anonymous Coward · · Score: 1, Informative

      Sorry, no one gives a rats ass who you are. You're just wrong. Muscles signals (EMG) are pulsed, and it isn't hard at all to get a range of values out of that (by counting pulses over time). My company has an EMG detector (that we built) and it's not hard at all to vary the amplitude of your muscle signal over a range.

    6. Re:Sensitivity by Anonymous Coward · · Score: 1, Informative

      Sometimes you are just plain wrong. And sometimes people call you out when you post utter bullshit. It isn't a gender thing. I didn't even notice your nick before this comment and I doubt most others noticed either.

    7. Re:Sensitivity by RobVB · · Score: 1

      So yes, it'll be great for mouse clicks (binary), but I'll still own your ass in a video game in anything that uses a vector (analog).

      But with this new technology, I can kick your ass in TetriNET while lifting weights!

      --
      I'd rather you rationally disagree than irrationally agree.
    8. Re:Sensitivity by Nathrael · · Score: 2, Informative

      Eh...paranoia much?

      --
      A good education is a bit like a STD - it makes you unsuitable for a lot of jobs and gives you a desire to spread it.
    9. Re:Sensitivity by beckett · · Score: 1

      anal sphincter? not sure i'd like the application of the input device though.

    10. Re:Sensitivity by beckett · · Score: 1

      it would make it much harder to lose the car keys though.

    11. Re:Sensitivity by MacJedi · · Score: 1

      No. You have some facts that are correct, individually, but you are drawing nonsensical conclusions. True, individual muscle fibers are either contracted or relaxed--on or off as you say, but surface electromyography records from far more than a single muscle fiber. So at the population level, measuring a graded response is not only possible, but typical. Furthermore, the signal recorded is roughly linear and proportional to the number of fibers and motor units recruited (let's ignore the differences between type I and type II skeletal muscle fibers for the moment). Your aside about the sympathetic and parasympathetic nervous system appears completely off topic.

      --
      2^5
  5. true mobile computing? by hitmark · · Score: 1

    so how long until the computer is a box on the hip/back/wherever, and the IO is a pair of semi-transparent oleds (see recent samsung and lg product demos) glasses and this worn up the sleeve somewhere?

    --
    comment first, facts later. http://chem.tufts.edu/AnswersInScience/RelativityofWrong.htm
    1. Re:true mobile computing? by mossy+the+mole · · Score: 1

      so how long until the computer is a box on the hip/back/wherever, and the IO is a pair of semi-transparent oleds (see recent samsung and lg product demos) glasses and this worn up the sleeve somewhere?

      Not with this technology, It seems fine for little things like the examples but not for anything major. Imagine writing an email with muscle movements (or doing the hokey-Cokey as is will appear)

    2. Re:true mobile computing? by hitmark · · Score: 3, Interesting

      i did a bit of math ones, as i considered doing a input glove using pressure sensors on fingertips.

      what i found that 2 to 4 finger combos should cover the keys of your average keyboard.

      that is, pressing between 2 and 4 fingers in sequence, before releasing one or more could act as a single key input.

      and this muscle sensor system could work the same way, as the thumb against other finger system could just as easy be replaced with a system where you have a rest state, and then a added strain state where the muscle controlling the finger is being tensed.

      i suspect that with training, a person could type out messages while gripping some object, simply by tensing the finger muscles in specific sequences, just like touch typing today, or for that matter playing a piano...

      --
      comment first, facts later. http://chem.tufts.edu/AnswersInScience/RelativityofWrong.htm
    3. Re:true mobile computing? by incubbus13 · · Score: 1

      Actually, I thought that you could create a + sign of keys for each finger. Which is essentially how it works for a keyboard, but there's more diagonals. Basically, each finger has five positions, up, down, left, right, and center/"home row". That allows you to type everything but 'z' with just quick flicks of your fingers in a variety of directions. Since writing is a linear means of information storage (you can only enter one character at a time, no matter how fast you can type, or otherwise it loses its value as a recording) it shouldn't involve any more complicated learning processes for us than typing now does. Just a new way of doing the same thing. But it does reduce a keyboard to 10-small divits in a piece of plastic, and a few sensors which really aren't any more complicated than current touch screen technology. And, as an added advantage, in 10 years, it should be relatively easy to track the positions of each finger in a dynamic/uncontrolled environment. Which means it should get easier and easier to completely do away with this "keyboard" and go with one of those laser-projected keyboard systems, thus reducing the size of hardware even further, by doing away with any kind of hardware keyboard at all.

      K.

  6. Moving by BluePeppers · · Score: 1

    Like the wii, touchpads, motion tracking and countless other control methods, this one fails to address a key issue: I don't bloody want to move my hands other than to type.

    --
    Penguins can be fascists too
    1. Re:Moving by Abstrackt · · Score: 1

      Like the wii, touchpads, motion tracking and countless other control methods, this one fails to address a key issue: I don't bloody want to move my hands other than to type.

      Looking at Wii sales I think it's safe to assume that there are people who do want this technology.

      --
      They say a little knowledge is a dangerous thing, but it's not one half so bad as a lot of ignorance. - Terry Pratchett
    2. Re:Moving by Zakabog · · Score: 1

      The Wii sells because it's cheap and the games are fun... well, fun for a little while. My Wii is currently sitting in my closet, haven't played it in a bit and I haven't wanted to. The controls didn't sell the system and there are some games where they really are annoying to use. I also have an XBox 360 and a PS3 that I play far more often, and if the Wii games I have were available on either one of those systems then I probably wouldn't have any games for the Wii.

    3. Re:Moving by incubbus13 · · Score: 1

      My concern with the Wii, as a gamer, has always been about the lack of quality. So many of even the PS3/xbox games are becoming 'rhythm games'. Which is code for button-mashing nonsense. In the PS3's motion controller, there's already games like Godfather (&2) where you have to fling the controller through space, while holding buttons, to get it to do something.

      All of these gimmicks, all of these excuses to skimp on game development, graphics, story, take away from games. The Wii's 'goal' is to sell gaming to a bigger market. To induct non-gamers into becoming gamers. Except. Those people aren't going to want deeply involving stories supported by state-of-the-art graphics and sound-scores, with ever-closer-to-reality physics engines. They just want the ball to bounce on the screen it a soft, gentle arc when they fling their Wii-daikatana around the room. Or the little cartoon character to jump (and giggle obnoxiously simultaneously).

      In other words, once we accept that motivated self-interest is the only way to get people to act, do, or perform in a certain way, Nintendo is created a motivated self-interest to create 'easy', 'fun', 'simple' games with a lot of pretty pictures. There's even apparently a cheat mode in the newest Wii that lets you skip parts of games that are too hard for little Johnny or Susie. And the market is about 100 times bigger than the market for CoD or GTA or anything else you'd care to play.

      This is compounded by the problem of market cohesion, there are fewer and fewer, but bigger and bigger, participants in the market. With Activision's, for instance, known history of screwing players, skimping on game quality (long term sales) for making E3 release dates (short term sales dollars), and other wise being 'in it for the money', how long is it before some big company realizes anybody with an AA in CS can write the equivalent of original Donkey Kong for the Wii, in two weeks and that at that rate, they can pump out 26 games for the 'casual gamer' a year from one code/dev team. /rant

      K.

  7. Isnt this just Electromyography? by Anonymous Coward · · Score: 0

    http://en.wikipedia.org/wiki/Electromyography

    http://en.wikipedia.org/wiki/Myoelectric_prosthesis

  8. obligatory... by Hybrid-brain · · Score: 0

    I for one welcome our new muscle controlling overlords

    --
    Five words describe me on a normal day. two words describe me the rest of the time. can you guess?
  9. I did it again... by pyr02k1 · · Score: 1

    One of these days, I'll stop opening my trunk every time I type something

  10. Re:you insensitive CNlod! by Anonymous Coward · · Score: 0

    Best Jack-o-Lantern ever.

  11. Cybering by Tibia1 · · Score: 1

    will be changed forever.

  12. Don't Taze M- grglgrgg *pop* by Makido · · Score: 1

    If this catches on I can retire my slimjim and lock picks and just get a tazer for all my wretched car-thieving ways.

  13. Physical Passwords by allknowingfrog · · Score: 0

    A couple of people have mentioned this already, but unintentional activation seems to be a likely problem. Maybe opening your trunk should require a series of motions, like a physical password of sorts, to make sure you're doing it intentionally.

    "Let's see...right foot in, right foot out, right foot in, shake it all about..."

    1. Re:Physical Passwords by Hal_Porter · · Score: 1

      Maybe opening your trunk should require a series of motions, like a physical password of sorts, to make sure you're doing it intentionally.

      "Let's see...right foot in, right foot out, right foot in, shake it all about..."

      How about the Truffle Shuffle?

      http://www.youtube.com/watch?v=t5whaRkuipU

      --
      echo -e 'global _start\n _start:\n mov eax, 2\n int 80h\n jmp _start' > a.asm; nasm a.asm -f elf; ld a.o -o a;
  14. I have one of these! by Anonymous Coward · · Score: 0

    An IBM Model M keyboard! Why, it changes small muscle movements in my forearms into letters you can read on the screen!. All without any fancy electrodes.

  15. Only on /.... by Anonymous Coward · · Score: 0

    Honestly, guys. If we go to that level, it shouldn't take long that we can monitor the tongue and vocal cords. Using those to produce words is superior to any cryptic 2 to 4 finger combos (requires less training, etc.). Only exception would probably be programming, etc. in which you write stuff that is slow to say out lout ("if($ > x) { ... }") but I am sure that you could make some shortcuts for that use.

  16. ihnrtfa, but... by Anonymous Coward · · Score: 0

    It sounds perfect for amputees. Guitar Hero without hands has got to be a total pain in the stump.

  17. Input device for a real Power Lifter by bughunter · · Score: 1

    Sounds like the ideal input device for something like this!

    --
    I can see the fnords!
  18. OCZ NIA by Anonymous Coward · · Score: 0

    The OCZ NIA already does this for face muscles (glance left/right and jaw muscle) and is years old.

  19. Atari invented this in 1984... by atarimuseum · · Score: 1

    What is funny is Atari did this back in 1984 with their "Mindlink" headband controller: http://www.atarimuseum.com/videogames/consoles/2600/mindlink.html It was quite good, too bad the company switched gears to primarily computers at the time and didn't go forward with the controllers. Its amazing how people forget about stuff that was invented decaded ago and announce them today as if they just discovered the wheel for the first time. Then you have the exact opposite - you have older technologies that have been reborn with a whole new purpose - take the old Atari joysticks from the 70's and 80's - just simple 5bit hardwired controllers, nothing really special in today's world, but they were the controllers by which all others were guaged in the 70's and 80's... Now there is a company who has brough those very same controllers back, but now they pack USB circuitry in them and can be used on PC's and Mac's - cool stuff indeed! http://www.legacyengineer.com/storefront Next thing you know, someone will bring Pong back and claim its this innovative immersive head to head combat game! ;-) Curt

  20. so what? by jipn4 · · Score: 1

    EMG sensors have been around forever; why would you want to attach them to healthy people? If you attach them to a functional muscle, you end up overloading functional signals in a way that's going to cause problems sometimes.

  21. Cool Stuff by nw-moose · · Score: 1

    This is some pretty cool looking stuff! The person talking in the video sounds kind of like Johnny Chun Lee (the guy that did the Wii Remote whiteboard & head tracking projects before Microsoft employed him), is he involved in this at all?