Slashdot Mirror


Robot Swarm Control On Microsoft's Surface

zerOnIne writes "Dr. Mark Micire of UMass Lowell has built an intriguing new user interface on the Microsoft Surface, a multitouch-capable table computer. The interface is being used to control swarms of robots for disaster response, search, and rescue. One of the most interesting things about it is the intuitive tabletop joystick widget. Using a very fast hand-detection-and-identification algorithm, they can paint a touch joystick (dubbed the DREAM controller) directly underneath the hand. This joystick conforms to the size of the user's hand and tracks with hand movements, making sure that the control is always directly under the hand where the user expects it, even without haptic feedback. I've had a chance to go hands-on with this system, and I think it's truly remarkable."

28 of 106 comments (clear)

  1. Let me know when the price drops by IndustrialComplex · · Score: 3, Insightful

    Still somewhere over $5k?

    I'd love to build this into something... if I could sell it to someone other than businesses looking for a way to waste money.

    --
    Out of modpoints but really liked a post? 1BDkF6TtmmeZ3yqXbz9yhdYVqRYnwFoXDj
    1. Re:Let me know when the price drops by Anonymous Coward · · Score: 2, Insightful

      Surface uses infrared reflections to generate an image which is then processed. Alot of the challenge for a device of this type is the analysis of the IR, handling ambient light, and diffusion. If you were going to roll your own, I'd start with a project such as: http://digitalstratum.com/programming/ftir_build
      or a similar FTIR method. However, if you just want multitouch, then you've got other options such as project based (search for wii whiteboard).

      Surface provides a fairly substantial SDK with various multitouch controls, a simulator, and other such things. It also has a tag/barcode recognition system.

  2. Unbelievable by thePig · · Score: 2, Interesting

    This is amongst the coolest things that I have seen in a long time. Unbelievably cool and useful. Microsoft, I bow to you - the table seems to be the future of computing - if not amongst the masses - atleast amongst planners and decision makers.
    Although it makes me sad about my existence as a person though - doing a 9x5 job which is neither cool, nor very useful for humanity as a whole.

    --
    rajmohan_h@yahoo.com
    1. Re:Unbelievable by cptdondo · · Score: 4, Insightful

      It's a very, very cool front end. Where does it get the data in the real world?

      I'm running a construction job with 3 contractors. Today they had 8 excavators, 3 backhoes, half-dozen off-road trucks, a water tanker, a vactor truck, and probably a few things I've forgotten. They were hammering up concrete, ripping down a building, tearing up a dam, and moving an incredible amount of dirt. They still managed to dump the dirt in the wrong place.

      A webcam http://www.ci.springfield.or.us/millrace/images/ww.jpg and a cell phone do more for me than this.

      Don't get me wrong, I think it's cool as hell, but GIGO applies to this in spades.

      I'd love to see a real application for this; not a simulation.

    2. Re:Unbelievable by cptdondo · · Score: 2, Informative

      I've done disaster response. While robots may have some very limited applications, dogs and eyeballs and boots are better. You need lots of them and each one has to be thinking and looking on their own.

      You are looking for survivors and making decisions that are not quantifiable at all. It'll be a long time before we can send out a "swarm" of robots into a disaster area and have them work as well as a trained dog or a trained SAR person.

    3. Re:Unbelievable by Hunter0000 · · Score: 2, Informative

      The swarm of ground robots is a bit of a reach goal. Current applications of this include showing manpower and equipment (and current robotic resources, like UAVs). The idea being, issued orders to a person or vehicle's representation on the map will send orders to a communication device they posses. The map would also serve to display data that currently is compiled (slowly) onto paper maps. This way, the interface becomes useful for combining data and asset display that also allows orders to be issued to those assets.

      (I work next door)

    4. Re:Unbelievable by mmicire · · Score: 3, Interesting

      We have done this with real robots also. See links in posts below. Basically, if you had the ability to geo-locate your vehicles (via cell phone gps for example) and had a camera that could watch them, then you would have the circles drawn around them just like in this video. The simulation is on a completely separate server and is modeled as if it is being viewed from an overhead camera. So, the simulation is just for the robots. All of the registration and camera modeling is done as if it is real.

      We hope to be deploying this in the near future with real responders in the field.

      Hope that helps.

  3. Whats the worst that could happen? by gmuslera · · Score: 4, Funny

    Using a Microsoft product to control a swarm of (potentially killer?) robots?

    1. Re:Whats the worst that could happen? by Ethanol-fueled · · Score: 3, Insightful

      It's nothing new. They've been facilitating botnets for at least a decade now.

  4. NOG by Nethead · · Score: 2, Interesting

    Finally. A decent interface for C&C.

    --
    -- I have a private email server in my basement.
  5. Re:RTS games? by SharpFang · · Score: 3, Informative

    The problem with RTS games path finding vs robot path finding is that the games have a flawless description of the environment (the description IS the environment) and no inherent physical limitations on the driven device, only ones programmed in.

    A real obstacle can move, can be hard to spot or misrecognized as non-obstacle, can resist traditional methods of surmounting it (say, is slippery or crumbles). A real robot has to deal with traction slipping, in route deviating due to slipping on the surface, limited acceleration and braking power, environment behaving against specs (tell a crumbling building to follow the computational model...), communication shortages and so on.

    Also, this is a demo, to let people see how that works. I believe it could be done 20 times faster by an experienced operator doing actual work instead of a demo.

    --
    45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
  6. Is it just me? by dangitman · · Score: 4, Interesting

    Isn't the "swarm of robots" aspect slightly more interesting than the "touchscreen interface" aspect?

    --
    ... and then they built the supercollider.
    1. Re:Is it just me? by herks · · Score: 3, Informative

      The swarm stuff is neat, but has been done in RTS games. Nothing really revolutionary, except that it's never been done for swarms, using a touch table. The work defines some easy gestures and methods that have been studied and found to be what the majority of people would naturally do. What I find truly revolutionary about this interface is the dream controller. It doesn't get much play in the video, but it is remarkable. You put your hand down, and it draws the controller around it. It's exact every time. No matter what size hand, or orientation, it draws it perfectly for 100% of the human population. No more xbox/ps3 controllers that by their physical nature can only be built to be comfortable for the average. (50% of the population.) Having the tactile feedback of a keyboard or gamepad button has it's benefits, but if the controller is drawn directly under your hand, You can very quickly learn to press a button without having to feel it. Having this possibility opens up a whole new world of controller design with applications in video games, touch screens apps, every day appliances, and more. We can rethink controller design, and make the controller conform to the user, rather than the user conforming to the controller. I think it has a lot of potential, and am looking forward to seeing where this goes.

    2. Re:Is it just me? by Hognoxious · · Score: 2, Funny

      Not as frightening if you are sitting at the controls

      It might be a mistake in their programming. It might be due to a cunning ruse by the hero. But in the final scene, they always turn on their master. Them's the rules, see.

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    3. Re:Is it just me? by Jane+Q.+Public · · Score: 2, Insightful

      Here's what I perceive to be a real problem: from what I could see in the video, using the "Dream Controller" requires the constant and complete use of both hands. That drastically reduces its usefulness in the real world. As long as you are using the Dream Controller, you can't do ANYTHING else. And that's just bad. At least with a traditional joystick, you can at least temporarily work most functions with one hand. With other kinds of physical controllers like the Cyberman II you can control 6 complete degrees of freedom with one hand, while you temporarily do something else with the other (which is normally used to press buttons).

      I'm just not impressed.

  7. VIRTUAL robot swarm control on MS surface by Michael+D+Kristopeit · · Score: 3, Insightful

    NOTICE: no robots swarms were actually controlled in the making of this story

    1. Re:VIRTUAL robot swarm control on MS surface by zerOnIne · · Score: 2, Informative

      Not in this video, but they have controlled real robots, too:

      http://www.youtube.com/watch?v=48A8vdJ68lI

      --
      09
  8. Re:RTS games? by Psaakyrn · · Score: 2, Insightful

    The whole point of this is to micromanage troops. Sure, you can control large numbers of units efficiently using that, but can you control a trio of SCVs to heal each other, another 4 to poke a group of irritating zergling, swap roles to make sure all of them receive no casualties, and initiate an SCV rush on a zerg base? (well, unless you're korean)

  9. Let Me Think, m$'bots Walking Around A City by LifesABeach · · Score: 3, Funny

    I don't know weather to laugh or scream.

  10. Controlling Home Robots from "Smart Surface" by HizookRobotics · · Score: 2, Informative

    In case you're interested, there has been some nice work in using tabletop interfaces (ie. Microsoft Surface) to control home robots, like the iRobot Roomba.

  11. Mod This Shit Down by Anonymous Coward · · Score: 4, Insightful

    Yeah, I saw that episode of Saturday Night Live too. I laughed at it. I enjoyed it. With me so far?

    It aired April 8, 2000, more than ten years ago. It got old. You're not funny or clever. You're not some kind of deep, cultured, worldly guy. You're some dude on Slashdot who's repeating something from ten years ago that was amusing but not all that funny in the first place. It was a bit funny back when that ugly fuck Christopher Walken said it and it was a new skit. It's repetitive now.

    You might as well parrot another predictable, cookie-cutter, assembly-line Slashdot meme. The only difference is that it would be a Slashdot meme and not a pop culture meme. Nothin' personal. Just that Slashdot will be greatly improved when we stop celebrating this bullshit.

    1. Re:Mod This Shit Down by Ihmhi · · Score: 4, Funny

      This rant needs more cowbell.

      Can do!

      Yeah, *tunk* I *tunk* saw *tunk* that *tunk* episode *tunk* of *tunk* Saturday *tunk* Night *tunk* Live *tunk* too. *tunk* I *tunk* laughed *tunk* at *tunk* it. *tunk* I *tunk* enjoyed *tunk* it. *tunk* With *tunk* me *tunk* so *tunk* far?

      It aired *tunk* *tunk* April 8, 2000, more than ten years ago *tunk* *tunk* *tunk*. It got old *tunk*. You're not funny or clever *tunk* *tunk* *tunk*. You're not some kind of deep, cultured, worldly guy *tunk* *tunk*. You're *tunk* some *tunk* dude *tunk* on *tunk* Slashdot *tunk* *tunk* *tunk* who's repeating something *tunk* from ten years ago *tunk* that was amusing but *tunk* not all that funny *tunk* in the first place *tunk* *tunk*. It was a bit funny back when that ugly fuck *tunk* Christopher Walken said it and it was a new skit. *tunk* It's repetitive now. *tunk* *tunk* *tunk*

      You *tunk* might *tunk* as *tunk* well *tunk* parrot *tunk* another *tunk* predictable, *tunk* cookie-cutter, *tunk* assembly-line *tunk* Slashdot *tunk* meme. *tunk* The *tunk* only *tunk* difference *tunk* is *tunk* that *tunk* it *tunk* would *tunk* be *tunk* a *tunk* Slashdot *tunk* meme *tunk* and *tunk* not *tunk* a *tunk* pop *tunk* culture *tunk* meme. *tunk* Nothin' *tunk* personal. *tunk* Just *tunk* that *tunk* Slashdot *tunk* will *tunk* be *tunk* greatly *tunk* improved *tunk* when *tunk* we *tunk* stop *tunk* celebrating *tunk* this *tunk* bullshit. *tunk*

      (*tunk*.)

    2. Re:Mod This Shit Down by lgw · · Score: 3, Insightful

      You might as well parrot another predictable, cookie-cutter, assembly-line Slashdot meme. The only difference is that it would be a Slashdot meme and not a pop culture meme. Nothin' personal. Just that Slashdot will be greatly improved when we stop celebrating this bullshit.

      In Soviet Russia, cowbell needs more Natalie Portman!

      Yes, yes, memes are lame. You're not some kind of deep, cultured, worldly guy for pointing that out. Being lame is sort of the point of memes. We're a bunch of lame nerds sharing a silly, arbitrary common culture. Sorry, but it's what we do.

      Kirk would totally kick Worf's ass. Emacs > VI! Dr. Who is better than Star Wars or Trek! This is an ex-Parrot! Mandatory XKCD reference!

      Yeah, it's all lame repetition of old stuff, some of which wasn't even that great to begin with, but, hey, that's culture for you.

      --
      Socialism: a lie told by totalitarians and believed by fools.
  12. EULA? by Mr.+Freeman · · Score: 3, Funny

    Microsoft surface being used for search and rescue. Doesn't the EULA specifically state "not to be used for life saving operations or operations in which failure of the system could result in bodily injury"? A search and rescue robot certainly qualifies as a life saving device, the failure of which could result in people being dropped or crushed.

    I wonder why the researchers that made this felt that disregarding the EULA was acceptable.

    --
    -1 disagree is not a modifier for a reason. -1 troll, flaimbait, redundant, overrated are NOT acceptable substitutes.
  13. Re:RTS games? by SnowZero · · Score: 3, Informative

    This demo was a simulation, so there really isn't that much difference from a good RTS. Motion control and local path planning are well understood.

    While the demo looks cool, it is not really much different than the robot GUI I wrote for my robocup team that could control 10+ robots (real or simulated). It used a mouse and any number of ps3-style controllers. And yes, I got my ideas from RTS games, and some other teams had even better GUIs.

    Also, it would have been nice if they didn't speed up the video, so we could better understand how well it worked. One thing I often find in touch interfaces is that selection is slower than you'd expect since you block the item with your finger. A mouse pointer is much smaller, and works well for skilled operators.

  14. Re:Boooring by macshit · · Score: 5, Insightful

    "Surface" is yet another piece of slightly-interesting-in-theory-but-kind-of-meh-in-reality tech that microsoft has been trying to get people enthused about for ages -- and consistently failing. You know, kind of like tablet computers before Apple actually made people want them.

    I wouldn't be at all surprised if Apple suddenly teams up with Ikea and has every living room in the country computing with their coffee-table within 3 months though.

    --
    We live, as we dream -- alone....
  15. Re:RTS games? by icebike · · Score: 2, Insightful

    The problem with RTS games path finding vs robot path finding is that the games have a flawless description of the environment (the description IS the environment) and no inherent physical limitations on the driven device, only ones programmed in.

    Well, that and the utter LACK of any ACTUAL ROBOTS to guide. (Or those few that do exist are so damn expensive nobody can afford a swarm of three).

    Oh, and those Surface tables... Those aren't exactly laying around anywhere either.

    But overlooking that minor obstacle for the moment, the first thing that goes to hell when you actually NEED a robot swarm is that flawless environment description. And of course situations needing swarms never occur where you have maps, they happen annoyingly in some remote area, immediately adjacent to (but outside of) the area your funding covered.

    Perhaps thats why the summary is so ga-ga about the joy stick simulation, because its the only part of this that seems remotely usable on Joe SixPack's touch screen device.

    --
    Sig Battery depleted. Reverting to safe mode.
  16. I can picture it now by Hognoxious · · Score: 3, Funny

    1) User swats away a fly
    2) A thousand robots charge off the edge of a cliff
    3) ???
    4) *Sigh* Back to the drawing board

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."