Slashdot Mirror

← Back to Stories (view on slashdot.org)

Kinect Hack Builds 3D Maps of the Real World

Posted by CmdrTaco on from the skipping-straight-to-six-d dept.

Lanxon writes "Noted Kinect-tinkerer Martin Szarski has used a car, a laptop, an Android smartphone and the aforementioned Xbox 360 peripheral to make a DIY-equivalent of Google Street View. The Kinect's multi-camera layout can be used to capture some fuzzy, but astonishingly effortless 3D maps of real world locations and objects. As we saw in Oliver Kreylos' early hack, you can take the data from Kinect's depth-sensitive camera to map out a 3D point-cloud, with real distances. Then use the colour camera's image to see which RGB pixel corresponds to each depth point, and eventually arrive at a coloured, textured model."

70 comments

  1. I wonder how it picks up... by Anonymous Coward · · Score: 0

    astroturf.

    1. Re:I wonder how it picks up... by Anonymous Coward · · Score: 0

      I'm sure it'd recognize your post just fine!

  2. Wow by MonsterTrimble · · Score: 1

    That is a cool hack. I wonder if Microsoft had any idea the Kinect would become such a cool hacker piece of equipment?

    --
    I call it 'The Aristocrats'
    1. Re:Wow by Anonymous Coward · · Score: 0

      I think they already let that cat out of the bag. Kudos for whoever advocated for or let this slip at Microsoft.

      I'm sure the business model was to use this to drive XBox 360 sales. But if sales outside of this are significant enough (noticable), they just might let it keep going.

      Instead of DRM, though, they'll just use their patents to keep it from getting TOO big and out of control.

    2. Re:Wow by Anonymous Coward · · Score: 0

      WTF are you talking about? Everything they've done since release has been encouraging to the Kinect hackers, and they've made quite a number of press releases to that effect. While it was certainly not their original intent, the Kinect is sold for a profit and thus there are no objections by Microsoft to people using it for other purposes. There are even rumors of an SDK coming out in the near future.

      On a side note, Microsoft loves to tout how many Kinects they've sold, particularly in attempts to attract developers to the platform. While I'm sure the current "Kinect hackers" make up a tiny percentage of sales, it will be interesting to see what happens if this trend continues.

    3. Re:Wow by Is0m0rph · · Score: 1

      MS has announced that releasing their own driver for the PC and Kinect in the future is likely.

    4. Re:Wow by Dunbal · · Score: 1

      How do you expect them to use patents, exactly? If you paid for the Kinect and incorporate it in your own project, Microsoft doesn't acquire any "rights" over your work. What, you think that seat belt manufacturers get a cut from every commercial flight?

      --
      Seven puppies were harmed during the making of this post.
    5. Re:Wow by Zerth · · Score: 1

      WTF are you talking about? Everything they've done since release has been encouraging to the Kinect hackers, and they've made quite a number of press releases to that effect.

      Except for the first one, where they responded to news of PC Kinect drivers with "Microsoft will continue to make advances in these types of safeguards and work closely with law enforcement and product safety groups to keep Kinect tamper-resistant."

    6. Re:Wow by bami · · Score: 1

      That's just a knee-jerk reaction from the legal department.

      MS Employee 1: Hey have you heard? They hacked the kinect to do some cool stuff.
      MS Employee 2: Sweet
      Lawyer 1: HACKED? THEY ARE HACKING OUR SYSTEMS, SUE SUE SUE.
      Laywer 2: S&D tube one: fire!

      Once they all got the memo that the hardware guys built the thing to be toyed around with (and seeing the hardware in the thing, they make some profit on it, selling it for $150, add that to the fact that it only has authentication for use with the xbox, and outputs unencrypted data), they issued the statement that they were allright with it.

      Cool thing about the Kinect is that it gives developers a platform, combining thinking power. Everybody can hook up a couple of cameras and some leds to a computer, but now this 3D sensing thing has a standard and everyone works with the same thing, pretty much like when they were hacking away with the wii-motes, which Nintendo didn't object to either (and sold them some wiimotes, with no harm done to the system itself). Microsoft just tries it that way, and I hope that more companies follow suit. It's out hardware damnit, I want to do with it as I please!
      I'm thinking of buying one for myself just to toy with it, but can't really justify the price just yet though :/

    7. Re:Wow by tehcyder · · Score: 1

      I'm sure the business model was to use this to drive XBox 360 sales. But if sales outside of this are significant enough (noticable), they just might let it keep going.

      Don't be an idiot, the sales outside of use with the XBox 360 will be minuscule in number, but can be used to create some good PR with no real downside for Microsoft.

      --
      To have a right to do a thing is not at all the same as to be right in doing it
  3. Actual blog post by icebraining · · Score: 5, Informative

    Link to the actual blog post: http://blog.decoratorpattern.com/2011/01/23/real-world-mapping-with-the-kinect/

    --
    Dilbert RSS feed
    1. Re:Actual blog post by Anonymous Coward · · Score: 0

      you post a good link for the top is broken +++ if i wish to moderate you when god approves

    2. Re:Actual blog post by Idbar · · Score: 1

      Also hackaday.com has some interesting projects about it. So far I remember this one among those trying to make the kinect portable to gather data.

  4. Eh that's nothing by kellyb9 · · Score: 1

    I like to play Kinect Sports when I'm driving.

  5. Scary applications by Anonymous Coward · · Score: 0

    How about a home brew cruise missile? Couple kinect and a gps device to a small quadrotor craft. Add explosives or maybe a low-caliber gun. Fly to location, blast away. Seriously scary... /captcha: headroom

    1. Re:Scary applications by Umuri · · Score: 1

      Umm, why a kinect? Any camera could give you basic visuals, and likewise an IR rangefinder would be much more useful at avoiding colissions on your path, the gps gives you position data.

      I just don't see how this is new

      --
      You never realize how much manually made unmanaged "linked" lists suck, till you have src.link.link.link.link...
    2. Re:Scary applications by intellitech · · Score: 2

      Not necessarily new, but cost-reduced and accompanying open-source code for effective use.

      --
      vos nescitis quicquam, nec cogitatis quia expedit nobis ut unus moriatur homo pro populo et non tota gens pereat.
    3. Re:Scary applications by IronHalik · · Score: 1

      rc heli wit a real gun Nuff said ;>

    4. Re:Scary applications by TheSync · · Score: 4, Informative

      an IR rangefinder would be much more useful at avoiding colissions on your path,

      The Kinect is an "IR rangefinder", but a cheap mass-produced one. Unlike the more expensive laser scanned parallax or time-of-flight sensors, it uses a special diffraction grating to produce a "structured light field" for a 2D camera to measure localized parallax.

      There are also cheaper pulsed IR time-of-flight depth sensors coming on the market for home use, these could have higher spatial resolution.

      The cool thing is that these things don't cost tens of thousands of dollars.

    5. Re:Scary applications by Anonymous Coward · · Score: 0

      GPS is accurate within what, 5-15 feet depending on strength and clarity? If you're attempting to actively track positional data as described in the link, GPS is not a viable option. Accelerometers and gyros are best the most accurate solution.

      I agree with your comments regarding an IR rangefinder and video camera however the Kinect does simplify the problem by streaming both sets of data through a single device in sync with one another. It's a proof of concept. I'd like to see others greatly expand on it.

    6. Re:Scary applications by techSage · · Score: 1

      I believe that software algorithms are able to be used to "throw out" "dirty" data points so that your resulting 3D pointcloud is clean enough for most practical (non-scientific) uses. Or so I've been told by those performing such operations at certain national institutions. I think they know what they are talking about.

    7. Re:Scary applications by Anonymous Coward · · Score: 0

      Sure, GPS is only good for absolute positioning down to about ten feet, but relative positioning (which is relevant to the current scenario) is much better.

      Is it good enough? I don't know. But it should be good for better than one foot resolution.

    8. Re:Scary applications by Missing.Matter · · Score: 1

      The cool thing is that these things don't cost tens of thousands of dollars.

      Yeah, that's the great thing about them, but the downside to this sort of sensor is that it doesn't scale well in size. That is, you can't use more than one kinect at a time, as the structured light fields from one will interfere with another. I suppose you can create some timing where one field will be off while another is on, but you can only have so many kinects in an area before this is impractical.

      Of course it's not a problem for gaming in a living room, but it if you want to do any sort of multi robot navigation you'll have to resort to the sensors that cost thousands of dollars.

    9. Re:Scary applications by tlhIngan · · Score: 1

      Yeah, that's the great thing about them, but the downside to this sort of sensor is that it doesn't scale well in size. That is, you can't use more than one kinect at a time, as the structured light fields from one will interfere with another. I suppose you can create some timing where one field will be off while another is on, but you can only have so many kinects in an area before this is impractical.

      Well, two Kinects do interfere, but not much - http://www.engadget.com/2010/11/29/two-kinects-join-forces-to-create-better-3d-video-blow-our-mind/

      Now, it's true that it doesn't scale as adding additional kinects will cause more interference, but two well-positioned ones can be made to not interfere too badly...

    10. Re:Scary applications by batkiwi · · Score: 1

      I think that you, like many people who enjoy bashing popular things, haven't actually done any reading to learn what you're talking about.

      Kinect IS a camera with a 640x480 (not simply a swiviling spot rangefinder) IR rangefinder built in. And it's well under $200. And it supports USB using a protocol so simple an outsider, with no documentation, was able to build a driver in 1 week.

      So what competes with it? Where is a 640x480 IR rangefinder for under $200 with an easy to use interface (not a single spot using a pedantic and difficult to debug 1wire interface), with or without an integrated color camera?

      Oh... you mean you assumed kinect was simply a $20 webcam rebranded and resold for $149 or whatever it goes for in the USA?

    11. Re:Scary applications by Anonymous Coward · · Score: 0

      "The cool thing is that these things don't cost tens of thousands of dollars."

      They do when you're buying from me.

    12. Re:Scary applications by TheSync · · Score: 1

      the structured light fields from one will interfere with another

      Indeed, I think a better solution is a time-of-flight sensor such as pulsed IR phase or flash LIDAR.
       

    13. Re:Scary applications by Missing.Matter · · Score: 1

      We use these sensors on our robots. They aren't very high resolution (64x48) but the're accurate and provide good pictures over time. The time of flight design makes the more practical than the kinect for our applications, but they're much more expensive.

    14. Re:Scary applications by Anonymous Coward · · Score: 0

      If you aren't from the Bay Area, then you should never use the word "hella". It always sounds wrong when outsiders try to use it.

    15. Re:Scary applications by tehcyder · · Score: 1

      why not just steal a fucking cruise missile and have done with it? moron.

      --
      To have a right to do a thing is not at all the same as to be right in doing it
    16. Re:Scary applications by sznupi · · Score: 1

      Still here? ;) ("wit"?...hm, that did end up rather curiously)

      --
      One that hath name thou can not otter
  6. Nifty! by Dreth · · Score: 1

    I can't imagine how this awesome will be with simultaneously-connected rotating Kinects.

    --
    All glory to Arstotzka!
  7. But will he WAR drive & snoop WiFi by mschaffer · · Score: 0

    It just isn't a real replacement to the Google Street View unless he snoops onto the WiFi.
    Of course, maybe he just doesn't realize that the software is there to do it ;-)

  8. Need for Speed Hot Pursuit - Kinect Version by riskeetee · · Score: 1

    The new version of Need for Speed Hot Pursuit uses the Kinect mounted on your car's dashboard. The chase scenes are way more exciting, and the cops use real bullets! Replayability, however, suffers greatly.

    1. Re:Need for Speed Hot Pursuit - Kinect Version by Anonymous Coward · · Score: 0

      Just switch over to pursuit mode!

  9. Mapsoria! by jimmerz28 · · Score: 0

    Awesome! Now we can make our own distributed map system that isn't governed by a central organization!

    Maybe we can call it Mapsoria?

  10. This is trivial by Missing.Matter · · Score: 1, Insightful

    This is trivially done with ROS and the Kinect stack. I went out and bought a kinect and plugged them into my robot platforms, provided a transform between robot base and kinect sensor, and was done. It's a great application and anyone who owns a kinect should try it out, but at this point it's trivial and hardly worthy of a /. post.

    1. Re:This is trivial by Anonymous Coward · · Score: 1

      I love that "trivial".....like my dad could just go out and do it in 3 clicks.
      Deutsche!

    2. Re:This is trivial by clyde_cadiddlehopper · · Score: 1

      It appears that putting working links in slashdot posts is non-trivial.

      --
      Obi-Wan: "I felt a great disturbance in the Force, as if millions of voices suddenly cried out in terror and were sudden
    3. Re:This is trivial by kiwimate · · Score: 4, Insightful

      Well, nifty. Good for you. Instead of acting super cool and belittling what someone else has done, can't you just say "nice one, and if readers are interested in this then look at what I did"?

    4. Re:This is trivial by Missing.Matter · · Score: 2

      It takes maybe 4 commands to get this working.

      sudo sh -c 'echo "deb http://code.ros.org/packages/ros/ubuntu maverick main" > /etc/apt/sources.list.d/ros-latest.list'
      apt-get install ros-cturtle-ros
      apt-get install ros-cturtle-kinect

      That will get the Kinect working. Then you just need a GPS and the appropriate transform between the GPS antenna and the sensor (easiest to put the antenna on top of the kinect), and you're done.

      And I don't see what me being German has anything to do with this.

    5. Re:This is trivial by Game_Ender · · Score: 1

      Not only is it trivial, it doesn't even run any optimizations passes over the resulting point cloud in order to reduce error and produce nice surfaces. This process is very similar to the work done here except the resulting solution will be quicker because you already have some decent depth information for the points.

    6. Re:This is trivial by Game_Ender · · Score: 1

      His point is that the research field has been doing this (3D reconstruction with range sensors) for years, so long in fact they have created nice open source solutions for it. The tech news sphere, and most hobbyist it seams, are completely ignorant of this.

    7. Re:This is trivial by QuantumG · · Score: 1

      It's a shame everyone is still looking for the low hanging fruit rather than putting in the hard effort required to make those 3d point clouds into actual surfaces and producing something that doesn't look like a pile of crap.

      Also, describing this as "google streetview like" is just stupid. It's a true 3d point cloud, not a bunch of panoramic images.

       

      --
      How we know is more important than what we know.
    8. Re:This is trivial by Missing.Matter · · Score: 1

      There's a whole lot of stuff like that going on. For example, the whole ROS PointCloud Library is a major effort focused on surface reconstruction, object detection, etc. You just don't see front page /. posts on the really cool stuff.

  11. Videogame peripherals by Dusthead+Jr. · · Score: 1

    The thing that I find cool about modern video gtame peripherals is that they seem more standardized than in the past (with the exception of the Sega Genesis/Atari 2600 controllers.) They either use USB or Bluetooth, and to the 360's credit you can still use its proprietary wireless pad on a PC.

    1. Re:Videogame peripherals by VortexCortex · · Score: 4, Informative

      The thing that I find cool about modern video gtame peripherals is that they seem more standardized than in the past (with the exception of the Sega Genesis/Atari 2600
      controllers.)

      Perhaps the connection is more standard, but I also see a fair amount of non standard shit going on, and not just in video game peripherals. The bundled Kinect doesn't come with the adapter so you can connect it to your PC (or older Xbox360). Don't fool yourself for one moment -- If MS could have ignored the existing XBox360 interface your Kinect they would have gladly made it more difficult to connect a Kinect to your PC.

      Xbox360 headsets have nonstandard pin-outs, so you can't use other PC headsets on on XBox or vise versa. Garmin's GPS uses a proprietary USB cable (having a small resistor across two pins), and refuses to charge when connected to a PC or via standard USB plugged into a voltage inverter. Zune uses a standard AV connection (Camcorder 3.5mm), but they swap the audio & video jack pins so that you have to plug white to yellow & vise versa, (many assume only the more expensive Zune brand cable works). Even the power cable on a Apple G5 I serviced last week was non-standard (-_-) instead of ('.'). The iPhone/iPad Touch & Zune use encrypted protocols (keys changed on each firmware upgrade) so you can't use them without the crappy bundled media manager software.

      They either use USB or Bluetooth, and to the 360's credit you can still use its proprietary wireless pad on a PC.

      The 360's wireless is NOT bluetooth. You must use MS's proprietary wireless receiver. The controller will not work though a blutooth receiver (built-in or external). The play&charge USB cable is for charging only, and won't allow you to connect it to your PC.

      Also, What's the use of a standard plug (USB), if the protocol or other proprietary quirks are introduced to make it incompatible (ala Zune, iPad Touch, Garmin resistor).

      I have to disagree with you on the standardization trend; To me, it seems that more companies are figuring out how to proprieterroize the "standard" connections; less of my "standard" cables work with my devices.

      PS: Just because the end that plugs into your PC is a standard USB connector, doesn't mean the end that plugs into your device isn't a proprietary shaped USB connector...

    2. Re:Videogame peripherals by Missing.Matter · · Score: 1

      The bundled Kinect doesn't come with the adapter so you can connect it to your PC (or older Xbox360).

      This wasn't a result of any sort of lockdown attempt. The Kinect has motors in it, which draw more power than USB can provide, so it uses a proprietary connector for the XBox. The adapter to use on the old Xbox requires an AC connection to provide the additional power.

      The 360's wireless is NOT bluetooth. You must use MS's proprietary wireless receiver.

      Which costs all of $10 on ebay. Anyway, it seems more efficient with power than USB; I've been using the same two batteries in my controller for... well I don't even know how long, they're that good.

    3. Re:Videogame peripherals by Missing.Matter · · Score: 1

      And by "more efficient with power than USB" I meant bluetooth.

  12. Stepping stone to the future by Anonymous Coward · · Score: 0

    The ability to take real-world 3D environments and throw them into a digital program for manipulation in a matter minutes is where this is being pushed.

    I wanted to do this almost exact scenario: Multiple Kinects on an telescoping pole (to use on scaled models vs. real buildings/hallways/rooms), connected to a local embedded device/pc, all rigged on an RC car that records the indoor mapping and converts it on-the-fly to a Blender compatible file. Remove SDHC card, plug in to computer, open/import Blender. Eureka!

    Do THAT, and voila! You can now create a Quake-3 map (insert game-of-choice here) in a fraction of the time it would take otherwise. Also, the applications for implementing real-world augmented reality aren't far off when coupling GPS with the above scenario. (I know a few people who are applying for grants on this specific implementation)

  13. 1MB test database? by Yvan256 · · Score: 1

    Why is that website asking permission to create a 1MB "html5 test db"? I can understand doing some testing, but on a live website?

  14. Awesome. by __aaelyr464 · · Score: 1

    I love seeing hardware being hacked and used for purposes other than what it was designed for. To me, that is the essence and lifeblood of true innovation and engineering.

    Sidenote: why is this article tagged "microshit"? Really?

    1. Re:Awesome. by RudeIota · · Score: 1

      [blockquote]Sidenote: why is this article tagged "microshit"? Really?[/blockquote]

      It's tagged !microshit, which means NOT microshit. ! is a reference to programmatical methods for representing "NOT" equal to.

      --
      Fact: Everything I say is fiction.
    2. Re:Awesome. by Anonymous Coward · · Score: 0

      The '!' in front of the microshit tag indicates that the tag is actually "NOT microshit", but that exclamation mark is easy to miss.

  15. Can it tell where my key is? by Anonymous Coward · · Score: 0

    If I put many kinects in my house as an array, create snapshots of the house of certain intervals, and be able to compare to one version to another, Will I be able to ask the array (or what ever is controlling it) where I left my keys in the house?

  16. Next logical Steps.... by Anonymous Coward · · Score: 2, Interesting

    Build a 3D model of your entire neighborhood.
    Convert it for use in your FPS of choice (Onslaught/ Tower defense style gameplay a plus).
    Add realistic weapon drops and "spawn" points.
    Drop in a few hundred AI bots, track their movements to see which positions are most/least effective.

    Congratulations, you now have a battle-plan for the zombie apocalypse.

  17. Game Map Makers Rejoice... by pRtkL+xLr8r · · Score: 1

    Wouldn't it be great - tweak this with GPS data or something similar, and then find some building that you think would be great for your multiplayer game. Simply walk around the building with a laptop and a Kinect, and after a few sweeps the software models the interior which you could use to import into your map editor...

    1. Re:Game Map Makers Rejoice... by noidentity · · Score: 1

      Not a good idea, unless the game involves fluffy bunnies and rainbows.

  18. A game controller by Anonymous Coward · · Score: 0

    Epic fail for Microsoft. They invent such a versatile device, and all they can think to do with it is use it as a game controller.

  19. Meh. by kuzb · · Score: 0

    Call me when a kinect or kinect-like device builds a 3D model that doesn't look like complete ass.

    --
    BeauHD. Worst editor since kdawson.
  20. "appropriate" (verb) by Anonymous Coward · · Score: 0

    not "proprieterroize" :-)

  21. How long until Google's driving down the street w/ by dwightk · · Score: 1

    Kinects?

    --
    Like anyone can even know that
  22. wordplay (n): A pun often missed by grammar nazzis by VortexCortex · · Score: 1

    proprieterroize (V): the introduction of proprietary or incompatible features to an otherwise standard product for the express purpose of limiting the customers' purchasing options in order to artificially drive up the product's cost.

    A portmanteau of proprietary and terrorize often used to describe vendor lock in strategies.

    "Although the two cables appear identical, I can't use a standard USB cable to charge my GPS; I have to use Garmin's proprieterrorized USB cable instead."

    ----

    Appropriation would mean that they took control of the standard USB protocol to bend all users of it to their ultimate will; Instead they simply produce incompatible USB cables for their own products (not all USB products) to bend consumers to their will and establish vendor lock in.

    It would be silly to think that a company would have to appropriate their own products. How does one forcefully gain control over something they already control?