Slashdot Mirror

← Back to Stories (view on slashdot.org)

Optical Camouflage Puts Kinect Into Stealth Mode

Posted by timothy on from the where-are-you-anyhow dept.

UgLyPuNk writes "Takayuki Fukatsu, a Japanese coder who works under the name Art & Mobile, has done a bit of trickery with Kinect and openFrameworks. The peripheral will still track your movement and position, but turns your image nearly transparent. Take a look (it's particularly obvious at about 1:30):"

60 comments

  1. The background doesn't change by igreaterthanu · · Score: 5, Informative

    The article states that he hasn't said how it is filling in the gaps, it's really easy to fill in the changes in the background with a static image and this may be how he is doing it, I could achieve the same effect with a regular webcam.

    --
    I dream of a nation where a man is not judged by his skin color but by an number assigned by a credit rating agency.
    1. Re:The background doesn't change by Sockatume · · Score: 5, Informative

      It's a shade more sophisticated than that. I think he's using the existing make-a-texture-mapped-model-of-the-space code, but telling it to texture map anything that's non-background with an pre-existing image of the background. It's a cute project, obviously intended to recreate the look of the sci-fi cloaking effect, rather than do anything clever. After all, you could achieve a much more effective result by just replacing the feed with one in which the person was not present.

      --
      No kidding!!! What do you say at this point?
    2. Re:The background doesn't change by Pseudonym+Authority · · Score: 4, Insightful

      Sure, maybe you could. But you haven't, so don't be a dick.

    3. Re:The background doesn't change by ikkonoishi · · Score: 1

      Looks like it takes a static shot, and then projects it on top of the 3D data from the kinetic.

    4. Re:The background doesn't change by Sockatume · · Score: 1

      Well, clever's the wrong word. Functional. It is certainly clever.

      --
      No kidding!!! What do you say at this point?
    5. Re:The background doesn't change by PremiumCarrion · · Score: 2

      Can't speak for GP but I've done better using just a JPEG of a background. You can't see when I move at all.

    6. Re:The background doesn't change by gabebear · · Score: 1

      The edge artifacts in the video really make it look like they aren't doing anything special... Pretty much the same as http://www.youtube.com/watch?v=FXb8OORjh3Q

      It's kind of a neat effect, but it's not apparent at all what the Kinect's tracking stuff is used for.

    7. Re:The background doesn't change by Ecuador · · Score: 2

      But he does have a point. How is this using kinect's abilities? How is this different than EffectTV's PredatorTV filter? Is the guy just using all the late kinect dev rage to get credit for doing something simple and available for years, or he is really doing something new?
      I can't really see how the kinect depth info could help. Now, the other guy with 2 kinect's providing a 3d view really did have something, and he could easily add a predator filter to make something invisible, as he would be able to see behind it anyway as long as it was not a blind spot.

      --
      Violence is the last refuge of the incompetent. Polar Scope Align for iOS
    8. Re:The background doesn't change by Sockatume · · Score: 2

      The depth info tells you what's non-background in the scene unambiguously, and allows the texture map to distort to follow the object, which is a bit closer to the way the effect was depicted in GitS and the latter MGS games.

      --
      No kidding!!! What do you say at this point?
    9. Re:The background doesn't change by emm-tee · · Score: 1

      Can't speak for GP but I've done better using just a JPEG of a background. You can't see when I move at all.

      Are you joking? The whole point is to be able to see when he moves. It's a special effect to show a sci-fi kind of "cloaking". Sure you could implement something similar with a standard webcam, but the novelty here is that he seems to use the Kinect's depth information to work out how much distortion/lensing effect to apply. Hence when he stands against the bookshelf in the background, he disappears completely.

    10. Re:The background doesn't change by Ecuador · · Score: 1

      Hmm, upon watching more of the video, he seems to stop the effect or "disappear" completely if he goes far back enough, so I guess that is a way he is using the kinect depth data. However, when he walks forward or backward, nothing changes on how the background is seen through him, I would have expected something like the effect of a lens moving closer or farther, in general different distortion depending on how close he is to the cam.
      Anyway, without knowing what he is trying to do, I can't be impressed by this video.

      --
      Violence is the last refuge of the incompetent. Polar Scope Align for iOS
    11. Re:The background doesn't change by evilbessie · · Score: 2, Interesting

      I think it very much more simple than all of that, he's obviously just taken a couple of polaroids stuck them in front of the cameras and then he's just walking around in front of the depth sensor, so the software is just using the depth information to alter the still images. Not really clever at all and as far as I can see damn obvious from the video (the pink border which shows on the right hand side).

    12. Re:The background doesn't change by Ecuador · · Score: 4, Interesting

      Notice how when he walks far back, the distortion goes away and he disappears completely. What does that tell you?
      So, he is using depth info, but what he is doing with it is rather lame. He still has a static image of the empty room, otherwise when the person went far enough back, he would have to appear uncloaked not disappear completely. Of course you would need 2 kinects and much more work to avoid the need for a static background image and just apply the "cloak" to objects nearer than the background. But that would certainly be cool.
      What we have here, you can do better without a kinect by simple diffing of the background image. If he at least used the depth info to alter the distortion it would be interesting, but it seems to me that when he is walking towards or away from the camera the distortion does not change at all.

      --
      Violence is the last refuge of the incompetent. Polar Scope Align for iOS
    13. Re:The background doesn't change by Lumpy · · Score: 3, Interesting

      "pre existing image of the background" this makes it an epic fail, but cute project.

      I'l be impressed if it generates the background without ANY reference images or reference data.

      --
      Do not look at laser with remaining good eye.
    14. Re:The background doesn't change by Lumpy · · Score: 2

      There has been a plugin for after effects to do this EXACT effect but in more detail and clairty for years. all you need is a green screen shot of the target and the footage you want the effect on.

      --
      Do not look at laser with remaining good eye.
    15. Re:The background doesn't change by Anonymous Coward · · Score: 1

      Well, maybe a peripheral for the next Harry Potter Kinect game - bundle an "invisibility cloak" which is just a cloak in a certain colour or something and use that to make your image vanish somehow. Fun, but yes, struggling to think of any practical uses.

    16. Re:The background doesn't change by Anonymous Coward · · Score: 2, Informative

      The pink border is the wallpaper of his desktop computer. He is taking a video to his monitor.

    17. Re:The background doesn't change by mikeru22 · · Score: 1

      Yeah - my Mac has been doing this for years...with ONE camera angle.

      --
      Go study.
    18. Re:The background doesn't change by mikeru22 · · Score: 1

      From october 2007, iChat Invisible Effect: http://forums.macrumors.com/showthread.php?t=377104

      --
      Go study.
    19. Re:The background doesn't change by Anonymous Coward · · Score: 0

      I've done better with my eyes. I just shut them.

    20. Re:The background doesn't change by Q-Hack! · · Score: 1

      "pre existing image of the background" this makes it an epic fail, but cute project.

      I'l be impressed if it generates the background without ANY reference images or reference data.

      Except for the fact that he moves the camera, thus changing the POV. Whatever he is doing to achieve this effect is live.

      --
      Some days I get the sinking feeling Orwell was an optimist.
    21. Re:The background doesn't change by shadowrat · · Score: 1

      i think he takes a reference frame and depth sample when the software starts up. then he just blends it with the live video based on a difference between the current depth buffer and the reference one. The piano keys are moving when he's not in front of them. It seems like it's the live video feed in areas that are not the user. I imagine it would look pretty odd if someone were to walk behind him.

    22. Re:The background doesn't change by leuk_he · · Score: 1

      The camera he moves is the camera taking the image of the screen. He is not using screen capeturing technology to post the image to youtupe, he is using an external camera to take screenshots.

      Somewhere in the video he is moving to the back, and then the effect disappears completely.If he was using a live stream for the background then the fact that depth information was lost, he should have been appeared, From this fact you can assume his using a static image where he uses the depth camera to make a preadator like effect. Looks nice, but technically not very interesting.

    23. Re:The background doesn't change by Anonymous Coward · · Score: 0

      > but what he is doing with it is rather lame

      1. Capture a snapshot.
      2. Apply a displace map based on depth info. (very easy)
      3. If you want something a bit nicer, just update bg when huge depth is detected.

      I agree, this is totally lame. Slashdot used to have more impressive stuff, this newbie stuff is for Digg's untrained minds.

    24. Re:The background doesn't change by Anonymous Coward · · Score: 0

      If he at least used the depth info to alter the distortion it would be interesting, but it seems to me that when he is walking towards or away from the camera the distortion does not change at all.

      Seems we were watching different videos, when he kicks his foot up it definitely distorts different from his body (which is further back). I think it was largely a 'paint the background image on the nearest surface' type distortion.

    25. Re:The background doesn't change by Ecuador · · Score: 1

      I saw that of course. But the distortion is either movement or color (in any case without an explanation of what he is trying to do we can't judge), because when he walks slowly forward or backward there "distortion" is simply a static offset of the background. With a simple camera you can have different distortions with movement or brightness or color etc, with a kinect you could make the body look like a lens and that is not done here.

      --
      Violence is the last refuge of the incompetent. Polar Scope Align for iOS
    26. Re:The background doesn't change by Lepitius · · Score: 1

      He's obviously being funny, guys.

    27. Re:The background doesn't change by seeker_1us · · Score: 1

      Doesn't the kinect have multiple cameras? Together they might be able to fill in the gaps.

  2. This looks somehow familiar... by Anonymous Coward · · Score: 1

    I would say it's similar to the recent hack that uses TWO kinect devices working together, but this time the programmer has used to to simulate a "predator-like" effect. I might be wrong about it, but if you watch carefully there's an obvious alignment/sync problem inside the "predator" shadow with the actual background (possible due to the image coming from a different angle).

    Cheers!

    1. Re:This looks somehow familiar... by Anonymous Coward · · Score: 0

      Oh my god!! TWO Kinect devices?! *Amazed* This Kinect stuff is all so exciting!

    2. Re:This looks somehow familiar... by Anonymous Coward · · Score: 0

      He's basically texture mapping the background image onto his body. He's not using a second kinect here. In any case, the 'alignment' issue is by-design for a predator effect don't you think?

    3. Re:This looks somehow familiar... by Anonymous Coward · · Score: 0

      Oh my god!! TWO Kinect devices?! *Amazed* This Kinect stuff is all so exciting!

      I think it's somewhat interesting what this guy has achieved with TWO Kinect devices:

      http://www.youtube.com/watch?feature=player_embedded&v=5-w7UXCAUJE

      Maybe not as *exciting* as a the newest real time kernel patch, but I wouldn't call this a "dancing bear" either...

    4. Re:This looks somehow familiar... by evilbessie · · Score: 1

      No is low tech, take couple of polaroids, stick in front of cameras. leave the depth censor alone, walk around in front and ta-da your invisible.

  3. Alpha people! by Anonymous Coward · · Score: 0

    Adjust their gamma!

  4. a bit slashdotted by Anonymous Coward · · Score: 4, Informative

    http://www.engadget.com/2010/12/02/kinect-now-offers-a-stealth-mode-courtesy-of-optical-camouflage/
    http://www.youtube.com/watch?v=4qhXQ_1CQjg

  5. Interesting, but... by Anonymous Coward · · Score: 0

    1 - Background is static. How does it work on moving images.

    2 - This...doesn't seem all that useful? I guess if you can overlay another image OVER your image instead of the stealth thing, CG will probably be a lot simpler to do? Get a guy and have it overlay a dinosaur over him or something and we can have dinosaur fights?

    1. Re:Interesting, but... by SuricouRaven · · Score: 1

      This was something I proposed as a good use for a kinect, thinking of weekly or more-often scifi/fantasy series and children's shows with CG presenters. Production schedules are tight on those, very tight, and it could be a lot faster to have a person act the part of the desired CG character and overlay using this low-cost motion capture than to have a character animator position every joint. It's a technique already used in making movies, but kinect technology brings the cost down greatly.

  6. I can't watch this video by pinkushun · · Score: 2

    It's camouflaged by work's firewall :P

  7. Griffin? by Knx · · Score: 1

    Griffin? Is that you?

    --
    The problem with Slashdot memes is that YOU INSENSITIVE CLOD!
  8. Stupid... by Anonymous Coward · · Score: 0, Flamebait

    Here's an idea for "optic camouflage" using the kinect:
    1) Take a still shot of the room
    2) Make still shot the static image
    3) walk in front of camera
    4) OMG! I CANT SEE MYSELF! IT'S JUST SHOWING A STILL IMAGE! I AM CAMOUFLAGED!!!1111oneone

  9. direct link to video by Anonymous Coward · · Score: 1

    http://www.youtube.com/watch?v=dPV2jmgecfU

  10. Cat got your tongue? (something important seems to by Anonymous Coward · · Score: 0

    What the hell, you lot already slashdotted that
    Screw you people :(

  11. IMDB Reference by gmthor · · Score: 1

    In case you don't get it Predator . This video is not about doing the greatest CG. It's kind of obvious that the distortion was on purpose.

    --
    How do I uncompress my MD5 archive?
    1. Re:IMDB Reference by EdZ · · Score: 1

      Or Ghost in the Shell (the Kinect does use an IR camera, so you could construe a Termoptic Camouflage joke). Or Metal Gear Solid. Or Neuromancer (the Panther Moderns). Or any of the many other science fiction stories that mention optical camoflage.

    2. Re:IMDB Reference by nomorecwrd · · Score: 1

      Also, if you see the glitch at 2:21, you can see that he is not even moving on the same part of the room shown at us.

      I think he has a camera pointing in one direction and the Kinect pointing to another, so he is actually projecting the Kinect volume info over an arbitrary image with a cool effect, he is not really cloaking against his background.
      Not much of a difference with current blue/green screen effects. Final result is exactly the same.

  12. Cool! by Anonymous Coward · · Score: 0

    Now I can play Kinect in the noodz without fear of Microsoft & associated advertisers spying on me!

    Stealth nood, nananana stealth nooooooood nananana

  13. why do you need a kinect for this? by sersch · · Score: 1

    effectv has this effect for years. http://effectv.sourceforge.net/predator.html
    no specific kinect feature needed btw

  14. YouTube direct link by Marbleless · · Score: 1

    http://www.youtube.com/watch?v=4qhXQ_1CQjg

    --
    --I thought I was wrong once, but I was mistaken.
  15. hmm by Anonymous Coward · · Score: 0

    To the people saying the ps2/eyetoy have been doing this for years, you are partially wrong - with the depth detection, this is still able to distort objects infront of you, rather than a composited 2d plane. So an arm moving infront of your body keeps its shape while moving, as it has a different distortion vector than your body.

  16. Microsoft is already one step ahead. by Anonymous Coward · · Score: 0

    My webcam on my windows box shows the background whilst I am walking around the room as though I am not there. Also, my mouse pointer also stays in the same spot while I move the mouse around :P

    Now that's true stealth!

  17. Site by matt007 · · Score: 1

    looks like their site has also been well camouflaged.

  18. Simple Static Image by DarwinSurvivor · · Score: 1

    Pretty simple trick. He took an image (static) of the empty scene, then he just overlays that over any part that the kinect detects is more than certain distance closer than it used to be (radar like). You can tell it's a static image because when we walks in front of the self-playing piano, the keys you see "through" him are stopped (in up position).

    Really don't see *any* use to this what-so-ever. The only difference between it and a live feed is that anything "not" covered up is live (as long as it doesn't move too much).

    In fact, you could achieve this EXACT same effect with a single regular webcam and dressing in all green.

  19. Once you are invisible and use no controls... by digitaldc · · Score: 1

    ..are you playing a game? Or did you just render your Xbox 360 obsolete?

    --
    He who knows best knows how little he knows. - Thomas Jefferson
  20. Rule 34 by CuteSteveJobs · · Score: 1

    I salute Steve Balmer for giving us the Kinect - perhaps the greatest pr0n device ever invented. Well done, Steve!

    1. Re:Rule 34 by mr_gorkajuice · · Score: 1

      I've heard this claim before. Can you actually explain to me how this works?
      You lie alone in your couch, doing humping moves, while looking at the TV where an animated chick tells you that it feels good?
      You jerk off, while an animated character undresses and tells you how big you are?

      Seems sad to me, even by slashdot standards.

    2. Re:Rule 34 by CuteSteveJobs · · Score: 1

      That sounds plausible and I'm sure this software is already on the way. Someone said if drugs were banned people would run in circles on the front lawn until they fell over; "people want to get high." Humans are animals and spend a lot of time catering to animal instincts.

  21. Why is this newsworthy? by Anonymous Coward · · Score: 0

    I can't figure out why this is on slashdot. Someone enlighten me.

  22. And still no Skype support by jgtg32a · · Score: 1

    Why make useful applications when we can make cool applications.

  23. Why is this a slashdot story?? by tommyhj · · Score: 1

    This is easily done with a regular webcam (apart from the extra depth-data). This story is lame, even if it were on a M$ fanboy game-blog...