Combining Two Kinects To Make Better 3D Video

← Back to Stories (view on slashdot.org)

Combining Two Kinects To Make Better 3D Video

Posted by Soulskill on Monday November 29, 2010 @11:35PM from the this-is-what-happens-when-yo dept.

suraj.sun sends this quote from Engadget about improving the Kinect 3D video recordings we discussed recently: "[Oliver Kreylos is] blowing minds and demonstrating that two Kinects can be paired and their output meshed — one basically filling in the gaps of the other. He found that the two do create some interference, the dotted IR pattern of one causing some holes and blotches in the other, but when the two are combined they basically help each other out and the results are quite impressive."

106 comments

Min score:

Reason:

Sort:

Well taht is by Anonymous Coward · 2010-11-29 23:47 · Score: 2, Funny

awesome
Two eyes are better than one by anss123 · 2010-11-29 23:49 · Score: 0

It's amusing how Kinect needs four microphones + calibration to replicate a feat we humans only need one ear for. To see 3D it apparently have to send out infrared dots, and even then it probably does a worse job than the good ol' brain.
1. Re:Two eyes are better than one by Anonymous Coward · 2010-11-29 23:52 · Score: 0
  
  What feat would that be that one stationary ear could do as well as kinect?
2. Re:Two eyes are better than one by anss123 · 2010-11-29 23:56 · Score: 4, Funny
  
  What feat would that be that one stationary ear could do as well as kinect?
  Recognize your voice from the kitchen
3. Re:Two eyes are better than one by fuzzyfuzzyfungus · 2010-11-30 00:00 · Score: 2, Interesting
  
  There is a class of visual inputs that makes the human brain just tie itself in knots, even once you know that the trick is, "optical illusions", Escher stuff, and the like.
  
  I wonder what the class of "optical illusions" for the Kinect's vision system and algorithms is... Off the top of my head, I'd imagine that retroreflective materials might kind of freak it out; but I'd be curious to know if there are any stimuli that cause it to wig out in weird ways, the way that optical illusions do the human visual system.
4. Re:Two eyes are better than one by MrQuacker · 2010-11-30 00:04 · Score: 1
  
  Think about it; the Kinect is given a job most of us would laugh out of town. Build a sophisticated camera capable of full 3-D input and peripheral pickup, using only water and jelly. Build an eye and ears.
  We don't know how to use jelly yet, so we settle with plastic and metal.
  Still a crazy task.
5. Re:Two eyes are better than one by anss123 · 2010-11-30 00:10 · Score: 1
  
  I wonder what the class of "optical illusions" for the Kinect's vision system and algorithms is
  I'm guessing kinect makes assumptions based on common human bone structure, e.g. something like a dog might freak it out and make it explode.
6. Re:Two eyes are better than one by JanneM · 2010-11-30 00:16 · Score: 5, Insightful
  
  The "good ol' brain" does a fairly crappy job, actually. 3D vision systems like these tend to perform quite a bit better than we do. And we only do as well as we do because we can use a lot of indirect clues based on our long experience with a 3D-world - we know how big stuff normally is, for instance, so we can judge distance from size. Mess up those clues and we completely lose it.
  And even with good clues we don't actually measure distance well. Have somebody place items on a parking lot or some place like that, then try to guess the distances. Not going to be very accurate. Try to estimate distance vertically rather than horizontally and you'll do even worse; you have fewer clues and less experience to fall back on.
  
  --
  Trust the Computer. The Computer is your friend.
7. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 00:18 · Score: 0
  
  I think what's most amusing is that Microsoft will be patting itself on the back soon for their recent "We meant to make it open" stance.
  Oh Microsoft. The amount of good press you're getting from the Kinect is worth more than the money you'll be making selling the damned things.
8. Re:Two eyes are better than one by Takichi · 2010-11-30 00:19 · Score: 1
  
  You are essentially just comparing the brain to the computer. We would likely have better spatial resolution if we had more ears and eyes as well. And most of the capabilities of the ear, especially in regards to space, is learned based on the combination with other responses like vision and touch. If you lived your life from the beginning with your only sense being a single ear, you'd probably do worse than a Kinect unless someone explicitly taught you what the things you were hearing meant, if you could ever learn to understand them at all.
9. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 00:36 · Score: 0
  
  Ha! I get the joke about judging heights; less to fall back on!
10. Re:Two eyes are better than one by L4t3r4lu5 · 2010-11-30 00:46 · Score: 1
  
  I would say both would be very accurate, considering no actual measuring would be taking place. You can extrapolate points of reference:
  
  - A car is approx 3m long 2m wide. A parking space is about same
  - The lanes between spaces are 2 cars wide, to allow for idiots who can't follow the arrows.
  - Basic trig can give you any distance in a parking lot.
  
  The same applies to buildings. The average person is 6' tall, with 18" spare to the roof. The floor space is approx 6", making each floor approx 7'. Multiply $floors by 7. For offices, assume false ceilings; 9' per story.
  
  This does go back to your "time spent in the 3D world" though. If we had no point of reference, yes we'd suck. However, we do, so we don't.
  
  --
  Finally had enough. Come see us over at https://soylentnews.org/
11. Re:Two eyes are better than one by SuiteSisterMary · 2010-11-30 01:05 · Score: 1
  
  Which is exactly what the parent said. Besides, look at it this way. You're using cars as an overlay grid. The Kinect is using a dot patter projected in infrared. What's the difference? Or, if you were to go to an empty grassy field, how would you distance estimates do?
  
  --
  Vintage computer games and RPG books available. Email me if you're interested.
12. Re:Two eyes are better than one by TDyl · 2010-11-30 01:11 · Score: 1
  
  And even with good clues we don't actually measure distance well.
  
  Yep, just look at the quarterback for the Carolina Panthers.
  
  --
  Todd: I hope it proves as delicious as the farmers that grew them
13. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 01:13 · Score: 1, Interesting
  
  Ear's can't do that, that's the brain. And for the kinect that would be the program in the device it connects to.
14. Re:Two eyes are better than one by The_mad_linguist · 2010-11-30 01:14 · Score: 1
  
  You can get training if you really care.
15. Re:Two eyes are better than one by EdZ · 2010-11-30 01:20 · Score: 2, Interesting
  
  As the video demonstrates, the Kinect is fooled by spurious pattern projections from other Kinects in the vicinity. This could be solved by replacing the IR source in the 'projector' (actually a point source and a pinhole grid) with one of a different wavelength, and adding appropriate filters to the IR cameras in each Kinect. Each Kinect would then only see IR light of the 'colour' it emits. This would probably require the use of slightly brighter IR emitters.
16. Re:Two eyes are better than one by Abstrackt · 2010-11-30 01:22 · Score: 1
  
  Your comment reminds me of an interesting experiment you can do in 2D. Show people a page containing nothing but a creature that can't possibly exist and ask how big it is, obviously there's no way to answer without scale. If you put a picture of an elephant next to the creature it looks huge but if you put a picture of a mouse next to it the creature looks small.
  
  --
  They say a little knowledge is a dangerous thing, but it's not one half so bad as a lot of ignorance. - Terry Pratchett
17. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 01:26 · Score: 0
  
  Why the apostrophe for the plural of ear? And why no apostrophe for "connects"?
18. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 01:27 · Score: 0
  
  You'd better start dreaming if you think that the brain does a crappy job. If in this world the brain may seem to be doing some crappy job is due to the optimizations being made for the REAL life, perhaps these values you refer while being very important at specific cases (such engineering) weren't find that much important over other aspects of the real world.
19. Re:Two eyes are better than one by L4t3r4lu5 · 2010-11-30 01:30 · Score: 1
  
  Rugby pitches. 100m long. Divide or multiply as required. Plus, a healthy background in outdoor pursuits gave me a good eye for horizontal distance.
  
  Plain buildings a la MiniPeace. however, would throw me completely.
  
  --
  Finally had enough. Come see us over at https://soylentnews.org/
20. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 01:47 · Score: 1, Funny
  
  And why no apostrophe for "connects"?
  Because that's not a plural. What do you think I am, an idiot?
21. Re:Two eyes are better than one by VShael · 2010-11-30 01:57 · Score: 1
  
  That's typically a product of training. We don't have much experience with it, because we don't need it.
  But take an aborigine, and ask him to estimate how far something is, and you'll get a good accurate answer, even if it's not in feet and inches.
22. Re:Two eyes are better than one by Missing.Matter · 2010-11-30 02:19 · Score: 1
  
  And even with good clues we don't actually measure distance well. Have somebody place items on a parking lot or some place like that, then try to guess the distances. Not going to be very accurate.
  And yet we are able to navigate and interact with our environment with a high degree of precision. When I'm driving a car, for instance, without looking at how fast I'm going, knowing distances, the weight of the car, my acceleration and deceleration capabilities, I'm able to stop at a line painted on the road to within half a meter. Just with my eyes!
  I work with robots, and even knowing all this information to a high accuracy, there is so much work that needs to be done with localization, navigation, planning, etc. to get it to mimic my performance. The robot must be equipped with laser range finders, wheel encoders, global positioning systems, and an array of other sensors. If only I could slap a vision system on it and call it a day. Whatever the human brain is doing under the hood, it's incredibly sophisticated. We're bad at estimating distances because we don't need to.
23. Re:Two eyes are better than one by JanneM · 2010-11-30 02:36 · Score: 1
  
  And yet we are able to navigate and interact with our environment with a high degree of precision.
  Yes, we are. Our vision system is pretty successful when you look at how we actually use it in the real world. We don't actually need to know the precise distance to things; what we want to know is rather direction and time to impact and similar and we're really, really good at that (look up tau-margin estimation for instance). Though note that with a human-level vision system you would still need a lot of those sensors you talk about. Our vision system absolutely depends on proprioception to figure out where we are in the world and compensate for our own movements; we need separate dead-reckoning systems and (again) a lot of experience to be even somewhat correct about our movements over large distances and so on.
  But I wrote this in reply to a poster that seemed to believe we humans are actually better than Kinect at the specific vision tasks it's built to do. Too many people seem to believe that the mammalian vision system is inherently great, at whatever tasks we imagine, and that if we could only make something like it our machine vision problems would be solved. That is simply not the case.
  
  --
  Trust the Computer. The Computer is your friend.
24. Re:Two eyes are better than one by Combatso · 2010-11-30 02:37 · Score: 2, Funny
  
  I do, kinda.
25. Re:Two eyes are better than one by anss123 · 2010-11-30 03:11 · Score: 1
  
  But I wrote this in reply to a poster that seemed to believe we humans are actually better than Kinect at the specific vision tasks it's built to do.
  But we are better. Kinect is built to recognize faces and body postures, it’s not built to estimate the distance from you to the TV even if it can do that more accurately than we can.
26. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 04:29 · Score: 2, Insightful
  
  I do, kinda.
  Well, your wrong. "Connects" is a verb, and everybody knows that even plural verbs do not get apostrophe's. Sheesh man, do some research.
27. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 05:53 · Score: 0
  
  Think about it; the Kinect is given a job most of us would laugh out of town. Build a sophisticated camera capable of full 3-D input and peripheral pickup, using only water and jelly. Build an eye and ears.
  We don't know how to use jelly yet, so we settle with plastic and metal.
  Still a crazy task.
  Nobody touch this post! I'm having it for my fucking dinner!
28. Re:Two eyes are better than one by drinkypoo · 2010-11-30 05:55 · Score: 1
  
  But we are better. Kinect is built to recognize faces and body postures, it’s not built to estimate the distance from you to the TV even if it can do that more accurately than we can.
  That is a ridiculous statement. Kinect builds heightmaps. If that's not estimating the distance from you to the TV then I don't know what is. Kinect in fact does the other cool things it can do specifically because it is built to estimate the distance from you to the TV, when other camera systems are not. If this was ALL it would do you could still do the same stuff on the 360 in software, but it would take away from the available processing power which is why embedding it as a complete solution was the smart thing to do.
  
  --
  "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
29. Re:Two eyes are better than one by jrobot · 2010-11-30 08:50 · Score: 1
  
  CMOS sensors light gathering capabilities fall off over increasing wavelength.
  Silicon's quantum efficiency at NIR is much lower than visible. There's not a
  huge range of NIR to play in without QE falling off.
  IR diodes don't emit light over a single wavelength. Not only do they shift long with
  temperature, but the rated wavelength is really an average of the range the wavelength
  drifts over.
  Very tight bandpass filters tend to drift shorter in wavelength off axis.
30. Re:Two eyes are better than one by JuzzFunky · 2010-11-30 11:08 · Score: 1
  
  Check out this video where a guy has taken the model from the kinect and replaced the points with variable sized blobs. Looks cool. Fat Cat
  
  --
  Unexpect the expected!
31. Re:Two eyes are better than one by Traksius+Egas · 2010-11-30 11:08 · Score: 1
  
  This could be solved by replacing the IR source in the 'projector' (actually a point source and a pinhole grid) with one of a different wavelength, and adding appropriate filters to the IR cameras in each Kinect.
  
  Or maybe timing the grid light to be off while the other camera is on and vis-versa. Alternating back and forth quickly like the 3D LCD 'shutter' lenses. This way the grids would not interfere with each other. Don't have to turn off the cameras, just don't use the 3D grid data from those frames where the opposite grid is being used.
  Just my .01 worth. :)
32. Re:Two eyes are better than one by anss123 · 2010-11-30 11:37 · Score: 1
  
  I think you misunderstood me. Building height maps is just a means to an end; the end being figuring out just what you're doing with those limbs of yours. The Kinect wasn't created/built for the purpose of measuring objects, even if it's better at this than us humans.
33. Re:Two eyes are better than one by drinkypoo · 2010-11-30 12:11 · Score: 1
  
  The Kinect wasn't created/built for the purpose of measuring objects, even if it's better at this than us humans.
  That's a big fail of a response. The statement that prompted your original comment was "But I wrote this in reply to a poster that seemed to believe we humans are actually better than Kinect at the specific vision tasks it's built to do." and you said "But we are better. Kinect is built to recognize faces and body postures, it’s not built to estimate the distance from you to the TV even if it can do that more accurately than we can." But that is plainly false. Kinect is built to measure the distance from you to the TV, it has hardware specifically for this purpose. We do not have hardware specifically for this purpose; we infer the information from our existing sensors. So no, I understood your comment perfectly, and I simply disagree with everything you said.
  
  --
  "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
34. Re:Two eyes are better than one by anss123 · 2010-11-30 12:48 · Score: 1
  
  I simply disagree with everything you said
  So you believe that Kinect is superior to humans at recognizing faces and body postures?
35. Re:Two eyes are better than one by Anonymous Coward · 2010-11-30 13:01 · Score: 1, Insightful
  
  Thi's is a brilliant thread.
36. Re:Two eyes are better than one by EdZ · 2010-11-30 21:57 · Score: 1
  
  The Kinect uses an LED laser, so is truly monochromatic. You'd need some very narrow band-pass filters, but these are available, albeit sometimes bespoke.
37. Re:Two eyes are better than one by Combatso · 2010-12-01 00:44 · Score: 1
  
  I wasn't reading the thread or commenting on your grammar... I just think you're an idiot.
38. Re:Two eyes are better than one by fuzzyfuzzyfungus · 2010-12-01 01:55 · Score: 1
  
  The (testable but not yet tested by the public, to my knowledge) question is whether a Kinect unit needs a prohibitive number of frames withits own IR unit on in order to figure out what is going on.
  
  If it takes 30-60 frames to do so, that is only a 1-2 second delay, which is nearly irrelevant from the perspective of the standard use case. Just have the menu do some slightly sci-fi transition during that time and they will never even notice.
  
  If, however, you are trying to use two or more Kinects with shutters, you go from "two Kinects, each at half frame rate; but no distortion" to "two kinects, approximately 1 usable fix every second or two' but no distortion". That isn't totally useless(if imaging a static object, throwing a few kinects on the floor and letting each take their turn, then crunch the results, is still pretty easy; but it is way too slow for 3d video purposes.
  
  If a Kinect can get a fast fix on startup, all is well and shutters would work, if not, you'd pretty much have to play wavelength tricks or put up with some interference...
39. Re:Two eyes are better than one by gibsganich · 2010-12-01 09:36 · Score: 1
  
  Do you really consider it a fair competition: your brain against a device that has a sales price of $150?
this-is-what-happens-when-yo by L4t3r4lu5 · 2010-11-29 23:51 · Score: 1

u don't conform to the character limit for sub-headings?

--
Finally had enough. Come see us over at https://soylentnews.org/
Anybody in optics? by fuzzyfuzzyfungus · 2010-11-29 23:55 · Score: 4, Interesting

How cost and/or physics prohibitive would it be to exploit the fact that "IR" actually covers a number of frequencies of invisible-to-the-naked-eye light with similar properties? Could one modify a Kinect with appropriate narrow-band filters, so that a second Kinect, with filters for a different narrow band wouldn't even see the dot pattern of the first? If possible, how many Kinects would it be possible for(or, at what point does the required narrowness and wavelength tolerance requirements become absurdly costly?)

Is that A)Wholly impractical, because of some sort of effect the reflecting materials would have on the IR wavelengths, B)Sure, it's possible; but have you checked the supplier's price list for narrowband IR filters recently, or C)Just a bit of ebay and some steady hands?

Perhaps more practically, I wonder if the Kinects could(with some mixture of hardware shutters and firmware or driver mods) be made to trade off sample rate for coverage(ie. if the kinects are ordinarily taking 60 frames/second, could two kinects be made to take 30 frames/second each, turning off their IR source when it isn't their turn, and turning it on when it is) or does their mechanism of operation require too much time to calibrate itself on startup?
1. Re:Anybody in optics? by Xelios · 2010-11-30 00:49 · Score: 3, Informative
  
  He touched on these ideas in another of his videos from before this latest one.
  
  --
  Murphey's fighting Occam, and we're in the stands.
2. Re:Anybody in optics? by Vario · 2010-11-30 00:52 · Score: 2, Informative
  
  It is definitely possible to use some narrow bandpass filters. In the infrared region there are various filters for available that have a wavelength window of 10 nm at 1000 nm. These filters are not available at Walmart, but they are not too costly either. Depending on size, quality, wavelength and other parameters you should be able to buy some for $50 (Thorlabs).
  To actually hack the Kinect you have to test, whether there are other infrared filters used and if the camera is sensitive enough at different wavelengths. I don't think the properties of the reflecting materials should be of any concern. The reflection of materials in a household room should not change for a small frequency difference in the infrared region.
  Using a time-multiplex approach with shutters or just software which switches the cameras on and off might work well in theory but should be rather impractical to do without significant changes to the Kinect hardware.
3. Re:Anybody in optics? by Ceriel+Nosforit · 2010-11-30 01:49 · Score: 1
  
  Wouldn't polarized filters do the trick?
  
  --
  All rites reversed 2010
4. Re:Anybody in optics? by slim · 2010-11-30 02:06 · Score: 1
  
  Wouldn't polarized filters do the trick?
  As someone in another thread points out, polarity is lost when light is scattered as it reflects (3D cinemas have special screens).
  Also, polarizing gives you two channels. Bandwidth selection gives you many.
5. Re:Anybody in optics? by Mr+Thinly+Sliced · 2010-11-30 02:07 · Score: 1
  
  Not really as the surface absorbing the light has preserve the polarisation - and anyone who's setup a dual-projector 3D rig with polarised light can attest - you need a special surface coating to get good preservation of polarisation.
  Paint with silver particles in it is typically used for painting 3D screens, for example.
6. Re:Anybody in optics? by warmflatsprite · 2010-11-30 03:18 · Score: 1
  
  The best way to do this would be to modify the firmware to include some kind of pseudorandom modulation scheme (think binary chip sequence). However, the processor on the Kinect is a PrimeSense proprietary ASIC. Good luck reverse engineering it.
  
  Shuttering might work, but as you said, you'd reduce the overall framerate, meaning worse motion capture. Also you'd need to synchronize the shutters somehow, and that'd be a pain.
  
  Filtering would change the sensitivity of the camera, but it won't do much to the laser. You'd have to swap it out for the specific band you're filtering to. Also, I'm pretty sure optical band pass filters aren't cheap.
  
  My personal hope is that we see some kind of modulation in later versions of the device, either because Microsoft asks for it, or because PrimeSense just starts including it by default in their ASICs.
7. Re:Anybody in optics? by damien_kane · 2010-11-30 09:57 · Score: 1
  
  Why couldn't you filter both the diode and the camera?
  I.e. (Normal/Today's world):
  Diode 1 emits light across the entire IR-A spectrum (700 - 1400nm)
  Camera 1 detects light across entire IR-A spectrum (700 - 1400nm)
  
  Diode 2 emits light across the entire IR-A spectrum (700 - 1400nm)
  Camera 2 detects light across entire IR-A spectrum (700 - 1400nm)
  
  Apply filters to both emitter and detector, on both Kinect 1 and 2:
  Diode 1 emits light across the entire IR-A spectrum (700 - 1400nm), filter is applied so that only 700-800nm wavelengths actually leave the kinect (all other absorbed, for the most part)
  Camera 1 detects light across entire IR-A spectrum (700 - 1400nm), filter is applied so that only 700-800nm wavelengths enter the kinect (all other absorbed, for the most part)
  
  Diode 2 emits light across the entire IR-A spectrum (700 - 1400nm), filter is applied so that only 900-1000nm wavelengths actually leave the kinect (all other absorbed, for the most part)
  Camera 2 detects light across entire IR-A spectrum (700 - 1400nm), filter is applied so that only 900-1000nm wavelengths enter the kinect (all other absorbed, for the most part)
  
  Granted, the filter won't stop 100% of the other wavelengths, but it'd probably mute them enough that it's not detectable (or less so).
  Some recalibration may be required, but other than that it should just work.
8. Re:Anybody in optics? by warmflatsprite · 2010-11-30 10:47 · Score: 1
  
  It depends on the laser diode they're using. If it emits a wide enough band of light, then sure, it can be filtered. I just doubt that it does.
So wont 3 Kinects make 3D video? by MrQuacker · 2010-11-30 00:00 · Score: 1

So wont 3 Kinects make 3D video?
1. Re:So wont 3 Kinects make 3D video? by arshadk · 2010-11-30 00:22 · Score: 1
  
  You can't have 6 minute abs! 3 or 4 could make for a pretty cool image all the way round. I wonder if this could be paired with CAD software and a 3D printer.
2. Re:So wont 3 Kinects make 3D video? by TuringTest · 2010-11-30 00:36 · Score: 1
  
  I'd expect that 3 Kinects would make 4D video.
  
  --
  Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
3. Re:So wont 3 Kinects make 3D video? by noidentity · 2010-11-30 00:52 · Score: 1
  
  I'd expect that 3 Kinects would make 4D video.
  
  What about four Kinects? Oh man, 5D... they'd like create a new dimension!!!
4. Re:So wont 3 Kinects make 3D video? by Mister+Whirly · 2010-11-30 04:53 · Score: 1
  
  No, the 5th Dimension already exists.
  
  This is the dawning of the Age of Aquarius!
  
  --
  "But this one goes to 11!"
5. Re:So wont 3 Kinects make 3D video? by clyde_cadiddlehopper · 2010-11-30 05:01 · Score: 1
  
  Yep, except for that darned "light only travels in one direction in time" problem... sigh.
  
  --
  Obi-Wan: "I felt a great disturbance in the Force, as if millions of voices suddenly cried out in terror and were sudden
6. Re:So wont 3 Kinects make 3D video? by MorpheousMarty · 2010-11-30 05:14 · Score: 1
  
  So wont 3 Kinects make 3D video?
  I get what you're saying. With 3 of these you should be able to get x,y,z coordinates. However, each of these is capable of getting the x,y,z for surfaces facing the camera, the problem is you need to hit all the surfaces. With 6 Kinects to cover front, back, left, right, top and bottom you could probably have the best coverage, but I expect four of them, one in each corner of the room like security cameras, would provide similar results.
This is immense... by L4t3r4lu5 · 2010-11-30 00:01 · Score: 1

This makes for real 3D movies. Capture the streams from both sources, combine in real time in the viewer, and you're able to change your PoV and focus independently of any other observer.

This is revolutionary for entertainment. Not stereoscopy.

--
Finally had enough. Come see us over at https://soylentnews.org/
1. Re:This is immense... by nicholas22 · 2010-11-30 00:30 · Score: 0, Flamebait
  
  I think this is similar to what James Cameron did for the (bad) movie called Avatar.
2. Re:This is immense... by L4t3r4lu5 · 2010-11-30 00:38 · Score: 2, Interesting
  
  It's a mixture of the two. He used two cameras to film the live action scenes, but the output was reduced to stereoscopic 3D on the screen.
  
  This is actual 3D on the screen, like a 3D game. You can't zoom in, or even focus, on the background in Avatar. In fact, attempting it gave me a massive headache. With this true 3D rendering of an object, you can zoom, focus, and more importantly pan around objects in the scene, in real time. That is the breakthrough this hack has brought about.
  
  --
  Finally had enough. Come see us over at https://soylentnews.org/
3. Re:This is immense... by slim · 2010-11-30 00:41 · Score: 1
  
  Only for a very broad kind of "similar".
  The live action parts of Avatar would have been filmed using traditional stereoscopic techniques; two cameras imitating two eyes.
  The CG elements would have been traditional CG; models created by a combination of artists and 3D scans.
  The CG animation would have been motion capture as used by Peter Jackson and countless video games: multiple cameras tracking reflective points attached to an actor's body.
4. Re:This is immense... by slim · 2010-11-30 00:46 · Score: 2, Informative
  
  With this true 3D rendering of an object, you can zoom, focus, and more importantly pan around objects in the scene, in real time.
  
  Er, if neither of the Kinect cameras is focused on the background, then it's going to be blurry no matter what.
  Assuming we're talking about a recording, you'd be able to move the virtual camera, but you wouldn't be able to bring things into focus that were not in focus in the recording.
  What this gives you is a 3D model, with an many textures mapped onto it as there are cameras.
5. Re:This is immense... by L4t3r4lu5 · 2010-11-30 00:51 · Score: 1
  
  True, thanks for the clarification. Ok, you won't get to focus, but you will get to pan. That's awesome with a capital sweet.
  
  --
  Finally had enough. Come see us over at https://soylentnews.org/
6. Re:This is immense... by Anonymous Coward · 2010-11-30 00:51 · Score: 0
  
  This makes for real 3D movies. Capture the streams from both sources, combine in real time in the viewer, and you're able to change your PoV and focus independently of any other observer.
  What? There are no focus controls on this. Knowing the depth won't magically allow you to recover an unblurred image, it's not a light field capture device. You won't be able to virtually refocus, it still falls short of "real 3D".
7. Re:This is immense... by tibit · 2010-11-30 04:18 · Score: 1
  
  You miss on how much more expensive the non-CG movies would have to be to allow this. In many sets, if you'd move the camera just a bit outside of what it views, you'd see all of the production equipment, other people, etc. In classical 2D and stereoscopic (market-speak 3D) filming on a set, you only build enough of an expensive set to let you film what's in the screenplay. Anything more is a waste.
  Same goes for 3D CG movies: no point in making the character and scene models any more detailed/extensive than they need to be -- it all requires work: either for modelmaking, or to develop software to procedurally generate the model (usually for environments).
  
  --
  A successful API design takes a mixture of software design and pedagogy.
8. Re:This is immense... by drinkypoo · 2010-11-30 05:57 · Score: 1
  
  I thought that the Kinect camera was fixed-focus with infinite DoF? In that case, you can emulate focus by blurring everything not in focus in the final scene. Of course, everything will be equally out of focus...
  
  --
  "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
9. Re:This is immense... by grumbel · 2010-11-30 06:21 · Score: 1
  
  Er, if neither of the Kinect cameras is focused on the background, then it's going to be blurry no matter what.
  Unless you want to do extreme close ups, focus isn't much of an issue, as the depth-of-field of any webcam and things like Kinect is rather large. The blurring of the background you have in cinema, doesn't happen by accident, but by design, if you just point your regular webcam at the scene you wouldn't get that. The bigger problem would be resolution, as anything further away would naturally have an ever lower resolution then stuff in the foreground. So you couldn't freely float around a room without things getting mushy, but panning around an object should be perfectly fine.
10. Re:This is immense... by slim · 2010-11-30 23:07 · Score: 1
  
  Unless you want to do extreme close ups, focus isn't much of an issue, as the depth-of-field of any webcam and things like Kinect is rather large.
  I'll take your word for it.
  You might expect shallow depth-of-field in low light conditions, since one easy way to get more light is to open the aperture wider.
That's nothing... by Anonymous Coward · 2010-11-30 00:03 · Score: 2, Funny

Can you imagine a beowulf cluster of kinects??
1. Re:That's nothing... by Anonymous Coward · 2010-11-30 00:19 · Score: 0
  
  http://www.breezesys.com/MultiCamera/index.htm all around your "bullet time" 3d Kinetic subject?
2. Re:That's nothing... by JustOK · 2010-11-30 00:36 · Score: 1
  
  or Natalie Portman with hot kinects?
  
  --
  rewriting history since 2109
Fun, but expected by paxcoder · 2010-11-30 00:03 · Score: 0, Redundant

This, however is awesome.
It's not for realtime moving objects, but does its job perfectly.
1. Re:Fun, but expected by Anonymous Coward · 2010-11-30 00:12 · Score: 0
  
  You just linked to the same ./ article that you're posting on...
2. Re:Fun, but expected by samjam · 2010-11-30 02:35 · Score: 2, Funny
  
  He's not the only one. My depth-first recursive post counter has found hundreds of such posts.
  
  --
  blog.sam.liddicott.com
Home Survivelance by NBolander · 2010-11-30 00:16 · Score: 1

Am I the only one imagining getting a Kinect or two in every room of their home and then use it to fly through the 3d video feed of their apartment?
1. Re:Home Survivelance by mrsurb · 2010-11-30 01:29 · Score: 1
  
  Plus potential advertisers will be able to try to sell you what they know you don't own!
2. Re:Home Survivelance by NBolander · 2010-11-30 02:43 · Score: 1
  
  No need to export the video feed to them as I just want the 3d video stream for myself and perhaps a few selected friends.
  Would be cool to insert avatars from several people there though. A 3d video version of the old MOO concept, or a local second life with a live video background to use a more modern analogy.
  Bandwidth will be an issue however.
X-Ray machines by MrDoh! · 2010-11-30 00:20 · Score: 1

With all this stuff in the news recently about backscatter machines and the need for improved x-ray machines, this sort of system would be fantastic for improving the quality of screening, being able to look in and see depth in luggage.

--
Waiting for an amusing sig.
2 Kinects, 1 Box... by bhunachchicken · 2010-11-30 00:33 · Score: 4, Funny

... is good, but I'm holding out for 4 Girls, 3 Kinects, 2 Boxes, 1 Cup :)

--
THE HONOUR OF THE KNIGHTS - CC Licensed Sci-Fi Novel
1. Re:2 Kinects, 1 Box... by Anonymous Coward · 2010-11-30 01:48 · Score: 2, Funny
  
  ...and a partridge in a pear tree?
2. Re:2 Kinects, 1 Box... by acoustix · 2010-11-30 02:06 · Score: 2, Funny
  
  ... is good, but I'm holding out for 4 Girls, 3 Kinects, 2 Boxes, 1 Cup :)
  Correct me if I'm wrong, but if there's 4 girls then wouldn't there also be 4 boxes?
  Just sayin'...
  
  --
  "A plan fiendishly clever in its intricacies"- Homer Simpson
3. Re:2 Kinects, 1 Box... by Anonymous Coward · 2010-11-30 02:09 · Score: 0
  
  2 girls 1 cup, idiot.
4. Re:2 Kinects, 1 Box... by Anonymous Coward · 2010-11-30 04:06 · Score: 0
  
  Depends on how many of the girls are "girls"
5. Re:2 Kinects, 1 Box... by g1zmo · 2010-11-30 08:23 · Score: 1
  
  And a jar.
  
  --
  I have found there are just two ways to go.
  It all comes down to livin' fast or dyin' slow. -REK, Jr.
6. Re:2 Kinects, 1 Box... by jhantin · 2010-12-02 18:27 · Score: 1
  
  ... and zero good taste, apparently.
  
  --
  ...when you're writing a game...tweak the difficulty of "Easy" to something [your mother] can cope with. -- onion2k
Polarizing filters! by John+Pfeiffer · 2010-11-30 00:53 · Score: 3, Insightful

When I first saw the video of one Kinect, I immediately wondered how you could get multiple units working together.
It wasn't until I watched the video again later that day that it hit me. I had just explained to someone how 3D theater projection works, and so I had an epiphany: The most sensible course is to use polarizing filters.
With filters on the IR emitters and cameras, the units should be able to only see their own IR illumination. Of course, it would only work for two Kinects with maximum effectiveness, but considering how well this turned out with the units at right-angles from each other, I don't see why you couldn't combine the two ideas for 3-4 units and get sufficient quality.
I wish I had the money to get a couple Kinects and test my idea, but I'm no good with coding anyway.
It'd be awesome to see the Blender Foundation put out a bounty for a Kinect-based open source motion capture and 3D scanning suite though. :D

--

Friend: "The NIC is misconfigured..." Me: "No prob, I'll just telnet in and fix it." *Silence*
1. Re:Polarizing filters! by Anonymous Coward · 2010-11-30 01:19 · Score: 4, Informative
  
  Unfortunately, this wouldn't work very well. Light tends to lose its polarization somewhat when it bounces off of things. In a theater that's OK because you can use a special screen that maintains the polarization. Band limiting each kinect would be more effective than polarization (and would also scale better - polarization only allows for 2 kinects; the bandpass idea would only be limited by how good your filters are).
2. Re:Polarizing filters! by Anonymous Coward · 2010-11-30 01:37 · Score: 0
  
  Does scattering light still preserve polarization? If not, then it won't work.
3. Re:Polarizing filters! by MikeBabcock · 2010-11-30 02:48 · Score: 1
  
  Light is re-polarized when bouncing off of things, that's why people wear polarized sunglasses; it eliminates glare.
  Unfortunately, you wouldn't be able to predict the resulting polarization with great confidence off of curved surfaces at strange angles like bodies have.
  
  --
  - Michael T. Babcock (Yes, I blog)
4. Re:Polarizing filters! by tibit · 2010-11-30 04:33 · Score: 1
  
  I'd do it in a different way that may well be lower cost and more scalable than any wavelength- or polarization-based selectivity.
  1. Run the Kinects off a common reference frequency. The onboard circuitry probably uses one crystal oscillator and PLL-controlled VCOs to generate various derivative frequencies to time everything. A common reference will keep all Kinects phase-synchronized, while the phase itself may well be random.
  2. Figure out how to discover the phase angle when the IR camera shutter is open (vs. reference frequency), and figure out how is this angle initialized. Is that fixed at power-up, or is it related to the timing of the USB commands? All we care for is to know when the IR camera is sensitive to light, and how to control the initial reference phase for that. It may even be that the IR light is strobed for thermal or power management -- that would need to be in sync with the camera. Anyone out there with a Kinect and a scope to probe the drive voltage to the IR illuminator? Or perhaps the camera is a separate chip and there simply are clock lines to be sniffed out. In any case this should be fairly easy to do.
  3. Set up a simple mechanical sector shutter for each Kinect's camera. A symmetrically notched CD glued to a spindle assembly from dead CD or DVD drive will do just fine. The illuminator can be chopped electrically.
  4. Add an optical interrupter sensor to each shutter for feedback, and run a PLL on a microcontroller to keep the mechanical shutter in phase sync to the IR camera's electronic shutter.
  Since we can control the camera shutter phase and can keep those phases synced across multiple Kinects, we can do time division multiplexing. With very many Kinects one needs to increase the power to the illuminator; perhaps moving to a higher-powered IR LED or laser diode as a light source. The camera shouldn't be a problem -- heck, it will become less sensitive to background IR as the time division gets shorter.
  
  --
  A successful API design takes a mixture of software design and pedagogy.
5. Re:Polarizing filters! by Anonymous Coward · 2010-11-30 08:29 · Score: 0
  
  but you could use left- right-hand circular polarization, not sure how you would do that on the cheap though.
Yo dawg... by Anonymous Coward · 2010-11-30 01:17 · Score: 0

I herd you like Kinects so I put a Kinect on each Kinect so you could move your arms while you were moving your arms.
3D Scanner by necro81 · 2010-11-30 01:21 · Score: 1

The results look an almost identical to the kind of data I get from the NextEngine 3D laser scanner. To create a 3D surface, the device sweeps a laser across the object in front of it. The laser sweeps a vertical line, and shines on the (arbitary) surface of the object in front of it. Stereo cameras capture the shape of the laser line from different angles, and software is able to extract the 3D surface from there. An accompanying visible light image from one camera or the other is used to apply a "skin" to what is otherwise a wireframe. By using a laser and taking its time, rather than broadcasting an infrared grid of fiducial dots, the results are very good: sub-millimeter accuracy is easy, though for handheld objects, not people in a room. Similar technology can be used for very large scale models, such as the I-35W bridge collapse in Minneapolis.
Oblig. by lloydsmart · 2010-11-30 01:46 · Score: 1

Wow, imagine a Beowulf.... blah blah.
In other words... by asCii88 · 2010-11-30 01:53 · Score: 1

1. Find YouTube channel with worthy content
2. Subscribe
3. Share new videos on Slashdot
4. ????
5. PROFIT!
1. Re:In other words... by Anonymous Coward · 2010-11-30 02:24 · Score: 0
  
  Yeah, but it's step #1 that is a bitch.
Basic Webcam by jgtg32a · 2010-11-30 02:23 · Score: 1, Informative

Ya know to the best of my knowledge you cannot use the Kinect as a webcam in Skype. I would love to buy a Kinect but I need a reason other than awesome tricks, I need useful functionality.
Be great for homemade pr0n by bemenaker · 2010-11-30 03:05 · Score: 1

Just wait for the flood of homemade 3d pr0n :) (hey somebody had to say it)
1. Re:Be great for homemade pr0n by vgerclover · 2010-11-30 03:58 · Score: 1
  
  Actually, it seems likely to me that geeky (as in geeks will be the ones doing it) amateur 3D porn will become commonplace before commercial 3D porn.
Purpose of Headings by nschubach · 2010-11-30 03:38 · Score: 2, Insightful

Headings are for brief topic summaries (a few words.) Not content.

--
Every time I start to have faith in humanity, I ruin it by driving to work between 7 and 8 am.
Alternating refresh rates? by Anonymous Coward · 2010-11-30 05:44 · Score: 0

Just like those clunky 3D glasses and your old CRT. Have one kinect's IR grid and camera set at a refresh rate that's offset from the other so that they can't see each other's grids. Of course I'm assuming that the IR projector even HAS a refresh rate. Just thinking.
1. Re:Alternating refresh rates? by Anonymous Coward · 2010-11-30 06:04 · Score: 0
  
  His followup http://www.youtube.com/watch?v=YB90t5Bssf8&feature=related addresses the different options. Polarizing the laser, syncing shutters, etc.
this+HUD by mugnyte · 2010-11-30 06:34 · Score: 1

Given a quality enough image, bandwidth, and some motion-sensing gear (ahem), any immersion-style display (HUD, dome, etc) could allow for real-time panning of a distant location.
Examples:
- shooting a net of these at an operating table would let remote viewers move around the room and view the procedure without crowding the room or limited to the perspective of the single camera.
- a web site could point this setup at anything interesting (lab experiment, box of puppies, anthill, construction site, political debate) and stream it live for an amazing viewer-decided perspective.
- live news could mount an array of this setup to a vehicle and capture a modeled view of anything they could reach, then pan around without much camera work.
Wow... it's just like a camera... by brirus · 2010-12-02 13:17 · Score: 1

Isn't it nice to know that someone at Microsoft could be checking in on our kids doing gymnastics? Most of us will just be leaving it plugged in all the time in our living rooms... I feel safer already.

--
Wikileaks Is Democracy