Slashdot Mirror
Cheap 3D Motion Sensing System Developed At MIT
Al writes "Researchers at the MIT Media Lab have created a cheaper way to track physical motion that could prove useful for movie special effects. Normally an actor needs to wear special markers that reflect light with numerous high-speed cameras placed around a specially-lit set. The new system, called Second Skin, instead relies on tiny photosensors embedded in clothes that record movement by picking patterns of infrared light emitted by inexpensive projectors that can be mounted in ceilings or even outdoors. The whole system costs less than $1,000 to build, and the researchers have developed a version that vibrates to guide a person's arm movements. Watch a video of Second Skin in action."
The tracking fidelity from the video seems low. For movie work you need a very smooth input, otherwise you end up spending a lot of money to smooth out the positional data which has the side-effect of making it look more artificial and robot-like.
What I do like is the use of projected patterns to track individual dots, that's pretty clever. But it seems like this won't be the final solution. Ultimately we're going to need to perfect a micro-GPS system, and that has many more applications than just use as movement-capture for movie production.
"I Don't Have Enough Faith to be an Atheist"
I'm an action blockbuster producer you insensitive clod!
Hey, watching movies is *fun*.
There's nothing like a good film to (temporarily) take your mind off reality.
When I saw the name of this, I immediately thought of Second Life.
Second Skin takes over Second Life!
Oh, the humanity! [or lack of...]
I bet the pr0n industry could have fun with this...
Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
Wii HD suit?
Here's a cookie... *psst* it's MAGIC
If the suit used to capture motion is not the standard black suit covered in little ping pong balls anymore, it's gonna make DVD "making of" extra features a lot less entertaining to watch.
because goodness knows in these troubling times, our society needs to concentrate our technological progress into the betterment of movie special effects, and a better cost structure for producers of action blockbusters.
Yeah, you wouldn't want people spending tons of money on frivilous things during an economic downturn.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
How exactly are radio/wifi/wimax/bluetooth at all relevant in relation to a motion capture camera?
I actually presented a poster next to Ramesh Raskar at CHI earlier this month. While a very interesting project, he seemed to indicate that it was still very much a work in progress.
I regularly report MSN spam to the Hotmail admins.
researchers have developed a version that vibrates to guide a person's arm movements.
One word: autopilot.
(Ironically, my captcha was "females")
That since most of the cost resides in doing something useful with the data (completely producing the images), the time and talent of the people that are _in_ the suits, etc, the producers really don't give a frak that their motion capture system costs $1000, $15000 or even $100000. What they want is something that is proved to work, that technicians are familiar with, and that you can readily rent by the hour along with the facility it's located in. So thank you Media Lab for another useless gadget.
What's interesting to me is, this is almost exactly how the WiiMote works so cheaply!
A lot of people assume that the Wii's sensor bar actually senses, and that it can tell where the WiiMote is. But that ain't so. The sensor bar is just a pair of IR emitters. The front of the WiiMote is an IR camera. The thing you hold in your hand is looking at the external IR sources and using those to try and figure out where it is, and then telling that to the base system, almost exactly as is described in this article.
It's like someone said "hey, let's do motion capture by gluing WiiMotes all over a person's body!".
How exactly are radio/wifi/wimax/bluetooth at all relevant in relation to a motion capture camera?
It's not. It sounds like he's confusing the use of IR with an IRDA port on a laptop. (BTW: His question wasn't off-topic. He asked an interesting question.)
IR is used to illuminate the balls on the mocap suit so that the cameras in the volume see little else but bright white specks to track. They use IR in particular because they can make those balls really bright for the volume, but still retain normal lighting on the stage. Besides not requring actors to act in the dark, they also do this so they can have regular video cameras capture the action for reference.
What he might have meant is using a technology where instead of looking at where sensors are from the outside, use sensors that transmit where they are via radio or something. There is some sense in that, provided the technology exists. I saw demos years ago of a suit sort of like that. I don't know if it transmitted translational data (as opposed to just rotational data...), but even if it did, there was a nice big cable coming from the actor to a computer somewhere. Useless for stunt work. Anyway, he might be thinking of that, but you cannot really tell from the way he positioned his rant. But, yeah, stunt work is done quite often with mocap and once you start putting wires and blinkie on actors you start losing flexibility.
His complaint, as it is stated, is akin to bitching about cars using petroleum when nuclear technology has proven to be a lot better. If you're really vague with the details and throw practicality out of consideration you can make a compelling-sounding rant.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
Amen! We should all be suffering! What is wrong with all these people with their fun games and entertainment, don't they know we're in a recession?
Srsly u guys. U guys, srsly.
It's not about the Institute, it's about the people. Each and everyone of them has made significant contributions to their respective field: astronautics, semiconductors, etc. You may not know now what can be done with these technologies, but in the long run they will be vital for new processes you can benefit from. Just wait and see...
Parent's attention is fixed on the existing moviemaking structure and is not directed to alternative distribution and creation channels. Those alternative channels are the wave of he future. The cheaper production gets, the more opportunity we'll all have for a greater array of diverse movies.
Someday a truly independent movie is going to hit it big via reasonably independent internet distribution. That will change everything. Technology like this only makes that day closer to reality!
I say hurrah!
Hey, watching movies is *fun*.
There's nothing like a good film to (temporarily) take your mind off reality.
Eh, but what is reality?
/randombanter
More appropriately I think would be to say, rather than to escape reality, would be that living vicariously through movies is entertaining. We generate work through our own desire for entertainment and luxury, otherwise must of us would be out of work cuz all we really need to survive is food and shelter and we're so efficient at those things that only a few can sustain very many. Entertainment and enjoyment of life brings meaning to it aside from mere subsistence.
What I mean to say is, everything is reality. And the OP is a jerk troll!
I honestly don't know why IR is chosen over radio.
It bogs down the actors with either cables or batteries. Also, radio requires a smaller volume and requires such a high frequency that it ends up becoming line of sight, anyway.
In short: It's inferior to IR and optical capture in real-world scenarios./
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I thought the most valuable part of motion capture data was the actor's face, as it's the most difficult to simulate in CG. This is a neat system, especially for the price, but it doesn't provide the best feature of the original.
This system will probably be used on athletes, ninjas and commandos. From the video, it obviously only works on an arm without any muscle tone.
I swear to God...I swear to God! That is NOT how you treat your human!
Surely with today's technology it shouldn't be difficult to build an internally-recording suit that doesn't require a tethered connection. Cell phones prove that GPS technology doesn't require anything too bulky. Why not adapt the idea a bit, and make a suit covered in small sensors that record relative positional data from static transmitters.
I envision establishing a "box" of eight transmitters (that many isn't technically necessary, but might provide more accuracy and error-correction; initial thought is literally a rectangular configuration with a sensor at each point, but the formation need not be that rigid) which is only limited in size and shape by the sensitivity (reception range) of the sensors. Once the transmitters are on, the sensors begin recording relative positions and storing them to distributed flash memory hubs; more hubs = smaller individual storage capacities needed, shorter transmission distances, and possibly smaller footprints. All that needs to be stored would be transmitter ID, sensor ID, relative distance, and a timestamp. The data can be uploaded to a server later, along with inputting the relative positions of the transmitters, and the data points can be calculated and compared to build accurate locations at specific times, and plot out the total motion capture. With a little modification, radio transmission from the hubs could enable real-time uploading to processing server.
Advantages I see to this: transportable (motion capture can be done on location, instead of set stages/rooms), untethered, scalable (use as many sensors as you like). Maybe more that I can't think of right now.
Disadvantages: processing-intensive to convert data points to motion plot.
Unknowns: power consumption of the suit, and how best to provide the power capacity it needs for sustained recording.
Anyone see any major flaws in my admittedly hasty design?
It'll sure have a huge impact on movies being made by five friends with whatever effects they and their buddies can put together! Hack together your own mo-cap studio for a couple thousand, and the amount of stuff you can do goes way up.
Also:
egypt urnash minimal art.
$1000 of blister-healing goodness! And at 5000 fps!
http://en.wikipedia.org/wiki/2nd_Skin
It relies on cycling a repeating pattern from every projector 500 times/sec. Every pixel in the pattern encodes a unique symbol by the colors & the changes in the colors over time. By sensing what symbol hits each sensor, you know what pixel from the projector is hitting the sensor & what position on the projector's XY plane the sensor is in. If you know the XY plane position from 2 projectors, you can triangulate the sensor's 3D position, but projectors with enough resolution & bandwidth to do the job are expensive. $1000 would be for very low resolution.
So how do they keep the projected patterns in focus as the actor moves towards & away from the projectors? What if you want to track a close actor & a distant actor simultaneously? Those projected patterns aren't going to be in focus & the sensors won't know where they are.
What he might have meant is using a technology where instead of looking at where sensors are from the outside, use sensors that transmit where they are via radio or something. There is some sense in that, provided the technology exists. I saw demos years ago of a suit sort of like that. I don't know if it transmitted translational data (as opposed to just rotational data...), but even if it did, there was a nice big cable coming from the actor to a computer somewhere.
Just get a spandex suit with tiny RFID tags embedded in it and build an array of receivers in the studio in fixed positions and do the equivalent of GPS triangulation on the smaller scale. Record the data and do the math later to whatever accuracy you need (you're not locating yourself on Earth so civilian GPS hardware limitations don't apply). Meanwhile, your actor is able to wear normal costumes on top of the spandex suit and you'll be able to augment his performance on a more practical set. Maybe even shooting on location.
And no heavy cables coming out your pants' leg.
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
Mobile Phones and GPS would like a word.
Both have been working for years without many problems.
You don't strap 40 cell phones to a human and expect them to, at 60fps, provide accurate position and rotation data down to the millimeter. That's about like saying "we landed a man on the moon several decades ago, there's no reason we can't get people to Mars."
It's a more difficult problem than it looks.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
Make it work in a 70' square volume with 18 actors moving in it and patent it.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
...relies on tiny photosensors embedded in clothes... ...and the researchers have developed a version that vibrates...
Someone will work the system into porn and THEN we'll have a video game that is REALLY addictive!
You have the right to remain sentient. If you give up the right to remain sentient, you will be elected to public office
They seem to use Gray code sequences (only one bit differs between to neighbouring codes). Johnny Chung Lee (the Wiimote Whiteboard guy) already demonstrated the use of structured light and optical fibers in his thesis. He used it to rapidly locate projection surfaces.
Combine Second Life and Second Skin with virtual reality "cave" technology and you have a low rent holodeck. Use it to interpret gestures like the Wii does, and yes, you have a revolution in cybersex and interactive pr0n.
I say it's a buy! Someone is going to make many millions on this. (Especially if they invent a Bluetooth API for optional teledildonics.)
I gave up on the idea of patenting it. It's too obvious. All the barriers to doing it are easily discoverable and the solutions too few and thus obvious to survive a patent challenge (such as coming up with a powerful enough signal to drive the RFID tags without violating FCC regs and not interfering with the reception of their signals, getting them all to uniquely identify both by identity and elapsed time from their driving signal (containing its own clock signal), optimum placement of receivers are a matter of mathematics). It comes down to just a matter of experimentation which I have neither the opportunity nor inclination to do myself.
And besides, I've already stated enough about it in a public forum to make a patent of it by anyone who hasn't already beaten me to the idea impossible. My postings on the subject are now prior art against anyone trying to bar anyone else from doing it. (Some of you were close to coming up with it too, just getting hung up on batteries and cables instead of going for RFID tags.)
Oh, say does that Star-Spangled Banner entwine / The myrtle of Venus with Bacchus's vine?
What I'm trying to say is 'easier said than done'. The IR technology has gone quite a ways. I think they'd all love to have something that's just as capable without occlusion problems or expensive solutions. Really, though, that many actors in that big of space with a minimum of 40 markers each, that's a tall order no matter which technology you use.
"I like to lick butts!" by MobileTatsu-NJG (#32700246) (Score:5, Informative)
I wonder if it was inspired by Johnny Lee's automatic projector calibration system (from 2003) which uses a very similar method. (Yes, that's the same guy that does the Wiimote hacks)
the Wiimote definitely [senses]. [...] [parent also implies that all Wii motion-sensing is done with IR and that isn't the case. The Wiimote has an accelerometer that can detect movement on 3 axes.
Keeping up with "your post doesn't contradict this", I want to add:
The accelerometers sense differential data (motion), whereas the IR camera senses static data (direction towards IR light).
If you assume that there are only two infrared sources out in the world (in either end of the sensor bar) and they don't move, you can use your camera reading to infer your angle in the horizontal plane as long as you can see the infrared sources. Using that, plus the strength of gravity at different points on the wiimote, you can compute its three-dimensional angle at any time.
If you knew your position at time t0 and all motion afterwards (but no IR camera information), you could in theory compute your position at all later times; in practice, due to the relatively low resolution (1 byte per accelerometer per (ISTR) 100hz sample), this doesn't work so well, so you need the IR camera.
Speaking of movie special effects.. anyone else see a striking similarity to the ractives in Neal Stephenson's book Diamond Age? The next step is sensors permanently embedded under the skin..
Hence "from the let-the-ractives-begin dept" d'oh
There are commercial products (MVN from Xsens (former Moven)) that use inertial sensors and gyros to derive the motion. One of the advertised uses is the movie/digital effects industry.
Don't about the real performance of the technology but the idea in itself seems to enable some freedom (no need for interior studios, less expensive).