Slashdot Mirror


Hardware for Homebrew Motion Capture?

goruka asks: "We are a small garage game development and 3D animation group, and as such, we try to develop by reducing costs as much as we can. Recently, it came to our mind that we could setup and develop a home-brew motion capture system by using three consumer USB web-cams to motion track bright objects attached to the body. However, we don't know which web-cam models can: capture at a decent frame rate (25fps) and resolution; are supported and easily programmed under GNU/Linux, since we'd like to later release our software as open source; and lastly, won't cost us a fortune. What are your experiences with such devices?"

82 comments

  1. Interesting... by andrewman327 · · Score: 1

    I have never tried to create a setup like the one that you are attempting, but I have had good experiences with Logitech webcams. Is there an existing program that combines the 3-camera stereo image or are you starting from scratch? What computer system are you using? Make sure that you are not running so many things off of USB (or IEEE1394) that you cannot keep up the throughput from your webcam. This could happen, for example, if you are streaming at maximum quality and saving all of the data to an external webcam. I wish you all the luck.

    --
    Information wants a fueled airplane waiting at the hangar and no one gets hurt.
  2. Watch out for the pixel count by hedwards · · Score: 1

    I would definitely be wary of anything too cheap or with a higher MP count.
    Preferably, try not to have the pixel count too high.

    Typically, webcams are fairly lousy in terms of noise and focus. Getting a
    cam which is pushing the MPs is really just going to be asking for trouble.

    As far as linux go, I believe that mtp cameras are pretty much all going to
    work.

    http://ptp.sourceforge.net/

  3. webcams are crappy by Anonymous Coward · · Score: 0

    why don't you get a dv camera? used ones are very cheap in ebay.

  4. Different solution by JanneM · · Score: 5, Interesting

    Since you probably don't need to do anything real-time with the capture data, I'd suggest that you use whatever inexpensive cameras you can - and record streams onto video. Ideally, you'd borrow three camcorders and use them. Then you can at your leisure transfer the streams to a machine via firewire and calculate 3d-data to your hearts' content.

    The benefit of this setup is that you can get away with very cheap hardware (you can probably borrow needed camcorders from friends and family if it's just a temporary deal), and the image quality - resolution, dynamic range, low-light performance, noise - will be a lot better than with a heavily compressed usb-cam stream.

    As for synking the streams, you have that problem with three usb cams as well (can't caprture three usb-streams on the same computer), and with camcorders at least one step up from the bargain bin, you should be able to use sync cabling if you're really concerned about capturing frames at the same instant. I doubt that would be necessary, though, for the kind of precicion you're looking at getting (just do a linear interpolation between captured points to do an approximate soft sync should be fine for any movement you can hope to capture at 25/50 frames/s anyway).

    --
    Trust the Computer. The Computer is your friend.
    1. Re:Different solution by MBCook · · Score: 1

      I agree, I think that you shouldn't need too much sync. As for how to sync them up, it would be trivial to just flash a LED in the middle of the scene for 1/30th of a second (1/25th in PAL) and use that flash for the sync. That way you don't have to do it by hand since it should be on only one frame or so.

      --
      Comment forecast: Bits of genius surrounded by a sea of mediocrity.
    2. Re:Different solution by JanneM · · Score: 1

      Syncing frames is quite easy. I was however thinking about capture sync - making sure all three cameras grab an image at the same time. With 30fps (PAL, full-frame), a fast-moving object - a hand and elbow doing a punch, say - there will be a fairly noticeable difference in position between the three cameras if they're not capturing in phase.

      The solutions are: ignore it, and don't rely on the motion capture alone for fast movement; figure out afterwards approximately what phase the cameras are in and interpolate the data; or use hardware to really keep them in sync.

      The first is easiest, and if you're not doing fast stuff, perfectly fine. The second can handle fast movement fairly well without needing extra hardware, but falls down for longer video sequences (since the cameras will tend to drift over time). The third is easiest if you have hardware to support it (and actually most video hardware, even inexpensive stuff, does).

      --
      Trust the Computer. The Computer is your friend.
    3. Re:Different solution by mausmalone · · Score: 1

      Syncing is easy... just walk up there and do a big arm "clap" in view of all three cameras. Then, delete anything that comes before the frame where your hands come together. This way, each video will start at the same moment in time.

      (of course, this doesn't sync refresh, but your animation package will be interpolating anyway, so it doesn't matter much.)

      --
      -=-=-=-=-=
      I'd rather be flamed than ignored.
    4. Re:Different solution by monopole · · Score: 2, Insightful

      Firewire cameras such as the UniBrain Fire-I allow for synced capture along a firewire dasiychain. You have to adjust the framerate and dynamic range to allow for bandwidth issues

    5. Re:Different solution by TubeSteak · · Score: 1

      Or he could go old school and buy a clapperboard like they use for filming movies.

      The only reason that board needed to clap was so that the audio and video streams could be synched.

      You can use that visual/audible cue to sync the video & audio from three different cameras with zero trouble if it's all going to be digitized anyways.

      --
      [Fuck Beta]
      o0t!
  5. hackaday had a link to a motion tracking... by Anonymous Coward · · Score: 1, Informative
  6. Axis by Southpaw018 · · Score: 2, Informative

    We run an Axis 207 at work. Pair it up with Zoneminder and you've got yourself a montion capture system, albeit in the form of home security system software.

    --
    ACs are modded -6. I don't read you, I don't mod you, I don't see you. Don't like it? Don't be a coward.
    1. Re:Axis by drinkypoo · · Score: 1

      Motion-activated video capture is not even close to the same thing as motion capture. It's not in the same ballpark, it's not even the same motherfuckin' sport.

      --
      "You're right," Fisheye says. "I should have set it on 'whip' or 'chop.'"
  7. Get yourself a Philips SPC 900NC by Ayanami+Rei · · Score: 3, Interesting

    They run for about $100 and they are available at most CompUSA stores (and nowhere else, it seems).

    Features:
        * 640x480@30fps w/high compression enabled (15 or 10 without)
        * 35mm camera screw mount
        * Manual adjustments on camera (sensor angle and focus ring)
        * Lots of software settings to play with (AGC, white balance, shutter speed, aperature)
        * Compatible with the PWC 10.0.12 drivers from http://saillard.org/linux/pwc/
        * Above all: stable.

    --
    THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
  8. The nature of your dots counts by robbak · · Score: 1

    One suggestion: Do this in a poorly lit area, and have lights mounted on your cameras. (Ideally, they would be almost the only light sources.) Use reflective tape for the dots.
    You will then have very bright spots on an almost black background. That will make recognising the spots easy, even for the most brain-dead of algorythms.

    Professional 'body mechanics' use reflective spheres.

    --
    Prediction for end of Universe #42: Fencepost error in Quantum_bogosort.cpp
    1. Re:The nature of your dots counts by mrchaotica · · Score: 1
      Professional 'body mechanics' use reflective spheres.

      AKA reflective tape on ping-pong balls

      --

      "[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz

    2. Re:The nature of your dots counts by mengel · · Score: 1

      ...or ping pong balls dipped in licence-plate paint.

      --
      - "History shows again and again how nature points out the folly of men" -- Blue Oyster Cult, 'Godzilla'
  9. Unibrain Fire-i by alazor · · Score: 1

    I'm not sure what constitutes a 'decent' resolution but the Unibrain Fire-i [unibrain.com] caps at 640x480p at 30 fps and is pretty cheap. I few months ago I would have recommended Apple's iSight but I understand it's been discontinued. They both connect to the box with 1394 instead of USB which I suspect will be a plus with something as timing sensitive as this. They use this lib [sourceforge.net] not any of the camcorder stuff.

    --

    -
    Systems Administrators: We read the manual so you don't have to.
    1. Re:Unibrain Fire-i by rlanctot · · Score: 1

      I few months ago I would have recommended Apple's iSight but I understand it's been discontinued.

      ?

      http://www.apple.com/isight/

    2. Re:Unibrain Fire-i by WasterDave · · Score: 1

      I had one of these and had to send it back, it's a piece of shit. To make things worse Unibrain have been astroturfing amazon reviews really transparently.

      In short: Fuck them.

      Dave

      --
      I write a blog now, you should be afraid.
    3. Re:Unibrain Fire-i by alazor · · Score: 2, Informative

      The iSight was only discontinued in Europe it turns out.

      http://en.wikipedia.org/wiki/ISight#Discontinued_i n_Europe

      --

      -
      Systems Administrators: We read the manual so you don't have to.
    4. Re:Unibrain Fire-i by archer,+the · · Score: 1

      I purchased two a year+ ago. Both failed a couple months ago. Unibrain did a quick job on the RMA. (The warranty was 24 months.) The specs indicate the CCD used is from Sony, and I vaguely remember a story about Sony having a problem with the epoxy used to attach the CCDs. Actually, I got lucky looking for it: http://hardware.slashdot.org/article.pl?sid=05/10/ 10/2235206

  10. Sorry, here's the URL: by Ayanami+Rei · · Score: 2, Informative

    http://www.compusa.com/products/product_info.asp?p fp=SEARCH&Ntt=philips+900&N=0&Dx=mode+matchall&Nty =1&D=philips+900&Ntk=All&product_code=337160&pfp=s rch1

    The reviews are not exaggerating, it's a nice camera.

    I forgot, it has a usb-audio device endpoint two that's a built in mic, but that's not important.
    The 1280x960 modes mentioned are software scaling, so they're useless. It's a fairly standard CCD board in the unit that is 640x480. Since it uses a Bayer pattern to filter color, you're going to want to throw away the chroma components in your analysis. You might be able to use chroma for helping it distinguish the balls from the background, but you'll want to use the luma information for accurate tracking.

    --
    THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
    1. Re:Sorry, here's the URL: by josepha48 · · Score: 2, Informative
      Yeah or you can get a quickcam 400 they work also. Or if he goes here http://linux-uvc.berlios.de/ he can get a quickcam 5000.

      I've got a 4000 and it does 640x480 at about 30fps.

      --

      Only 'flamers' flame!
      Does slashdot hate my posts?

  11. Raw idea by MBCook · · Score: 1

    Here is what I can tell you. My guess is that FireWire would be easier because all FW video uses the same driver. Find yourself (or just borrow or rent) two video cameras with firewire and use that. You only need two because one can capture one pair of axis (X and Z) and the other the other pair (Y and Z). If you put the cameras at right angles to each-other (pointing down the axis) you should have all you need. The programming half shouldn't be too bad, in relative terms. All you have to do is extract the coords of the white dots and then pair them up between the images. With that (and a little calibrations to fix the slight distortion the cameras will have) you should be able to get decent coords easily. If you get it working, write it up. I'm sure others on /. would love to hear about exactly how you did it as I would.

    The video camera approach also lets you just pre-record the sessions, you don't have to do the capture live if you're computers can't handle it. Standard video cameras would be 640x480.

    Now you could also do it by analog capture using some simple video capture boards. The WinTV series works VERY well under Linux, and the WinPVR series has models that work well and have hardware MPEG encoding to offload the CPU use. Just look at a project like MythTV and buy the cards they like/use.

    --
    Comment forecast: Bits of genius surrounded by a sea of mediocrity.
    1. Re:Raw idea by d3matt · · Score: 1

      The more cameras you have, the better for motion capture. It allows you to track the shiny parts from multiple angles. I would dare say that three is a bare minimum. At the UTD motion capture lab, they have 30 cameras (or did when I saw it). When capturing in real time, it still lost track of the balls on occasion.

      --
      I am d3matt
    2. Re:Raw idea by Anonymous Coward · · Score: 0

      How do you capture the other side of the object with only 2 cameras. I'd guess that 3 is the bare minimum set at 120 degrees if you're not too worried about up and down as much. 4 would be needed for better tracking of objects that are moving up and down in the air a lot more. The more the merrier.

  12. Did this 6 years ago with camcorders for a dem by Hufo · · Score: 4, Interesting

    This is a lot of work but also a lot of fun! I did it for a real-time demo project with a few friend. We used Christmas fairy lights and 5 mini-VHS camcorders. You can see the result at the very end of our Childbone demo.

    Nowadays, using webcams will save you a lot of troubles, and you can find lots of very useful codes on the Internet (such as Intel's OpenCV, however majors issues that you still have to solve would be calibrating camera positions and reliably tracking crossing markers in images. In my system I had to do an editor to manually reassign markers when incorrectly detected or labeled, which can be a very tedious task...

    I would recommend Logitech Quickcam Pro 5000 webcams, as they are USB 2.0, can do 640x480 at 30 fps, and most importantly use the somewhat recent generic USB Video Class spec, for which a driver for linux is available. I have a few of those and the image quality is quite good :)

    Good luck!

    1. Re:Did this 6 years ago with camcorders for a dem by munpfazy · · Score: 2, Interesting

      Interesting.

      I wonder how the big studios deal with marker crossings? (Then again, perhaps they just pay humans to do tedious work.)

      Seems like there must be a cheap hardware solution, given enough time and energy.

      For example, one could put colored filters on the reflectors. By replacing each dot with a cluster of colored dots and then selectively blackening them you could code each point uniquely. It would take some experimentation to figure out how to get the results you need with consumer gear. Presumably you won't have the resolution you'd need to pick out the individual dots, and it might take some work to identify an unresolved marker in a color image. If cameras are either very cheap or have a very fast and have an externally accessible frame sync, you could either use single color filtered cameras or a chopper wheel with several filters on each camera.

      Or you could try to use single-color dots with a range of different colors. It would take some experimentation to see how many unique IDs can be reliably identified. Seems like it could be in the low hundreds though, given the number of unique colors a human can pick out of an image taken with a cheap webcam. Finding suitable reflectors might be a challenge. Perhaps mixing dies with corner reflector granules would work.

      If you replace passive targets with LEDs and use very fast cameras, you could conceivably identify the dots by strobing them. If you can run at, say, 8 times the rate you need for motion capture then you've got plenty of bits to work with. Finding cheap consumer gear that will give you low noise images that fast may not be trivial. (Or, you could always do your motion capture work in slow motion, I guess.)

      A final option is to add extra cameras. By adding additional nonorthogonal views, it should be possible to unambiguously decipher most crossings. Actually, that might not be very much harder than doing motion capture with a small number of cameras, especially if it doesn't need to happen in real time. Might be harder to align things properly - but you could imagine playing tricks with a rigid grid to try to automate most of that.

    2. Re:Did this 6 years ago with camcorders for a dem by Niet3sche · · Score: 2, Informative

      I can second the OpenCV nomination.

      However, I think I may be able to add something to the puzzle: I was informed (but have not yet tested) that IEEE1394 (Firewire) cams will synch across the bus. This means that you no longer have to worry about adjusting for framedrops or timing or whatnot. Rather, the two cameras "see" their fields in lock-step with respect to time. I know that some folks here locally have had great success with Uni-Brain Fire-i cameras, but earlier in the thread someone reported a bad experience with them.

      However, this being slashdot, I must remind you that YMMV.

  13. Bump by mnmn · · Score: 1

    I was looking for a USB cam that would do 640x480 at 30fps at least. I was trying to put it on an RC airplane with a PC104 computer, drive and battery. Never did find a camera that did this native res at this speed. But I didnt look hard enough either.

    I should have posted on slashdot.

    --
    "Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
  14. Logitech quickcam 5000 by Anonymous Coward · · Score: 0

    640x480, up to 30fps, works with Linux.

    As seen last Sunday in ./, some guy has even used one of these as an alpha-particle detector and visualizer. It is not one of the cheapest cameras ($60 @ Newegg), but as the guy says: "the quality of this camera I have to point out is very high."

    You could borrow some ideas from the original article. Please, just don't attach radioactive objects to your body.

    For syncing the streams from the cameras, perhaps you could use some sort of electronic light control (are one of you into electronics?), to quickly flicker the lights, or slightly change the color. You could detect changes in lighting to sync the video streams.

  15. Why reinvent the wheel? by cjp · · Score: 1

    I can't imagine how little you must value your programmer time if you're considering a homebrew solution! If programmer time is cheap, then ok, but my preference would be to outsource it. There are plenty of contract mocap studios around, take advantage of that fact . . .

    1. Re:Why reinvent the wheel? by Anonymous Coward · · Score: 0

      Apparently you have no idea what mocap time costs in a decent studio, in addition to flights for producers & lead artists, lodging, lap dances...

      Do you even work for a game company?

    2. Re:Why reinvent the wheel? by SanityInAnarchy · · Score: 1

      Many consider a homebrew solution regardless, if for no other reason than "Open Source". Reinventing the wheel can be a good thing if it's something you enjoy doing, and if it means it will cost you (and everyone else) so much less in the future.

      I, for one, will be pretty interested in how this turns out. I'd love to be able to do motion capture for a little one or two-man game.

      --
      Don't thank God, thank a doctor!
    3. Re:Why reinvent the wheel? by cjp · · Score: 1

      I would go so far as to say that _most_ consider a homebrew solution. I don't think it's a great idea personally, as they're unlikely to realise beforehand just how large a task it can be. If your schedule relies on accurate estimates and fixed costs it's a dangerous thing to attempt. In the case of an indie game it may not be an issue, but yeah, in any commercial context . . .

    4. Re:Why reinvent the wheel? by SanityInAnarchy · · Score: 1

      There is no such thing as an accurate estimate in software development, so you're right there -- outsource absolutely everything to people who have already solved the problem, because then you know how much it costs.

      It also means you'll never innovate a thing.

      Or, try Doug's Law (correct me if this is from anywhere but my friend Doug): Make an estimate. Double it, then change the units. That's the estimate you give to the business end, and to the clients.

      So, for instance, if it'll take you 10 seconds, say "Gimme 20 minutes". If it'll take a couple minutes, that's four hours. If it'll take a good, solid hour, that's 2 days. A couple days is four weeks. Three weeks is six months. A month is two years.

      If, after that adjustment, it's still worth it to go homebrew, you've got a pretty damned safe bet.

      --
      Don't thank God, thank a doctor!
    5. Re:Why reinvent the wheel? by cjp · · Score: 1

      Thanks for the entertainment, always great when an Anonymous Coward questions someone's background :D Anyway, creating usable systems costs in programmer and scheduling time is often inefficient compared to using an off the shelf solution. Varies a lot, and some off the shelf solutions are definitely NOT cost effective. Of the five games I've worked on, varying in size from a few hundred thousand budget, to fifteen million, to a self-funded one, none have had a homebrew mocap solution, and none have had any issues with their off the shelf solution. Weird that you'd be so confrontational about it, really. Why's that chip on your shoulder?

    6. Re:Why reinvent the wheel? by cjp · · Score: 1

      Damned good rule of thumb, yeah :D It's tempting to apply that, but I'm thinking it wouldn't go down too well. Heh. In my experience the cause of the discrepancies is that the estimate only covers getting the feature to minimal functionality, whereas the final product requires something more polished and complete.

    7. Re:Why reinvent the wheel? by Osty · · Score: 1

      It also means you'll never innovate a thing.

      You just need to know what to outsource and what to do in-house. If you're an up-and-coming motion capture and 3D animation studio, then it might be worthwhile to try to innovate in the motion capture space. If instead you're an indie game developer that just happens to need motion capture for your next game, you're better off outsourcing the motion capturing and focusing on innovation in your game design and gameplay.

      Just because you can outsource something doesn't mean you have to outsource everything.

    8. Re:Why reinvent the wheel? by SanityInAnarchy · · Score: 1

      Well, the good news is, if you try to take all of that into account, and make a very conservative estimate to begin with, and you follow Doug's Law, then you end up with something that is pretty damn accurate, or at least close enough that your employers/beancounters/customers won't doubt it. Then, if you really can get it done in a fraction of the time, it's that much better for you.

      Oh, I'm going to add my own amendment to Doug's Law: Doug's Law only covers unknowns. Anything you know -- time cost by miscommunication, time spent polishing the product -- even if you don't know how much time it will take, if you know what to call it, you must add something to your pre-Doug estimate.

      The point is, you're going to have to give them some number, and it can't be too insane or you'll never get the go ahead. "When it's done" won't cut it. But it's far, far better to be ludicrously early than slightly late.

      --
      Don't thank God, thank a doctor!
    9. Re:Why reinvent the wheel? by budgenator · · Score: 1

      also by starting with a homebrew system a lot of knowlege is gathered in research which really can help you avoid the gotchas if you eventually go to a commercial system instead.

      --
      Apocalypse Cancelled, Sorry, No Ticket Refunds
  16. Cost analysis needs some rethinking by Anonymous Coward · · Score: 0

    Software development costs are usually dominated by the cost of developers. The cost of payroll usually eclipse software licenses and hardware costs. There are exceptions to this but I don't think you application is one of them. Also remember that 90% of your time will be spent finishing the last 10% of your applications. Commercial applications (that do this kind of stuff) are expensive because a lot of time has been spent on the last 10%: polishing, ease of use, reliability, etc. You may not be saving any money doing this yourself.

    That said, is your application realtime? If not, it doesn't matter where your video comes from: it's going to be a file anyway. Synch up the video with a strobe light, turning off an LED, or recording a line level audio signal to all three camerass. Godspeed.

  17. Cheap Mocap Solution by MacroMegaMan · · Score: 2, Interesting

    You could try "Optitrack" by naturalpoint software. Seems really useful, actually and for $249, it's worth taking a chance on too.
    Here's their link:
    http://www.naturalpoint.com/optitrack/

    If you have Poser(and free time), you can also try the Rotoscoper plugin by PhilC as well.
    Huge link follows:
    http://istore.mikrotec.com/philc/index1.html?page= catalog&trackerid=1661406456&category=a&vid=208024 5373&pid=924839477&oldvid=2143420604

  18. Would an easier idea be... by Disoculated · · Score: 1

    Wouldn't it be easier to rig up an armature using old joystick potentiometers at each joint than use three cameras and try to extrapolate position data from that? You could easily have a single machine running a perl script collect the position data from all the sensors at one time into a file and then write it out as bvh/whatever or convert it later.

    Something like Gypsy, only lots cheaper.

    1. Re:Would an easier idea be... by mathgenius · · Score: 1

      No cameras. Use accelerometers. Lots of them (20-40?). I guess this is a bit of a research project, but a small prototype (eg. one arm) would not cost much. Accelerometers are surprisingly accurate. However this would require some reasonably clever physics algorithms to recover body position (eg. gravity provides a constant 1g downwards acceleration).

      Simon.

  19. time syncronization by Anonymous Coward · · Score: 0

    You can also set up a single camera on each of 3+ computers and syncronize the time stamps via NTP...

    1. Re:time syncronization by SanityInAnarchy · · Score: 1

      The jitter would probably kill you -- remember, it's mostly not about making sure we know what frame 1 is, and frame 2, and frame 3, from each of the cameras. It's about making sure that a given frame isn't off by some tiny fraction of a frame. I don't know if NTP has the resolution you need, but even if it does, you need some damn precise control of the hardware to make it work. Or you could just use some sync cables of some sort and let the cameras worry about it...

      --
      Don't thank God, thank a doctor!
    2. Re:time syncronization by d3matt · · Score: 1

      My group where I work has been dealing with headaches invovled with timing a lot recently. The jitter will definitely kill you. It is acceptable for NTP to be off by 10 milliseconds between a host and a client.

      --
      I am d3matt
  20. Hitlab by kramulous · · Score: 2, Interesting

    Howdy,

    Hitlab (NZ [hitlabnz.org] but also an American office [somewhere]) also have come out with some pretty funky motion tracking. Beit for other purposes, but the source is available (via SourceForge: ARToolkit).

    It may not be exactly what you are wanting, but with a little modification it should, and, importantly, is CHEAP.

    Good luck. Hope to see some break-through gaming experiences. Hooroo

    --
    .
  21. This may be a much bigger job then you think... by Anonymous Coward · · Score: 2, Informative
    I don't do MoCap, but I have worked in face tracking and I have been in major film industry motion capture studios and seen their set ups. It is very complex.

    First, they have many, many cameras, because you have to have 3 unobscured camera views to triangulate a point. I assume you want to mocap people doing game moves, so multiple camera are required. Also, they capture with infra-red, not visible light. They put infra-red reflective spheres on the people at key locations, so what the camera picks up are dots on a black background. This is for 2D tracking.

    Some real time tracking occurs, and and a rough 3D wireframe is generated so they can see if they have a good take. Note that it's not one computer for all the cameras because of bandwidth limits. You may not be able to support very many cameras per computer, because you need to save all the frames for post processing. The rough tracking data is sent via UDP to a single computer that does the wire frame display.

    After you have captured the data, then it gets really hard. You need to calibrate all the cameras so you can combine their data. You need to survey their positons and their pointing angles. It is possible to calibrate by locking down the camera and shooting targets before you start. I don't know if you need to correct for lens distortion or not, it may depend on your cameras.

    The cameras have to be synched. If they are taking picture as different times then 3D point positions will be not be right. Web cams don't have external synch.

    First, you have to do 2D tracking for each camera. Then you have to figure out which 2D tracked point on camera A corresponds to the same point on cameras B, C, D for each frame, typically while the mocap actors are being very athletic. The you need to combine multiple 2D points to 3D points. Remember that 2D points will dissapear and reappear during a move.

    After you have 3D points then you need to connect those dots into motion paths. This takes a lot of very complex motion filtering software. People often use Kalman filters for this. Sometimes they do Kalman filtering in 2D and in 3D. Multi pass filters can be used, where you go from 2D to 3D to motion paths, and then you take the estimated 3D positon and project it back to 2D. The back projected data is combined with the captured images to get better data for the next pass.

    Assuming all that works, then you can take 3D path data and translate to the frame of reference on the person so you can animate the character. Are you going to use inverse kinematics to to derive joint angles from end posions of arms and legs? Often times you measure points on both sides of a joint and directly measure joint angles so you can directly apply the measured data to the 3D model.

    Heck, I don't do this stuff, I just have been around it, and these are things I remember. Optical tracking is very hard. People still use magnetic tracking and joint flex tracking, sometimes in combination with optical, becasue they are better for some kinds of measurements. Now you know why movies and high end vidoe games are so expensive....

  22. cheap homebrew infra-red mocap rig by Robbat2 · · Score: 4, Informative

    Couple of things in here, from researching the field with a university research lab to see about buying commercial gear, I have a lot of suggestions.

    - For your camera, look for cheap used DV cameras on ebay. Not super high res, but lots of them 3 ain't going to cut it, consider at least 9 (high/low from each of the cardinal directions, and on top [might want a few for different sectors]) - occlusion is an absolute bitch of a problem.
    - This will provide reliable time-synced data, and NOT max out your USB bus.
    - USB cannot provide you with images from 3 cameras with the same timesync, it's just not capable of such behavior.
    - Firewire has a longer length limit on the cables, which is a big help for your work.

    - Cheap PCI firewire cards - two should be enough, this will give you 6 seperate firewire busses, and put you at the limit of your PCI bus.
    - Find filters that fit said cameras, and are opaque to visible light, but transparent to infrared.
    - Rig up really bright infra-red lighting, ideally with a low quantity of visible light output.
    - Go to an burgler alarm supply place, and buy infra-red reflective tape - I leart this tip from the EA guys a couple of years back, the 'official' reflective tape from 3M costs too much, and is a pain to order, but alarm places stock stuff that works even better, and is cheaper to boot.
    - Buy really small polystrene balls, and cover with infra-red tape. On one small part of the ball, put the hook side of a velcro dot. These are reusable now, avoiding problem with tape waste. You can also clean them easily to keep them very reflective.
    - For your subjects, get them to wear any clothing that velcro will hook reliably onto (pretty easy choice)
    - Place the reflective balls on either side of every joint, spaced not more than 90 degrees apart - eg your elbow should have 8 balls.

    Using infra-red helps reduce the data-set size way down, and also lets you use the cameras in monochrome for capturing, greatly reducing the data-set size.

    From working with several commercial mocap rigs, I'll say that the calibration routines are extremely important. You need to accurately map the entire volume that you wish to capture in. Depending on space available to you, consider building a simple frame or using a lighting rig to attach the cameras to.

    I will repeat again, occlusion is an absolute killer problem. From visting the EA facilities in Burnaby BC to specifically research their systems (I was working with a university research lab at the time), they estimated that they lost 2 hours of production a day to occlusion problems during mocap shoots.
    Your system must be capable of tracking all the balls, all of the time. If it loses one, it's almost impossible to pick it up again properly during a runtime - you'd need to recode the relative location of that ball before it gave you useful data again.

    --
    ICQ# : 30269588
    "I used to be an idealist, but I got mugged by reality."
    1. Re:cheap homebrew infra-red mocap rig by cerberusss · · Score: 1

      For your subjects, get them to wear any clothing that velcro will hook reliably onto (pretty easy choice)

      They're making a sequal to Leisure Suit Larry. Their subjects Won't wear clothes.

      --
      8 of 13 people found this answer helpful. Did you?
    2. Re:cheap homebrew infra-red mocap rig by LetterRip · · Score: 1

      I would think that the number of cameras needed can be greatly reduced by using better research and algorithms. There is quite a bit of research that has largely been ignored by most mocap software which I expect can greatly reduce the markers and cameras needed for good quality motion capture

      1) there is the research on animation from doing a series of figure drawing poses - the algorithm provides a set of poses that the user picks as being correct (a single figure drawing is ambiguos since there are multiple poses that can meet the required joint locations) this takes into account natural joint angles as well which generally brings the pose space to about 10 poses

      2) research using kinetics for interpolation - take the subjects mass and the individual is constrained by how fast they can change direction and need for dynamic balance, etc.

      3)motion capture from silhouette

      I would think that combining multiple methods - while being a bit more processor intensive could result in much superior results for much cheaper than using a large number of cameras.

      There is also a mailing list apparently dedicated to open source motion capture and creating free/home brew mocap setups.

      http://movies.groups.yahoo.com/group/motioncapture /

      LetterRip

    3. Re:cheap homebrew infra-red mocap rig by Emil+Brink · · Score: 2, Interesting

      Sounds interesting, the tip about IR-reflective tape especially. It got me thinking ... if reflective tape is expensive enough to warrant hunting for cheaper sources ... And you also need to get IR light sources, wouldn't it make sense to invert the lighting, and put IR-emissive dots directly on the mocap actor? Something like LED throwies but with IR LED(s) rather than visible light? Perhaps it's still too expensive and/or impractical what with batteries and so on, though. I do wonder how it would compare, brightness-wise. Anyone tried it?

      --
      main(O){10<putchar(4^--O?77-(15&5128 >>4*O):10)&&main(2+O);}
    4. Re:cheap homebrew infra-red mocap rig by Robbat2 · · Score: 1

      The primary reason for lots of cameras is still the occlusion problem. I agree that if you are only capturing one person at a time that the problem is reasonably reduced, but it still exists (consider the pose of bending over to touch your toes, which greatly occludes your front surface).

      The figure pose stuff I have dealt with as well - using early and recent versions of Credo Interactive's "LifeForms" and the mocap input form they take. It's good for helping to solve occlusion by predicting which location a ball belongs to, but still has issues.

      The kinetics/kinematics is the intrinsic source data of the figure poses, again mistaken identity is still bad (consider rotating your figure faster than the refresh on your cameras).

      silhouette fails with multiple subjects and shadows.

      cameras are cheap now - the ones I was pointing at are under $100 each. you could probably arrange $50 each if you were to buy a bulk set.
      Assuming my coding time is $25/hour (below my current wage), buying 4 more cameras is a better use of my money than coding.

      --
      ICQ# : 30269588
      "I used to be an idealist, but I got mugged by reality."
    5. Re:cheap homebrew infra-red mocap rig by Robbat2 · · Score: 1

      that's actually a very interesting, esp that you could make the LEDs flash (actually a dim-bright transfer, possibly using reed encoding, but never turning off totally) with an encoded number, and thus allowing them to be identified quickly by software after an occlusion event.

      --
      ICQ# : 30269588
      "I used to be an idealist, but I got mugged by reality."
    6. Re:cheap homebrew infra-red mocap rig by JoeKilner · · Score: 1

      Point gray do firewire cameras + SDK for computer vision research that I believe automatically synch if put on the same firewire connection.

    7. Re:cheap homebrew infra-red mocap rig by Grab · · Score: 1

      Hire hairier actors...

    8. Re:cheap homebrew infra-red mocap rig by Grab · · Score: 1

      Problem with this is getting a 360-degree output in all three axes. LEDs don't cut it for that - too narrow a viewing angle. Old-school incandescent bulbs are much better. Try digging out some of those little bulbs they used to use for school science experiments that used 4.5V - they're actually round too, which is good. Downside is that you need some way of attaching them to your actor, and they get hot to the touch, and the screw fitting means that they're going to be a couple of cm away from the actor which you'd need to take into account.

      Grab.

    9. Re:cheap homebrew infra-red mocap rig by cerberusss · · Score: 1

      LOL!! You bastard. I can't get the image out of my head.

      --
      8 of 13 people found this answer helpful. Did you?
    10. Re:cheap homebrew infra-red mocap rig by zeke2.0 · · Score: 1

      Thanks for the link, I just joined.

    11. Re:cheap homebrew infra-red mocap rig by jigyasubalak · · Score: 1

      There's goes the little-red-randy-pervert painting the town red, again.
      I just have to look the other way for a while and look you are upto :)

      --
      The best planning can be done after the project completes.
  23. I'm trying o build one... by Bones3D_mac · · Score: 1

    I have a set of three Canon ZR500's that I'm planning to use in a rig that will record video from the top, right and front view of a moving target simultaneously. However, there are going to be a number of limitations to overcome with such a setup.

    Luckily, it seems RealViz will soon be releasing a new software package, called Movimento, that claims to be able to obtain motion capture data from video of events taken from multiple camera angles. While this new software should definitely make things easier for me, it also appears that this software may be able to use video taken from non-stationary cameras as well. As long as there is simultaneous footage of a single event taken from two or more camera angles, the software should work.

    But until Movimento is released, it seems manually rotoscoping motion using a rig like mine may work best for the time being.

    --


    8==8 Bones 8==8
  24. face motion capture by Anonymous Coward · · Score: 0

    This site shows an accurate single camera face motion capture program, requires only lipstick on the face: http://www.idiom.com/~zilla/Work/V/oct1TregP2Sor3. mov

  25. I looked into this once by merlin_jim · · Score: 1

    You're going to have problems with webcams. The biggest problem is synchronization; getting them all to take their frames at the same time. Pinnacle Imaging is a scientific imaging device vendor that publishes res and framerate specs for all their cameras, and some of their cameras support frame synchronization... but a quite a price.

    When I was looking into it for a local doctor's office that wanted to do gait analysis research (something near and dear to my heart since i have a game leg and still no real explanation for why), the min we could get the hardware for was about $2200 IIRC... nw that was three years ago as usual the prices have dropped somewhat by now...

    --
    I am disrespectful to dirt! Can you see that I am serious?!
  26. Would RFID chips? by mikael · · Score: 2, Interesting

    Could you have each marker represented by a RFID chip? Then detecting the position of each marker would only require four RFID transmitters. The time delay would give you the distance to each marker and you could use triangulation to determine the current orientation. And each RFID tag would be easy to label.

    --
    Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
    1. Re:Would RFID chips? by cr0sh · · Score: 1
      From my research, doing this with a single RFID chip "marker" is possible, if you can deal with the low resolution of the device (+/- 1 foot or so - may be better with more transmitters) and the lag time (second or so delay). For realtime mocap, this isn't usable, but for "realtime" inventory tracking, perfectly acceptable. With more markers, things just get worse - and for mocap, you will need more markers, typically one per major joint, depending on what you are modelling.


      Unless things have come a long way since I last researched this (a couple of years ago, so who knows - please correct me if I am wrong), RFID just isn't ready for the mocap market. Now, what could be used (and might be helpful for the poster, if he can find a higher framerate capture system) is active IR markers - basically an LED blinking in a set pattern (use a small custom programmed 8-pin SMT PIC, epoxy blobbed to a small PCB - basically PIC and other parts on one side, SMT IR LEDs on the other). The software can then detect this "fairly easily" (not going to mince words - active or passive IR marker tracking is tricky software-wise), and it is easier to deal with occlusion of the markers (passing in/out of tracking range due to being hidden by other body parts), since you know which marker is which (with passive markers, you have to add motion prediction to the software to "guess" where the marker is going to be in the next frame, within a certain spherical range, once you know where all the markers are in the first - and dealing with occlusion isn't easy).

      --
      Reason is the Path to God - Anon
  27. NOT webcams! by Vo0k · · Score: 1

    Friends were doing a thesis on 3D image capture using 2 USB webcams, in paralell with my thesis, so we often exchanged ideas and progress details. They researched the topic of the cameras VERY deeply and one conclusion is ALL WEBCAMS SUCK. The most expensive and advanced models on the market (about 2000% the price of the cheapest) have about 40% better parameters than the cheapest ones. They just come with fancy software and SDK for them exists at all. But they have just the same distorting sucky optics, they have just the same impossible to tune parameters like exposition or focus, they have the same sucky 640x480 resolution and framerate rarely exceeding 24, plus built-in undisablablable compression that makes image processing a big pain in the neck.
    Either go for industrial quality cameras (expensive) or just pick standard video cameras instead.

    --
    Anagram("United States of America") == "Dine out, taste a Mac, fries"
  28. doug's law by Uzik2 · · Score: 1

    I believe Bill Burkett told me that estimation method in late 1983.

    --
    -- Programming with boost is like building a house with lego. It's a cool but I wouldn't want to live in it
  29. Thing about the Philips 900: by Ayanami+Rei · · Score: 1

    Among all webcams, its the only one I've been able to positively determine has a 35mm screw mount.
    If the intended use is motion capture, they're going to want to have tripods and booms and stuff to hold these cameras in precise positions. Most other webcams just have some kinda suction cup or LCD clamp that isn't very useful for arbitrary positioning.

    --
    THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
  30. Clean solution by John+Sokol · · Score: 2, Interesting

    I use the Kodicom 4400r board http://dvr.videotechnology.com/ this uses 4 Conexant 878 chips (formerly called Brooktree BT878)
      The default bt878 driver in FreeBSD works but I had to write a small driver to init the video switcher on the board.
      Using very simple code you can capture and process 4 full motion video channels in FreeBSD.

      I there is also the BTTV Linux driver for this board.

      CCTV Cameras can be had for $35 each and the board is $200. for a total cost of $300 for 3 cameras to do motion capture.

      I have used Blinking dual color LEDS on the target very successfully.
      Also retroreflective balls and LED lighting also works well. The $35 black and white versions of these camera come with IR leds for so called "Night Vision" and works great with the 3M reflective tape.
      See http://www.videotechnology.com/old1104.html Retroreflective Materials for more info on that also.

    --
    I am always doing that which I can not do, in order that I may learn how to do it. - Pablo Picasso
  31. Old video camera + IR leds by raluxs · · Score: 1

    This is what i would do: Get an inexpensive TV capture card and attach a video camera to it, configure so you can watch the live video on the computer, instead of reflective points use an infrared led+resistor+batery (have you ever point a remote control to a camera and click?). Some already mentioned zoneminder, you can use it as a starting point and as it is open source, you can hack it to your hearts content. Good luck!

  32. Inexpensive "real" camera for IR tracking: by Ayanami+Rei · · Score: 1

    Samsung SBC-331A.

    These run between $150 and $400 depending on where you get them and options.
    It's a standard B/W NTSC camera with a clock sync so you can chain them together. CS lens mount, with electronic iris/zoom lens control.

    Then you get yourself a few Osprey 1xxs in a mid-end server, and you can support 4 cameras pretty reliably. If you want to do more than 4, you might look at a Matrox Morphis; those are PCIe x4 or PCI-X, so they're a bit pickier. You could probably get 12 full frame streams from 3 of those on a suitably equipped system. But that's kinda expensive :(

    --
    THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
  33. Camera suggestion + some other stuff by isntwargreat · · Score: 1

    Hi. A friend and I have been writing some computer vision software for precisely this scenario. We intend to market it to consumers like you for a low cost (> $200) but it's not available yet. That said, a couple of things come to mind:
    First of all, the library of choice for all things computer vision is Intel's OpenCV library. They have a linux version. I know nothing about the linux library, but the windows version only supports a few webcams. The cheapest and best for the money is the usb 2.0 Logitech quickcam pro, which runs extremely well thanks to intel's performance primitives and windows' direct show. I doubt that you will have much trouble tracking at 24.0 fps which should be fine for you (I mean don't kid yourself, it's not like the results of your blobs analysis is going to be 100% consistent from frame to frame anyway). Programming a robust/complete motion capture solution could take months, if it works at all (assuming you have never done anything computer vision related). Outsourcing might be a better solution.

  34. Your idea is good, but the implementation... by cr0sh · · Score: 2, Informative
    ...is the most important thing. First off, I want to give kudos to everyone who has responded to these - just about everyone here has given great ideas and suggestions to the problem. These ideas should be listened to and evaluated. I myself have been researching the idea of sourceless and sourced motion capture and position tracking for a long time now, as it relates to virtual reality applications, simply because there is nothing commercial for the task that comes down to homebrew pricing. Personally, I only want to track two things - the position and orientation of my hand, and the orientation of my head. The second I (and anyone else) can easily do today with a cheap 3-axis accelerometer/compass system. The first, though, is not easy at all. Position is one thing, but the orientation is a completely different beast.

    With that said, your ideas on using webcams is spot on, but you are going to need more than three, mainly for occlusion handling. For the rig I was contemplating (using webcams much the same as you), I was thinking of at least four cameras. The main problem I ran into (just in thinking about it, no actual implementation), and as others have described, was timing issues. For best results, you need all the frames captured from the cameras to happen at the exact same time. Since with USB webcams this isn't possible, you either need to come up with another solution (people here have mentioned some "high end" cameras that have syncing systems), or deal with it in software (very difficult to do, in addition to dealing with everything else, and still getting a high frame rate).

    Another problem you are going to run into (and has been mentioned by others, but not much on the reason) is webcamera resolution. Most webcams that capture at decent framerates do so at QVGA (320x240). Even those that capture at a real 640x480 typically do so at only around 15fps, instead of 24 or 30. Rare (and more expensive) is the webcam that will capture at 24-30fps with VGA resolution. Even at VGA resolution, though, you are going to have to deal with the angular vs pixel resolution of the camera. What I mean by this is that as an object moves throught the FOV of the camera, it is going to only be imaged by certain pixels of the CCD imaging device. Depending on the distance away from the camera, the object may move say a foot, and only move (on camera) a pixel or so. The further away the moving object, the fewer pixels covered due to parallax. This translates into a lower resolution of pixels (on camera) to inches/cm (in real motion). In fact, this is almost the inverse problem of HMDs, where you can have high resolution, and low FOV, or vice-versa. In order to have both (in either cameras or HMDs), you have to pay a lot of money. In optical camera-based mocap, this means HDTV or better resolution cameras. I hope you understand what I mean here, because it is important for motion capture where you may be capturing large amounts of motion over a lot of area. For close-ups (like facial capture) it is less important - but remember, the higher the resolution of the camera, the finer the motion you can capture at all distances from the person/object to the camera. Higher resolution cameras translate into higher prices for the system, because you have to deal with more data, all in realtime. Not easy, not cheap.

    You might best be able to deal with this by going the custom camera route. What you would want to do is build a custom frame capturing system, using 640x480 (or better) b&w CCD cameras (you don't need color, you just need IR sensitivity - even with B/W cameras, you are going to filter the final image down so far that it is mostly only a true b&w 2bpp image - so the closer you can do that in hardware, the less you have to do in software). This won't be easy, but many people have done similar systems for homebrew robotic vision systems, so look there. Realize that this kind of a project will likely dwarf your game development project in both hardware and software needs, and you might end up with a system

    --
    Reason is the Path to God - Anon
  35. Webcam to 35mm (i.e. m42) by monopole · · Score: 1

    What you mean is a M42x1(42mm dia 1mm pitch Pentax, Practika, "universal" screw mount lens) to a M12x0.5(12mm dia 0.5mm pitch webcam and security cam) adapter.
    The trick is to get the adaptor from the astrophotography community which uses telephotolenses for astrophoto purposes. The adapters from Steven Mogg work well and incorporate a 1/4-20 tripod screw for mounting.
    http://webcaddy.com.au/astro/adapter.htm
    Also note that the FireWire Fire-I has an optional C mount lens adapter:
    http://www.unibrain.com/Products/VisionImg/Fire_i_ BC.htm

  36. PointGrey by cecille · · Score: 1

    We developed a motion capture system for a bit of a different application (6-DOF joint movement tracking for biomed research). Getting the cameras working is trivial compared to the processing required to actually get 3d motion capture working reliably. Of course, we were going for something that probably has to be much more accurate than what you need, but it's not a trivial manner to write the software for something like this. Of course, there may be stuff out there you can use. Anyway, here's a brief rundown of what we were using and what might help.

    We used 3 point-grey cameras (the flea). They might be a bit expensive for what you want, but all their cameras are top-quality, and they are intended for use in computer vision applications, and thus come with some source code pre-written that you can use for the interface. http://www.ptgrey.com/

    We actually used windows machines at the request of the lab, but we did originally look into linux. The cameras are firewire, and the best linux drivers we've found for these types of cameras are the the libdc1394 drivers on sourceforge: http://sourceforge.net/projects/libdc1394/

    The Open computer vision library is also invaluable. It has a lot of pre-written functions to deal with the more basic processing problems. It's got most of the major filters and algorithms in there that you'll need to extract the info from the camera pictures. Here: http://sourceforge.net/projects/opencvlibrary

    I'm not sure how you're planning on combining or calibrating the system, but we used a static set of known coordinates and used DLTs to actually give the real 3D coordinates. A good tutorial is here: http://www.kwon3d.com/theory/dlt/dlt.html

    Lastly, good trackers can really help the processing a good deal. Our trackers used a 4-ball system because we needed the accuracy and refrences for the angular rotations, but even a 1-ball tracker can be well designed. If the ball is a significant bright point on the image, simple thresholding is all that is needed in terms of preprocessing before you extract location in the image. Reflective paint or another bright source is KEY. If you're going colour, a distinct colour is also a good option.

    good luck.

    --
    ...no two people are not on fire.
  37. open source does not imply GNU/Linux by yuktar · · Score: 1

    are supported and easily programmed under GNU/Linux, since we'd like to later release our software as open source

    I'm wondering, if you're intending to release later as open source, then why does that necessarily mean that it must be supported under GNU/Linux? Seriously, open source can be for any operating system. Just pick the OS or vision API that makes camera interfacing the easiest, and write your code for that, while keeping the main parts as portable as possible. Open source shouldn't automatically imply either Linux as the system or GPL as the license (I am simply inferring GPL from context clues).

  38. Wiii by ripnet · · Score: 1

    Strap a nintendi wii-mote to each joint. These things are supposed to be basically bluetooth position and motion detectors.
    g

  39. mocap by Anonymous Coward · · Score: 0

    Hi Cliff,

                        I thought of doing a similar thing a while back, and bought a couple of webcams for the purpose. The cams I got were just a generic brand (I think they were called stareye or something). The trick to getting them to record a decent framerate at a decent res was a matter of tweaking. I used an MS program for capturing, but most capturing programs should have the basic options needed for tweaking. The main thing to do is to turn off any exposure options! these are handled by the camera, and no webcam will give you decent framerates while trying to correct lighting conditions. The program you'd want to use if you're doing this on the cheap is called Dgeeme. It's freeware and has brightmarker support. It is clunky as hell and a lot of work in the tweaking department to achieve results that are in anyway timesaving. To use this or any other optical system you will need a very controlled environment, that you can dedicate soley to mocap. It'll take a lot of time to get your setting right, and may be pretty tempremental to change. There's a software called Simi motion capture that costs around $4000, that was tested on webcam footage.Something to keep in mind for Dgeeme, is that the only way I got anything remotely useful was to set up a physical cube frame large enough to be clearly visable for calibration purposes but it has to remain in the capture scene the whole time. It takes a BIG working area to be able to achieve optical mocap.

    oh, and if you have any frame dropping during capture, dgeeme won't read it. If you're here in Australia, I've got my own mocap setup (gypsy 3), and would consider trading BVH data for programming depending on the project you're working on.

    Anyhow, Good luck with it and email me to let me know how it comes along.

    I'd appreciate any info on your game Ideas as well, if you've got a website, send it my way.

    best regards,

                                          -Heath Freeland(heath_freeland_3d@yahoo.com.au)

  40. DIY motion capture by yoshi-mocap · · Score: 1

    Hello, I am making a motion capture system with NaturalPoint's OptiTrack cameras: http://geocities.com/mocap_is_fun/ Here is a sample video: http://www.geocities.com/mocap_is_fun/mocap34.wmv It can output BVH which can be loaded by other apps like Poser: http://www.geocities.com/mocap_is_fun/Poser.wmv