Slashdot Mirror

← Back to Stories (view on slashdot.org)

3D Mars Scenes Recreated From Photos

Posted by timothy on from the back-formation dept.

Hoon Mihn Fao links to this BBC article about Microsoft Research scientists generating 3D models based on pictures beamed back from the Mars rovers. It begins "Using techniques originally applied to paintings, the researchers have written software that works out what flat images would look like from different viewpoints. The result is a navigable 3D model created from one or more pictures snapped by the Mars buggies."

24 comments

  1. Am I the only one who's thinking... by Ayaress · · Score: 2, Funny

    (insert favorite FPS game here) map?

    1. Re:Am I the only one who's thinking... by Patrik_AKA_RedX · · Score: 2, Insightful

      A FPS map of a Mars desert? Am I the only one who thinks that would be boring?
      A desert is a collection of sandy hills with an occasional cactus and Mars even lacks the latter.

      And there aren't any pictures of the secret Nazi Mars base, but don't worry, we've send our best Castle Wolfenstein player to deal with it.

  2. Mars in Stereo by stereo_Barryo · · Score: 5, Interesting

    To repeat myself, my friend Robert Clemenzi has put up a page on Mars in stereo and has written a program, that can be downloaded, that allows you to study the pictures from the rovers in either cross-eyed stereo or over-under. http://www.cpcug.org/user/clemenzi/science/MarsIma ges_3D.html

    1. Re:Mars in Stereo by Anonymous Coward · · Score: 0

      My eyes hurt when I cross them, and I don't see anything, gonna wait for some 3d (blue and red) glasses version.

  3. Where's the video? by Gyler+St.+James · · Score: 4, Interesting

    Wow. They said something to the effect of being able to see other objects that was not completely visible in the image. "We work out what's behind an object and how we can automatically fill in what's not seen by cameras or painted." I wonder if this would allow you to rotate around objects that are halfway hidden, but what about asymmetrical objects? How can they extrapolate what's not known? Interesting none the less, but where's the video they talked about?

    --

    1. Re:Where's the video? by Uncle_Al · · Score: 5, Informative

      Look there:
      Pictures and Videos

      Have fun...

    2. Re:Where's the video? by Tablizer · · Score: 3, Interesting

      How can they extrapolate what's not known?

      I don't think they do. You can calculate the position of most objects actually imaged in images taken from multiple viewpoints of the same scene using a modern version of old-fashioned triangulation like surveyers use, but you still have no idea what is behind a rock, for example. At best one can guess what the back looks like based on the front (if back not imaged).

      But I notice in some of the 3D models used by the Mars teams that "unknown" areas are filled in with gray. For example, a rock may have a gray "tail" for the areas that were not imaged. It is kind of a "viewpoint shadow". It might be safer scientifically to just have a gray tail instead of guess. Otherwise you may accidently plan rover journeys to places not actually imaged.

      I notice I tend to do the same thing in my brain. I build a 3D model of the scene, and mentally fill in the hidden parts with kind of a grey or nebulous fuzz. If safety is not an issue, then I might extrapolate based on what is visable, but if safety is an issue (such as driving), then I tend to color the unseen area with a vivid color (like orange) that means "be careful, unknown territory".

      --
      Table-ized A.I.
    3. Re:Where's the video? by theslashdude · · Score: 1

      Am I the only one who was expecting a flying-through-the-landscape type video rather then just a little wobble?

  4. NASA did this from day one by CXI · · Score: 1, Insightful

    So, NASA has been taking stereo images since the beginning and making true 3D models, and now Microsoft has figured out how to make fake stereo images and fake 3D models?... Um, nice step backwards?

    1. Re:NASA did this from day one by Xilman · · Score: 3, Insightful
      Try reading further down the page. They are taking a single image and building a steroscopic view. NASA were using pairs of images in the traditional manner.

      Paul

      --
      Lasciate ogne speranza, voi ch'intrate
    2. Re:NASA did this from day one by Anonymous Coward · · Score: 5, Informative

      Grandparent said:

      So, NASA has been taking stereo images since the beginning and making true 3D models, and now Microsoft has figured out how to make fake stereo images and fake 3D models?... Um, nice step backwards?

      Parent said:

      Try reading further down the page. They are taking a single image and building a steroscopic view.

      Try understanding what you read. :) Making a stereoscopic view from a single image is a step backwards. They're interpolating to "resolve" information that doesn't really exist. It's like when crime dramas zoom in about 10x and then magically clean up the image to see details that were not captured on a security camera! If you want to extract depth, you need two pictures taken from separate locations. Otherwise you're just making educated guesses.

      X...Y...Z
      ....w....

      A.......B

      Picture from camera A: X...Yw..Z
      Picture from camera B: X..wY...Z

      Please don't tell me you honestly think you can tell that w is in front of Y just by processing the picture from camera A (unless you know the exact dimensions of the WXYZ elements). You need the additional information provided by camera B. Consider a picture where you see a tall person and a short building. You might assume the person is standing in front of the building, but what if it's a miniature building in the foreground? Unless you have a 2nd picture, you can't know for sure.

    3. Re:NASA did this from day one by CXI · · Score: 1

      Try reading further down the page.

      Try reading my post. It is a step backwards to take a single image and generate fake data from it, when we already have real data from multiple stereo images!

    4. Re:NASA did this from day one by Xilman · · Score: 2, Informative
      Again: try reading the article. It's (relatively) easy to generate stereograms from two images taken from slightly different positions. That's been done for decades.

      What is non-trivial is generating a stereogram from a single image. In some circumstances only a single image is available and another will never become available. Are you seriously claiming that it's a step backwards to be able to generate a stereogram from a single image?

      The research was first applied (AFAIK) to Renaissance oil paintings where, almost by definition, only a single image from a single viewpoint at a single instant in time is available. They managed to produce very convincing 3D models of the scenes and, from them, stereograms.

      There are planetary images in existence which are essentially unique. The techniques of Criminisi and Blake allows 3D models and stereograms to be created from those images too.

      Paul

      --
      Lasciate ogne speranza, voi ch'intrate
  5. More Mars stereo goodness... by TheAngryArmadillo · · Score: 3, Informative

    A coworker of mine and I wrote a script to download and generate every possible 3D image from NASA's MER website. It goes through and finds all matching left and right images then makes them into an anaglyph.

    If you've got your red/blue glasses you can see them here. We update them with new images every morning. Some of the images are useless but there are also a bunch that NASA never generates for us. They are separated by rover and Sol.

  6. Other uses? by Pumpernickle · · Score: 4, Funny
    "We extract 3D information out of the image," said Mr Criminisi. "We work out what's behind an object and how we can automatically fill in what's not seen by cameras or painted."

    Would this technique work, for, say, objects hidden or obscured by some sort of flimsy semi-transparent thin fibrous material? Or only hard, rock-like substances? ^^
    1. Re:Other uses? by Anonymous Coward · · Score: 0

      Hmm, x-ray specs. They've been out for years. :)

    2. Re:Other uses? by dnahelix · · Score: 1

      "we can automatically fill in what's not seen by cameras"

      That's where the little green men come from!



      --
      Slashdot Eds Link Anonymous Posts With Logged Posts
      They Are Vermin Feeding On Each Other's Feces.
      I Hate \.
    3. Re:Other uses? by mandolin · · Score: 1
      I have (not) researched this carefully, and the answer to your 1st question is "yes", as long as the object also satisfies the condition raised in your second question

      IOW, hope you like dick.

  7. Are they interpolating between two cameras? by enosys · · Score: 2, Informative
    The range of motion seems extremely small. Are they just interpolating views from between two cameras?

    This sort of thing isn't new. Panorama Tools has PTInterpolate, which given two images of the same scene taken from different viewpoints creates any intermediate view. I've never tried it but this makes me want to.

    1. Re:Are they interpolating between two cameras? by kazbek · · Score: 1

      I agree with you. So they aren't making up any detail, because they don't go beyond the stereo angle.

    2. Re:Are they interpolating between two cameras? by enosys · · Score: 1

      You may have to make up detail even in this case. Imagine you have two photos of a fairly narrow slit. At the midpoint between these two photos an object can be seen through the slit, but from the viewpoints of the photos themselves this object cannot be seen. This sort of situation can happen with for example ground with lots of rocks on it.

  8. Better Format? by MagnusDredd · · Score: 1

    Anyone have a copy of the movies in an open format???

    Both Video Lan Client, and Mplayer refuse to play this thing.

    FFMPEG doesn't support it yet, I don't think.

    I'm becoming convinced that the only reason "WMV3" encoding exists is to lock out non-M$ users. There are a million different formats that work well enough and are open.

    What else should one expect from M$?

    Links are:
    http://research.microsoft.com/~antcrim/mars_ web_MS RC/MarsVirtualView.wmv
    http://research.microsoft. com/~antcrim/mars_web_MS RC/MSRC_mars2.wmv
    http://research.microsoft.com/~ antcrim/mars_web_MS RC/MSRC_mars3.wmv

    1. Re:Better Format? by HacTar · · Score: 1

      try this

      http://research.microsoft.com/~antcrim/mars_web_ MS RC/MarsVirtualView.mpg