Slashdot Mirror


Disney Research Creates Megastereo - Panoramas With Depth

mikejuk writes "Disney Research has made a breakthrough in implementing the technique of acquiring depth information from a simple camera scan of a scene. For a perfect panorama you need to rotate the camera around its optical center, i.e. just rotate the camera. However, if you just rotate the camera about itself you don't get any parallax effects — which is why it makes the stitching together easier. If you want to get 3D information from the sequence of shots you need parallax. This means rotating the camera mounted on an offset arm or just moving the camera along an arc in your outstretched hand. The big problem with this method is that the parallax makes it more difficult to fit the mosaic together, and this is the problem that the research team has been working on. Using a range of different scanning methods the results can be converted into high resolution panoramas automatically complete with 3D information."

48 comments

  1. More psuedo-3D by Animats · · Score: 1

    Nice. Given Disney's interest in converting 2D movies to fake 3D movies, it's not surprising that they're investing in this. It looks like they're able to do this without having to create imagery to fill in occluded areas, which is a problem with some of the other approaches.

    Next, Google StreetView needs to be redone with this level of quality.

    1. Re:More psuedo-3D by Anonymous Coward · · Score: 0

      I'm pretty sure it's just a processing run for them (if they haven't already) and a matter of bandwidth for it to be usable.

    2. Re:More psuedo-3D by Romwell · · Score: 1

      Google StreetView already has a 3D mode, though, although its output is not that controllable (probably because it has been preprocessed into anaglyph - a pity for those with 3D monitors). The ability to adjust separtion/convergence would be a simple add-on, though. Also, Disney's panoramas are closer to real 3D than just a stereo-pair based imagery; there is one degree of freedom in viewing positions - and that's all you really need for most applications.

    3. Re:More psuedo-3D by chrismcb · · Score: 2

      You can't use this technique to convert and already filmed 2D to fake 3D. This technique works by filming those "occluded areas" already, much like a 3d camera would.

  2. Mega Stereo? by Bing+Tsher+E · · Score: 0, Troll

    When I was a kid my dad had a big console stereo in the living room. I believe it's brand was Columbia. It was huge and long, about the size of a sofa.

    Back then, Walt Disney had a television show on Sunday nights that everybody watched. He'd personally introduce an interesting short film that we'd all watch.

    It's a shame what corporate interests have done to his name.

    1. Re:Mega Stereo? by Anonymous Coward · · Score: 0

      I don't really see how research is affecting Walt Disney's good name?

    2. Re:Mega Stereo? by phozz+bare · · Score: 1
    3. Re:Mega Stereo? by The+Cat · · Score: 1

      It wouldn't if the research weren't being performed in 婿¥å æ¥ å å±±è¥çoe æ(TM) è'(TM)åèæè'(TM)(å...è'(TM)å)

      Two miwyun Disney Ni Hao Princess plastic doll you buy?

      Disney has nothing at all to do with Walt Disney any more. They are a giant distended poisonous gullet that eats decency and craps hate.

    4. Re:Mega Stereo? by SharpFang · · Score: 1

      Unfortunately Walt Disney wasn't the sweet papa people sometimes picture him. During the witch hunt for the commies lots of his employees got arrested - thanks to him extending the hunt to the inside of the company.

      --
      45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
    5. Re:Mega Stereo? by Anonymous Coward · · Score: 0

      Interesting. I've read four biographies of Disney over the years and there was no mention of red-baiting within his company or employees being arrested for their political beliefs. But it sure makes a cool story. Especially if you have a knee-jerk loathing of entrepreneurs.

  3. Curious by StripedCow · · Score: 2

    Are there any algorithms out there that can take a movie, and produce a sharp photo (or a series of sharp photos)?
    By "sharp", I mean much sharper than each of the individual frames of the movie.

    I.e., the algorithm should use information between subsequent frames to sharpen the image.

    --
    If Pandora's box is destined to be opened, *I* want to be the one to open it.
    1. Re:Curious by Anonymous Coward · · Score: 3, Informative

      Yes there are. See http://en.wikipedia.org/wiki/Superresolution for methods and limitations. They are not yet in movie-scale but can convert a sequence of images from a single rigid target into a more detailed one if sub-pixel accurate registration is done between images.

    2. Re:Curious by NekoYasha · · Score: 1

      Yes; search for "super-resolution".

    3. Re:Curious by I'm+New+Around+Here · · Score: 3, Funny

      Sure, I see Abby do that on NCIS all the time.

      It works best if it's a reflection on a pair of sunglasses, though.

      --
      If you think I voted for Trump because of this post, you're wrong. I voted for Dr. Jill Stein of the Green Party. Again.
    4. Re:Curious by chrismcb · · Score: 2

      You have to have someone else stand over your shoulder and say "enhance image" otherwise it won't work.

    5. Re:Curious by drkim · · Score: 3, Interesting

      Are there any algorithms out there that can take a movie, and produce a sharp photo

      Of course. And it's free:
      RegiStax
      http://www.astronomie.be/registax/index.html

      It's mostly used by astronomers, but works quite well for any series of images. Of course, it works on repeating frames. If things are moving, them they aren't 'repeating' and won't be processed correctly.

    6. Re:Curious by chinton · · Score: 1

      Various CSI teams from Las Vegas to Miami have been using these algorithms years before they were adopted by the Navy. Although, the civilian applications seemed to be stronger at inferring pixels, thus allowing a near-infinite zoom.

    7. Re:Curious by Anonymous Coward · · Score: 0

      Theoretically, you could use a 'traditional' panorama software to align your many overlapping frames, then use registax on the overlapping strips and re-assemble the higher resolution stips again with the 'traditional' panorame tool

  4. Disney is seriously bad. by Anonymous Coward · · Score: 0

    They are largely responsible for never-ending copyrights which are really really bad. So screw them and everything they do.

  5. Reinventing the wheel by jofas · · Score: 1

    The author has a weak grasp on how stitching software is affected by parallax. Also, it would still be technically easier to to with two cameras or a stereo camera.

    1. Re:Reinventing the wheel by Romwell · · Score: 2

      Also, it would still be technically easier to to with two cameras or a stereo camera.

      Yes, it will be, but there is a serious lack of 3D stereo equipment on the market today. Affordable stereo cameras are not of high quality, and I have yet to see one with adjustable lens separation. Two camera solutions come at twice the cost, size, weight and battery worries.

      Also, if I can't afford one D3x, I surely can't afford two of those plus a sync kit and a tripod, nor would I want to carry such a setup with me on a hike. There are plenty situations when all you have is one camera, and you want to make the most with what you have.

      Additionally, there's plenty of panoramic data out there shot without a panoramic tripod that can be fed straight into their algorithm. I know I have plenty, since I've shot many panoramas, but never had a pano head (and many of those were shot hand-held).

      Therefore, while this may not be the best solution to shoot a 3D panorama, it still has plenty of applications in processing existing data or dealing with situations where specialized equipment is not available.

    2. Re:Reinventing the wheel by Anonymous Coward · · Score: 1

      Their algorithm works best with images taken no further than 4 degrees apart. They have examples with 8 degrees between the images, but it starts to fall apart at that point. With 4 degrees between successive images, you'd have to have 90 images for a full 360 panorama. Unless you've been filming your panoramas, you've probably only been taking about 12 images full circle, and not in the right geometry for their algorithm either.

  6. Amazing! 3D and panorama enthusiasts rejoice. by Romwell · · Score: 2

    I've been a long-time panoramic and 3D photography enthusiast, and have gigabytes of data that could be fed straight into this software.

    Shooting a 3D photograph is easy (just take two frames, correct issues in software later); shooting panoramas is easy (let the stitchers do their job); shooting a 3D panorama has always been too much work for me.

    It's a pity I can't get my hands on any working code yet, and any commercial product is probably way off in the future.

    Also, here's a link to their paper[PDF] for those interested. It's quite readable.

  7. Very old news by Begemot · · Score: 5, Insightful

    It's based on this invention: http://www.vision.huji.ac.il/stereo/

    Yael Pritch (who was involved in this project in Disney) was also involved in this research in the Hebrew University.

    So it's a breakthrough, but a very old one and, more importantly, somebody else's

    1. Re:Very old news by Romwell · · Score: 1

      Thank you for the link! It's old news for you, but some of us are not so up to date.

      It seems like some of this research might have been implemented in this commercial product. I wonder if it is so.

    2. Re:Very old news by phozz+bare · · Score: 1

      Cool of you for finding the original link! I actually remember this being exhibited at the HUJI Open Day fair in 1999, and (as a prospective student) had a chat with Yael who explained a bit about how it worked. I'm happy to see that this little project has progressed.

      Your criticism however seems out of place; if Yael is a Disney Research employee then this is a Disney Research invention, it is not "somebody else's". I'm quite sure that the team has not been sitting idly these past 14 years, and that the breakthrough in question is not what was shown back then but the fact that the technology is closer to being an actual product.

    3. Re:Very old news by Begemot · · Score: 1

      I do not criticize Yael, she's super cool. I do criticize the language of TFA that made it look like a new and original work, and not an improvement over a 15 year old invention, whose original inventors are Prof. Shmuel Peleg and Bob Rosenschein (the former CEO of Answers.com). Yael's current paper expands on it in details.

    4. Re:Very old news by neonsignal · · Score: 1

      Which points to the shocking state of the patent system, where 'omnistereo' can be patented in 2011 despite having been openly published years earlier. Blimey, even stereoscopic movies were still being patented in 2005! (oh yes, that 'rectilinear' makes all the difference...)

    5. Re:Very old news by Begemot · · Score: 2

      Sure, pleasure. Re commercial products. I found these guys: www.humaneyes.com and www.snapily3d.com. Not sure how successful they are. The app seems to be struggling (probably heavy duty calculations are not ready for mobile yet). I simply looked up what Yael Pritch did in the past, apparently she was the founder of this company. Also check out this: http://www.humaneyes.com/News/detail/130/66/. Apparently Sony have infringed this patent.

    6. Re:Very old news by retroworks · · Score: 1

      Disney linked to recycling very old tales? 3D Pics, or it didn't happen!

      --
      Gently reply
    7. Re:Very old news by Romwell · · Score: 1

      I have used Snapily3d services as of this year - so they are active at least - and was happy with their service. Didn't know about HumanEyes until today, though.

  8. Already available? by Doh! · · Score: 1

    Isn't this similar to the 3D Sweep Panorama feature that has been available in some Sony cameras and smartphones since 2010?
    http://www.sony.co.uk/hub/cyber-shot-digital-cameras/technology/article/3d-sweep-panorama

  9. Disney Creates? by onyxruby · · Score: 3, Informative

    Anytime someone reads Disney creates they should substitute the word "copies" until proven otherwise. Disney has a long and storied history of intellectual property theft when it was in their best interest. They are arguably the greatest hypocrites in the world about IP, even more so than Hollywood themselves. They are always the ones that hold the most radical of views in the MPAA and are very quick to hold condemn anyone else and take away their rights. They have also been stealing from the public domain and other individuals for ideas for decades. A quick Google search can find example upon example of their bad behavior.

    1. Re:Disney Creates? by Anonymous Coward · · Score: 2, Informative

      Disclaimer: I used to work for Disney Research.

      This may be true of other parts of Disney, but Disney Research is only a few years old, is managed by researchers, operates pretty much independently like an academic institution (though with the goal of developing tools the rest of Disney can use) and everything I've seen has been to the highest standards of scholarship. Take a look at the SIGGRAPH publications here: http://kesen.realtimerendering.com/ From the last few SIGGRAPHs and SIGGRAPH Asias almost 10% of the papers are from Disney Research (and they're mostly good ones too).

      A comment above says "So it's a breakthrough, but a very old one and, more importantly, somebody else's" -- but if you actually open the paper, you'll see that the very first comparison is to that previous work and the improvement is getting the quality and speed to the point of being usable. Anyone who's done research knows there are (almost) no "breakthroughs" -- only building on ideas prior researchers head.

  10. Re:It looks quite nice in Star Wars 7 by Billly+Gates · · Score: 1
  11. cross-eye by mapkinase · · Score: 1

    I repeat: cross-eye is the only stereogram that does not require additional equipment AND allows normal 2D view if you do not feel like crossing your eye.

    Why protein structure journals got it for at least 20 years now and general purpose 3D photography cannot get it?

    --
    I do not believe in karma. "Funny"=-6. Do good and forbid evil. Yours, Oft-Offtopic Flamebaiting Troll.
    1. Re:cross-eye by Anonymous Coward · · Score: 0

      cross-eye is the only stereogram that does not require additional equipment AND allows normal 2D view if you do not feel like crossing your eye.

      Well, there's also walleye (which is basically the same thing as cross-eye, but with the images flipped.) But you're right, side-by-side stereograms are unique in that regard.

      Why protein structure journals got it for at least 20 years now and general purpose 3D photography cannot get it?

      Pedantic nitpick - protein journals typically do walleye stereo. Walleye is easier to do close up, whereas crosseye is better at a distance .

      But the more general answer is that free viewing a side-by-side stereogram is hard for most people. I work in a lab where all we do is work with protein structures. Most people in the lab can't handle free viewing structures - despite the fact that when I pull something up in PyMol, they invariably say "oh, cool. You can do that 3D eye thing? I wish I could do that, I could never get the hang of it." It makes your life so much simpler - I couldn't imagine trying to work with protein structures without the depth perception that the (walleye!) side-by-side stereogram viewing gives you. I have no clue why free viewing is hard for most people, but I don't think it's from lack of trying. A large number of people need to have the assist of glasses to get the image fusion, despite the fact that the eye motion is exactly identical to looking up from the paper to a distant object.

  12. Old Panoramic Paintings by Anonymous Coward · · Score: 0

    I wonder if it may by applied to old panoramic paintings:

    http://en.wikipedia.org/wiki/Panoramic_painting

    http://en.wikipedia.org/wiki/File:Panorama_of_London_Barker.jpg

    I suppose it depends of how they were made.

    1. Re:Old Panoramic Paintings by neonsignal · · Score: 1

      Unfortunately it can't; the method they are talking about relies on the taking of multiple shots that overlap. Because the camera is mounted on a swing arm, the overlapping shots are also taken at different locations, which enables depth to be reconstructed from what are effectively stereo pairs. The only tricky part is that the stereo pairs have been taken facing in different directions, so the algorithm has to compensate for this.

      While it is true that the panaromic painting may have been observed from different points at a tower, each spot in the scene only corresponds to a single point in the painting, so the stereo information has been lost.

  13. Re:Please.... by Anonymous Coward · · Score: 1

    You sure don't speak for as many people as you think you do. Otherwise there would be no market for it.

  14. Hollywood does this all the time. by Anonymous Coward · · Score: 0

    Many CGI effects simply 'redress' a real environment with 'digital makeup', say adding 'damage' to buildings. The algorithms that allow this are part of proprietary programs used by many special effect houses across the planet. Essentially, you shoot video footage, and the program calculates depth geometry to aid in extracting non-changing textures from the scene. The texture data is presented to artists as 2D 'photographs' that the artist can re-touch (adding the afore-mentioned building damage, for instance). The program can now apply this 'new' texture data to the original video, placing the artists work in the context of moving video shots.

    In other words, imaging photo-shopping a SINGLE photograph of the front of a building. Now imagine shooting video including that building. Now imaging that your single photo-shopped image of the building appears throughout the video in the correct place.

    This cheap-n-cheerful approach to CGI doesn't stand up to close scrutiny, so it tends to be used for background shots that won't challenge the lack of true in-scene 3D re-modelling.

    And, of course, modern computer algorithms allow us to grab 3D scene data from almost any video, allowing the insertion of synthetic 3D CGI elements that track the camera movements perfectly.

    1. Re:Hollywood does this all the time. by The+Cat · · Score: 2

      This allows Hollywood to film a two-hour forty minute movie about a guy impersonating Superman who kills people with his bare hands and lets hundreds of thousands of people be brutally slaughtered while he punches a guy and calls him profane names.

      Hey Hollywood. Put After Effects down and hire a writer. Nobody gives a fuck about graphics any more. We're sick of watching cities get destroyed. Hire a fucking writer.

    2. Re:Hollywood does this all the time. by Spy+Handler · · Score: 1

      They hire too many writers already. Typical Hollywood movie goes through 5 writers and script doctors.

  15. Where by The+Cat · · Score: 1

    Disney researchers in Guangdong Ni Hao Liaoning Guangxi Zhuang performed the research.

    It's logo is Mickey Ni Hao Mouse

  16. HAFLER 3-D IS GOOD ENOUGH !! by Anonymous Coward · · Score: 0

    Center channel is L + R and WOW !! You are there !! Blow them Nikkos up real good !!

  17. Into *new* 3d/Photogrammetry? by Anonymous Coward · · Score: 0

    Check this out, an entire film made from SFM. Took me 3 years, did a lot of this *kinda* in the process of making my world, I just dropped photos all together and went for Point Clouds instead.
    http://youtu.be/TSOJl8SNZLM

  18. Re:Please.... by sjames · · Score: 1

    Sales figures (where it is possible to differentiate) suggest that 3D isn't really much of a selling point.