Slashdot Mirror


Technology for Capturing 360 Degree Video

Inzite writes "EnterNetica R&D is working on a new spherical video technology for capturing and presenting full 360 degree scenes using a 180 degree lens, by adaptively predicting the camera's surroundings. Video extrapolation techniques have been proposed in the past, but this is the first time I've heard of an entire hemisphere of the video image being "guessed". The article also talks about feature film presentation using fully-immersive video in the future."

45 of 151 comments (clear)

  1. Why not two cameras? by AKAImBatman · · Score: 3, Insightful

    Maybe I'm missing something, but the article doesn't explain why they don't simply place two cameras back to back, then use their software to splice the result and apply the correct perspective. All they say about it is that "it's against the laws of physics to take a 360 degree photo. That just seems odd.

    1. Re:Why not two cameras? by AKAImBatman · · Score: 3, Informative
      I knew I'd seen this technology somewhere, so I went digging. Here's a story that discusses IPIX using two fisheye lenses back to back.

      NetCam is a development of the original iPIX bubble camera, a unit that featured a pair of cameras with fish eye lenses in a single housing the size of a standard SLR camera. What was so special about the iPIX bubble camera was the fact those CCDs and lenses were facing away from each other.


      (Emphasis mine.)
    2. Re:Why not two cameras? by Surt · · Score: 4, Funny

      Well duh, they can't put two cameras back to back and take a 360 degree photo, that would be against the laws of physics. And man you do not want to be pulled over by god for a physics violation, that is a bad scene.

      --
      "Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
    3. Re:Why not two cameras? by zev1983 · · Score: 4, Informative

      It is odd seeing as there is a lens system in existence that can take 360 degree pictures. It uses a funky lens that goes all the way around like a donut IIRC, and the camera is mounted below the lens assembly... OH, here's one http://www.sony.net/Products/SC-HP/cx_news/vol34/f eaturing2.html and here http://www.bugeyedigital.com/product_main/036-0360 d.html , hell just type in 360 lens into google and tons of stuff pops up. Looks like the people marketing this are full of shit, and don't know anything about physics, or optics. Article is /.ed so I can't check out anything past the /. summary and what you said.

    4. Re:Why not two cameras? by op12 · · Score: 3, Funny

      He let me off with a warning.

    5. Re:Why not two cameras? by 't+is+DjiM · · Score: 4, Informative

      This lens does not offer you the possibility to record a full 360 image... There is no data about what can be seen below and above the viewer. What they do is creating a spherical image, not just a doughnut (which is what your "funky lens" is capable of).

      The recorded image can then be viewed (my guess is by a kind of VR goggles) allowing the viewer full viewing freedom.

      Next time you say somebody is full of shit, make sure you're not mistaken yourself.

      --
      --Use ant to make .war
    6. Re:Why not two cameras? by modecx · · Score: 3, Informative

      You're confused by the terminology, I think. The guys in the article want to capture a full spherical image (which is possible with 180 degree fisheye lenses, taking 6 pictures at various angles, and stitching them together), those lenses you presented capture 360 degrees in a cylindrical fashion, also cool, but not what they want.

      I've seen sperical quicktime files, a LONG time ago. They were neat.

      --
      Constitutional rights may be respected, repealed, or modified; but they must never be ignored.
    7. Re:Why not two cameras? by TheGavster · · Score: 2, Informative

      The issue that you encounter with taping two cameras together is that the depth of field is going to be different between them because the pupils of the cameras are at different points. When taking still panoramas from multiple shots, you can compensate by using a mount that swivels directly about the pupil of the camera (most mounts do not), but you can't so that with video.

      The other awesome thing about this software is that you can apply it to prexisting footage. It separates the background, and stitches together a full sphere from an entire scene. As long as the camera pans around a decent amount, this technique gives you a spherical panorama from a traditional video source.

      --
      "Because Science" is one step from "Because old book". Try "Because of my experiment testing my falsifiable assertion".
    8. Re:Why not two cameras? by eMartin · · Score: 3, Insightful

      "This lens does not offer you the possibility to record a full 360 image... There is no data about what can be seen below and above the viewer. What they do is creating a spherical image, not just a doughnut (which is what your "funky lens" is capable of)."

      The only requirement for "360 degrees" is that it see from a certain point in all directions on a single plane. Adding up and down means means adding another dimention to the process. Something that can cature a full spherical view would have to be called 360x180 degrees, where you capture 180 degrees of vertical for every one of the 360 degrees of horizontal.

      Or something.

    9. Re:Why not two cameras? by Traa · · Score: 2, Insightful

      my guess

      Let's see, TFA mentions that they did not in fact record a full 360 image, instead they record a 180 degree fisheye image and then "fill in the rest" based on smart guessing (no details provided). They happily adimit that the algorithm isn't perfect and that some environments work better then others. Without providing samples for us to see they allready admit that they had to go out of their way to come up with some samples that looked decent. I like the skydiving example they talk about, the algorithm is capable of filling in the empty sky behind you. Amazing!

      As for wether this will work in a regular scene (where stuff actually happends behind the camera), I'll just make a bold statement: It doesn't.

      Much better would have been if they used the camera as mentioned by grandparent, recording a 360 degree donut view, where at least most of the interesting information is captured and only the roof and floor need to be made up by the software (now that seems reasonable).

      I'll leave the stuff about pot and kettle for yourself to figure out (minor appology to gp would be nice, but I'm not counting on it).

    10. Re:Why not two cameras? by B1ackD0g · · Score: 2, Interesting

      The issue that you encounter with taping two cameras together is that the depth of field is going to be different between them because the pupils of the cameras are at different points.

      Correct me if I'm wrong here, but to project this, you have to make room for the viewer. Seems like the difference in perspective would be desireable. I'd think that you'd really want the camera's to be physically placed around the camera in as close to a spherical configuration as possible. This should leave room for the camera person, or, at least, the camera.

      My mind flashes back to a Superman comic book where Lex Luthor bends light around himself to become invisible. He is then left blind, because any flaw that allowed him to see out could be used by Superman to find him.

      --
      When I'm feeling down, I like to whistle. It makes the neighbor's dog run to the end of his chain and gag himself.
    11. Re:Why not two cameras? by fbjon · · Score: 2, Funny

      If you run really fast you'll disappear in the compression artefacts.

      --
      True confidence comes not from realising you are as good as your peers, but that your peers are as bad as you are.
  2. Great, now all we need is... by Stormwatch · · Score: 4, Funny

    ...a 360 Degree TV to watch it!

    1. Re:Great, now all we need is... by Anonymous Coward · · Score: 2, Funny

      Cool! I'll finally have something to plug my new XBox into.

    2. Re:Great, now all we need is... by xtracto · · Score: 2

      Just wait until George Lucas anounce the 360 version of the Star Wars hexology!

      --
      Ubuntu is an African word meaning 'I can't configure Debian'
    3. Re:Great, now all we need is... by justforaday · · Score: 4, Funny

      There goes the illusion that my pizza delivery job makes me irresistable to women.

      Well duh! You need to get into the plumbing business...

      --
      I'll turn into a supernova and burn up everything. Well I'll turn into a black little hole and you'll turn into string.
  3. Hidden messages by Anonymous Coward · · Score: 2, Funny

    Whilst the camera is looking the other way, someone could easily show in a message saying:

    FIRST POST

    but nobody would notice.

  4. Boo by Tomchu · · Score: 2, Insightful

    The site's already down, but based on the small blurb ... I don't see how this is possible. You're basically guessing at 50% of the scene, of which the only hints you have are land/sky textures. If that's all this is about, I've been doing it for years in Photoshop. It's called the Clone/Healing Brush tool.

    --
    I used to think Linux was cool -- then I turned 14.
    1. Re:Boo by AKAImBatman · · Score: 2, Insightful

      They explained the "how" in the article. Basically, the panning of the lense attempts to capture as much of the scene as possible. The final video is then put through video analysis software that uses motion prediction to stitch the future and past of everything visible based on the few clips it acquired.

      To put this in perspective, if you're panning through an aquarium, you'll see the fish swimming on as if nothing was wrong. However, while the camera isn't looking, a fish may get scared and dart off. Since that event wasn't captured, the software will show you the fish contining to swim in the final video.

      Unfortunately, things might get a little weird with more dynamic scenes. Consider taking video of a theme park like Six Flags Great America. Let's say during panning you capture a roller coaster as it's decending, but fail to capture anything after that. I don't know how complex their software is, but there's a high probability that it would show the coaster going right through the track and into the ground. Something like that would quickly spoil the illusion that the camera is truly 360 degree.

      Make sense? :-)

  5. first application... by ltwally · · Score: 2, Funny

    something tells me that something like this could usher in a new era of pr0n. 3d goggles and a vibrating chair, oh my!

    --



    /dev/random
  6. Google Cache Link by Anonymous Coward · · Score: 2, Informative
  7. Link / Movies? by RUFFyamahaRYDER · · Score: 4, Informative

    Here is google cache: thanks google.

    I can't wait to see what movie theatres do with this technology. Maybe not our typical theatres, but yenno... like the one's at theme parks or something. I'm sure can make a pretty cool short movie with this.

  8. Why guess? by sakusha · · Score: 3, Insightful

    The patent on Disney's Circlevision camera system has expired.

  9. Google search results by karvind · · Score: 3, Informative
    Seems like we killed it again.

    Google links on EnterNetica

    (a) OPTICS ADVANCES BRING VOLUMETRIC VIDEO TO LIFE

    (b) Pressbox link

    (c) Cleaner, Crisper Volumetric Images

    Company webpage

  10. Re:Woa! by WormholeFiend · · Score: 5, Funny

    This way we can be INSIDE of a pr0n movie!

    No frickin way!

    Assuming that the slashdotted article also describes a 360 degree viewing screen, you'd be looking at the porn in front of you, getting all aroused, and then you'd look behind you and see the filming crew with the dodgy looking pron director and his clapper.

    This would totally ruin the experience, IMO.

  11. Cinemoments by dada21 · · Score: 4, Interesting

    OT pondering instilled by TFA...

    In Jack McDevitt's Hutch series of books, the passengers on intersolar flights passed thetime by "starring" in movies digitally redone with the passengers as the characters.

    I was thinking about how hilarious this would be in real life, and how it could reinvigorate certain movies in theaters with minimal seating if they had decent hardware to sample random audience members (one person per group). I realized a fisheye lens can capture deptch with the right software.

    Imagine how "cool" it would be to revisit Indy Jones or Star Wars or Usual Suspects where someone in your group was one of the actors? Even a bit part would lead to great inside jokes, and meeting up with new groups would be easy, too.

    I'd spend $20/ticket for the social experience. //OT

    1. Re:Cinemoments by Datamonstar · · Score: 2, Funny

      Yeah, but us black guys would always end up dying somehow mid-ways through the picture.

      --
      The eternal struggle of good vs. evil begins within one's self.
  12. check out this ghetto 360 camera "hack" by enrico_suave · · Score: 4, Interesting

    360 degree camera hack

    pretty cool, simplistic yet inventive hack.

    e.

    --
    Build Your Own PVR/HTPC news, reviews, &
  13. VirtuSphere by mysqlrocks · · Score: 2, Interesting

    This would be pretty cool combined with the VirtuSphere.

  14. Not sure how you'd do it.. by nathan+s · · Score: 5, Insightful

    ...as a director.

    I mean, part of the idea of a "film" is controlling the perspectives and what you present to your viewer. Somehow I have a hard time imagining this going past the art-house type movie, because the amount of work that the director has to do increases dramatically. Since you can't be sure what direction your audience is looking it, it would make it difficult to have a feature film in the sense that we're used to it...for example:

    Jim: Wow, that guy just robbed a bank!
    Sue: What guy? I was looking at those flowers over there.
    [Camera whirls around, both get dizzy and throw up since they can't tell where to focus]

    I'm being silly, but it just seems pretty difficult. That said, it's a cool technology, and if someone could tell me how they plan to deal with that whole focus issue in the context of a feature film, I'd be interested to hear it.

    1. Re:Not sure how you'd do it.. by lpangelrob · · Score: 2, Interesting
      Well... you wouldn't necessarily have to have action going in every angle at once. That would be a little bit ADD. More likely, you'd have more first person experiences. What we have now is the camera depicting what the main character sees; what we'll have later is the camera dpicting where the main character is, and you can see all around. It's still up to the director to focus the audience's attention on where the action is, taking into account that wildly changing focus like handheld cameras do now would be thoroughly irritating.

      That said, about the only good use I can think of for something like this would be scary movies where the tendency of a Really Scary Character to jump into a scene from nowhere, punctuated by the inevitable scream and/or musical blast, is everywhere. Something like that would be much more effective in a 360 degree room.

      Heck, ride the Haunted Mansion ride in Disneyworld / Disneyland sometime for the same effect.

    2. Re:Not sure how you'd do it.. by Vitriol+Angst · · Score: 2, Interesting

      On the TV Show 24, they have multiple images. They control motion and sound to indicate "action". In a circular video, you could easily "direct" attention by increasing activity, sound or some "uniqueness"... it would definitely be more work to pull off. We have been used to a generation of TV and there is a lot of "understanding" between the producer and the audience ... but it isn't necessarily the only way to produce video. When I took classes on the basics, it was interesting to learn the rules of continuity that most of us aren't aware of consciously, but if any of the "rules" are broken, we notice immediately.

      I think 3D "puppets" of interactive movies are the future. A host system would render the perspective and real actors "script" the actions of the puppets. The wrap-around movie may be interesting as a novelty... but it would take too much of the director and audience (like 3D glasses on pseudo 3d today). I think it may have a great use in documentaries to "make people feel" like they are there ... a better witness, where you DON'T want to "direct" the audience. Authenticity or information is a fantastic use for this. Perhaps an undersea IMAX (read "fishbowl"). I would love it, but I doubt you'd want to see an action picture this way. A horror movie would be "too intense" to be enjoyable by anyone sane. Just my two cents.

      --
      >>"ad space available -- low rates!!!"
  15. This isn't new... by fragmentate · · Score: 4, Informative

    There were teams using Amigas doing this years ago. I recall 2 such products: ProVu, and Cosmo. ProVu was used for "interior design." And Cosmo was used by cosmetic surgeons.

  16. Panoramic Stitching? by MatrixCubed · · Score: 2, Informative
  17. You can buy two existing similar systems by Thagg · · Score: 3, Informative

    At Siggraph this year, there were two similar systems on display. They are unbelievably cool.

    1) Point Gray's Ladybug2 has five cameras mounted in a box about the size of, say, a stack of three decks of cards.

    2) Immersive Media's system has 11 (!) cameras in a sphere about 2 inches on a side.

    Both systems do real-time stitching of the multiple images into a panorama.

    We're looking into them for the obvious motion-picture visual effects applications. The resolution (both spatial and dynamic) is not ideal for motion-picture work, but the ability to have an extremely small, lightweight, panoramic capture is a tradeoff that is worthy of pursuit. In the past (say, on The Fast and The Furious) we used six ARRI 435 cameras mounted to the side of a motorcycle, to the tune of several thousand dollars a day rental, hundreds of pounds of weight, and fairly compromised images in other ways (bad lens flare, extremely bouncy images.)

    Thad Beier
    Hammerhead Productions

    --
    I love Mondays. On a Monday, anything is possible.
  18. Crop it in post-production by slim · · Score: 2, Insightful

    My local non-league football team (association football, that is... OK, soccer if you like) has a bloke who films every match using a camcorder, and produces DVDs for harcore fans (of which there can't be that many: 800 is a good home crowd).

    I've not seen the DVDs in question, but it seems to me that producing adequate footage of a football game from a single vantage point, zooming and panning on the live action, must be quite a challenge.

    It occurred to me that with a very high enough resolution CCD, and a very wide angle lens, one could capture the whole game using static cameras, and pan and zoom in post-production. With a few of these (which could run unattended) you could get very good coverage of a live event, deferring all decisions about zoom levels, pans etc. until afterwards.

    What's that you say? Off topic? Er, OK. While a 360 degree capture might not be appropriate for a sporting event (unless you were also interested in capturing the crowd -- since the camera couldn't be in the middle of the pitch), you could use the same technique in other circumstances to capture an event then edit it down afterwards.

  19. Re:Woa! by Orrin+Bloquy · · Score: 3, Funny

    with the dodgy looking pron director and his clapper. So that's what you kids are calling it today.

    --
    "Made up/misattributed quote that makes me look smart. I am on /. and I must look smart."
  20. What about a spherical lens with camera below? by RexRhino · · Score: 2, Interesting

    Back in the 90s dot com boom I saw a prototype camera for doing 360 degree panning quicktime static images that was essentially a camera pointed vertically, with an extreme almost spheroid "fisheye" lens. The image would be processed to change the distorted fisheye image into a panoramic 360 degree view (The only direction you could not pan in was down, because the camera was there. Obvously down is the least interesting direction for panning, although I suppose you could have mounted the camera upside down if the ground was important.). The prototype seemed to work fine.

    Why can you do the same thing with video? Is it because processing a "fisheye" image is just too processing intensive for 30 frame a second HD video? Is the technique patented and so off limits for other companies? Is it that a video image is too low-res to do translations from a distorted fisheye without blurring? Why do it the elaborate way described in the article when the fisheye technique seems a whole lot simpler?

    I saw the prototype in person, so unless the company was commiting outright fraud, I am pretty sure the fisheye thing works.

  21. How would Red Green solve this? by Quiet_Desperation · · Score: 3, Funny

    Or you could, like, duct tape a bunch of cheaper camers into, like, a ball.

  22. Re:I have this technology already by RexRhino · · Score: 2, Insightful

    I have had an xray/picture taken that was exactly the opposite (I forget what it was, it was my orthodontist who had the equipment... This was probably sometime in the late 80s, so at the time it was probably super high-tech)... I sat in one place, and the camera spun around my head.

    So what you are saying, while modded funny, is not too outlandish. You could have a camera spinning around on a motor, and if the frame rate of the video was faster than the motion blur, and it could spin around fast enough to capture enough frames per second, what you are saying would probably work. (Although I would prefer some solid-state technique than to be spinning a camera at 200 rpms).

  23. Look up instead... by MadCow42 · · Score: 2, Interesting

    360 degree views using a 180 degree lens are perfectly possible: you're just not looking at it right.

    If you point the lens directly UP (or down even), then it will cover 360 degrees around you. You could extract a 360 panorama from that fish-eye image and remove the distortion fairly easily (although it would be processing intensive).

    MadCow

    --
    I used to have a sig, but I set it free and it never came back.
  24. Columbia University has a patent on this by postbigbang · · Score: 2, Informative

    They've alread produced 360 degree lenses, and mounts for analog and digital cameras. The math's already been done to change this convex lens structure back to a linear view; it's been around for almost ten years. They have numerous licensees, and so 180 degree lenses seem like a cripple.

    --
    ---- Teach Peace. It's Cheaper Than War.
  25. 4 \pi Steradians by benhocking · · Score: 4, Insightful

    For it to be truly immersive, shouldn't they be bragging about 4 \pi steradians instead of 360 degrees? One's a measure of a spherical surface area and the other only describes a circle!

    Also, why doesn't π or π give me \pi? It seems to work in general HTML... Interestingly enough, & still works (and a handful of others).

    --
    Ben Hocking
    Need a professional organizer?
  26. What's the big deal??` by lcsjk · · Score: 2, Informative

    You take a 180 degree fisheye lens, attach it to a camera, choose a virtual 180 degree line and rotate the assembly around that line. Then stitch the pictures together. You don't have to extrapolate nor do you lose any of the picture and have to "guess" as the article mentions. I am not sure what they are doing that seems "revolutionary", except maybe they have packaged the full unit and are showing it off. Either way, why are they having problems extrapolating to the hidden part behind the camera.

  27. Let me predict... by localman · · Score: 2, Interesting

    feature film presentation using fully-immersive video in the future

    That will suck. First off, I can't really take in more than what's in front of me anyways. Am I going to have to twist around to see the characters and/or action? I remember a 360 movie in Disney world and it was more annoying than neat. Then there's the fact that part of the art of cinema is putting things in a frame -- if there is no longer a choice of what goes in and what doesn't, it's less and less of an art. Then there is the fact that by increasing the cost of filming, set design, distribution and projection, that these films will be even worse than what we are accustomed to.

    I think there's a great untapped use for immersive technology, but feature films are not it.

    Cheers.