Technology for Capturing 360 Degree Video

Inzite writes "EnterNetica R&D is working on a new spherical video technology for capturing and presenting full 360 degree scenes using a 180 degree lens, by adaptively predicting the camera's surroundings. Video extrapolation techniques have been proposed in the past, but this is the first time I've heard of an entire hemisphere of the video image being "guessed". The article also talks about feature film presentation using fully-immersive video in the future."

Comments

Why not two cameras?

[AKAImBatman]

Maybe I'm missing something, but the article doesn't explain why they don't simply place two cameras back to back, then use their software to splice the result and apply the correct perspective. All they say about it is that "it's against the laws of physics to take a 360 degree photo. That just seems odd.

Re:Why not two cameras?

[eMartin]

"This lens does not offer you the possibility to record a full 360 image... There is no data about what can be seen below and above the viewer. What they do is creating a spherical image, not just a doughnut (which is what your "funky lens" is capable of)."

The only requirement for "360 degrees" is that it see from a certain point in all directions on a single plane. Adding up and down means means adding another dimention to the process. Something that can cature a full spherical view would have to be called 360x180 degrees, where you capture 180 degrees of vertical for every one of the 360 degrees of horizontal.

Or something.

Re:Why not two cameras?

[Traa]

my guess

Let's see, TFA mentions that they did not in fact record a full 360 image, instead they record a 180 degree fisheye image and then "fill in the rest" based on smart guessing (no details provided). They happily adimit that the algorithm isn't perfect and that some environments work better then others. Without providing samples for us to see they allready admit that they had to go out of their way to come up with some samples that looked decent. I like the skydiving example they talk about, the algorithm is capable of filling in the empty sky behind you. Amazing!

As for wether this will work in a regular scene (where stuff actually happends behind the camera), I'll just make a bold statement: It doesn't.

Much better would have been if they used the camera as mentioned by grandparent, recording a 360 degree donut view, where at least most of the interesting information is captured and only the roof and floor need to be made up by the software (now that seems reasonable).

I'll leave the stuff about pot and kettle for yourself to figure out (minor appology to gp would be nice, but I'm not counting on it).

Boo

[Tomchu]

The site's already down, but based on the small blurb ... I don't see how this is possible. You're basically guessing at 50% of the scene, of which the only hints you have are land/sky textures. If that's all this is about, I've been doing it for years in Photoshop. It's called the Clone/Healing Brush tool.

Re:Boo

[AKAImBatman]

They explained the "how" in the article. Basically, the panning of the lense attempts to capture as much of the scene as possible. The final video is then put through video analysis software that uses motion prediction to stitch the future and past of everything visible based on the few clips it acquired.

To put this in perspective, if you're panning through an aquarium, you'll see the fish swimming on as if nothing was wrong. However, while the camera isn't looking, a fish may get scared and dart off. Since that event wasn't captured, the software will show you the fish contining to swim in the final video.

Unfortunately, things might get a little weird with more dynamic scenes. Consider taking video of a theme park like Six Flags Great America. Let's say during panning you capture a roller coaster as it's decending, but fail to capture anything after that. I don't know how complex their software is, but there's a high probability that it would show the coaster going right through the track and into the ground. Something like that would quickly spoil the illusion that the camera is truly 360 degree.

Make sense? :-)

Why guess?

[sakusha]

The patent on Disney's Circlevision camera system has expired.

Not sure how you'd do it..

[nathan s]

...as a director.

I mean, part of the idea of a "film" is controlling the perspectives and what you present to your viewer. Somehow I have a hard time imagining this going past the art-house type movie, because the amount of work that the director has to do increases dramatically. Since you can't be sure what direction your audience is looking it, it would make it difficult to have a feature film in the sense that we're used to it...for example:

Jim: Wow, that guy just robbed a bank!
Sue: What guy? I was looking at those flowers over there.
[Camera whirls around, both get dizzy and throw up since they can't tell where to focus]

I'm being silly, but it just seems pretty difficult. That said, it's a cool technology, and if someone could tell me how they plan to deal with that whole focus issue in the context of a feature film, I'd be interested to hear it.

Crop it in post-production

[slim]

My local non-league football team (association football, that is... OK, soccer if you like) has a bloke who films every match using a camcorder, and produces DVDs for harcore fans (of which there can't be that many: 800 is a good home crowd).

I've not seen the DVDs in question, but it seems to me that producing adequate footage of a football game from a single vantage point, zooming and panning on the live action, must be quite a challenge.

It occurred to me that with a very high enough resolution CCD, and a very wide angle lens, one could capture the whole game using static cameras, and pan and zoom in post-production. With a few of these (which could run unattended) you could get very good coverage of a live event, deferring all decisions about zoom levels, pans etc. until afterwards.

What's that you say? Off topic? Er, OK. While a 360 degree capture might not be appropriate for a sporting event (unless you were also interested in capturing the crowd -- since the camera couldn't be in the middle of the pitch), you could use the same technique in other circumstances to capture an event then edit it down afterwards.

Re:I have this technology already

[RexRhino]

I have had an xray/picture taken that was exactly the opposite (I forget what it was, it was my orthodontist who had the equipment... This was probably sometime in the late 80s, so at the time it was probably super high-tech)... I sat in one place, and the camera spun around my head.

So what you are saying, while modded funny, is not too outlandish. You could have a camera spinning around on a motor, and if the frame rate of the video was faster than the motion blur, and it could spin around fast enough to capture enough frames per second, what you are saying would probably work. (Although I would prefer some solid-state technique than to be spinning a camera at 200 rpms).

4 \pi Steradians

[benhocking]

For it to be truly immersive, shouldn't they be bragging about 4 \pi steradians instead of 360 degrees? One's a measure of a spherical surface area and the other only describes a circle!

Also, why doesn't π or π give me \pi? It seems to work in general HTML... Interestingly enough, & still works (and a handful of others).