Preview Jaunt's Made-for-VR 360 Degree, 3D Short Films

An anonymous reader writes Jaunt, a company that's raised more than $34 million to create a platform for live-action cinematic virtual reality experiences, has set out to demonstrate their toolset by producing three made-for-VR short films that are shot in 360 degrees and in 3D. Road to VR has an exclusive preview of the films, which the company says will have interactive trailers released very soon for Oculus Rift and Android (for use with Cardboard and other smartphone VR adapters).

Comments

What is it, an ad?

[Begemot]

If it wasn't, it could have mentioned also the Samsung Beyond camera project or the Panocam 3D or 360heros.

Oh well...

Re:What is it, an ad?

[Animats]

Technically, this is quite do-able. Then again, consider what a dud 3D TV was.

Headgear for a game, like an FPS shooter, should be fun. But for passive watching, it's too much work.

Re:What is it, an ad?

[kesuki]

3d tv was a dud because tv sets were in a bubble. it takes at least 10 years for someone to want to replace a perfectly fine tv set. 3d films are expensive are large enough that many of the 3d bluray are unable to use a single disc. flash memory is up to 128 GB in sd card format which makes it about as capable as a quad layer blu ray. 360 degree degree 3d films are huge on resources to create and store, though gpus can render worlds in realtime saving the need to store them, but rather to store the bookmarks for lack of a better word that allow hardware to recreate the computer generated vision. . lugging around a 6tb hdd to fit a few movies when lossy 1080p can use around 2.5 gb per layer and thus fit 400 films per terrabyte or 200 3d films, or perhaps fit 33 movies with annoying fisheye lenses that maybe can be software corrected... keep in mind i assumed fisheye and heavily lossy, and i didn't assume 4k or 8k resolutions. so really you can fit 1 movie on a 1 tb hdd. keep in mind the tablet and smartphone movement has made powerful computers something only corporations are allowed to own, there won't be general purpouse computers capable of handling all the data in all the right places to make this work. we need sustainable tech, using cardboard and a smartphone to make a 3d cinema is neat, and perhaps a step in the right direction, towards sustainability, but some of us want to be able to customize the interface however we like it, and if i can have a 8 channel tuner and 8 virtual 60 inch tv sets, with selectable audio like the football channel on direct tv, then a 3d headset capable of that is far more sustainable than buying 8 tv sets or buying two 4k 120" sets side by side...

360 3D

[Thanshin]

"shot in 360 degrees and in 3D"

Intuitively, this is impossible. Let's RTFA... Nope, I can't find that claim.

Ok, time to use the "Ask the Audience" lifeline: How does one shoot 3D in 360 without it needing an infinite amount of film?

Re:360 3D

[Thanshin]

If I get what you mean it's something like, for example, put 8 360 cameras in a cube with a vertex length equal to distance between eyes (although 4 cameras in a tetrahedron pattern should work). This way, you always have a 3D view on all directions, by choosing the right pair of cameras.

The problem is that, to avoid viewing the other cameras the system would have to switch points of view for each eye as the person looks right and left, for example.

Re:360 3D

[arnero]

I did not RTFA, but if you use a ball with lots of cameras for all directions and a lot of overlap you know your surrounding completely. Dunno how that "infinite" comes into your mind, I just see 12 films / streams and a modern GPU at work.

Re:360 3D

[rosshalz]

partly yes.. though it may not be necessary for the system to "switch" between cameras since this isnt live (direct from the camera in real-time) 360degree 3d we're talking about... you'd need one video stream for the sphere of view of the left eye.. and another stream for the sphere of view of the right eye they'd have to be constructed and stiched together like panorama or photosphere pictures beforehand (unless you have some really powerful computers doing the conversion in real time.. though not necessary for a movie) So it would boil down to the system (occulus..gearVR etc) doing the calculation of which part of the sphere to be shown based on the orientation of the head of the viewer.. and do this fast enough that he/she cannot notice any lag between head motion and video motion...

Re:360 3D

[taylorius]

Assume they have numerous cameras on the surface of a ball, with significantly overlapping fields of view. The reconstruction phase would be where the difficulty lies - normal image stitching wouldn't work, because it assumes one single optical centre for all shots, and treats deviations from this as an error to be smeared away. However in this case you need to use the varying optical centres of the cameras, to gain parallax / depth information. So it becomes a photogrammetry problem, recovering 3d points - with points beyond a certain distance mapped to a distant sphere. Then on playback the data could be reprojected correctly.... somehow. *waves hands* It's definitely not simple to do correctly.

Re:360 3D

[Spy Handler]

Ok, time to use the "Ask the Audience" lifeline: How does one shoot 3D in 360 without it needing an infinite amount of film?

It's not 3D in 360. It's stereoscopy in 360.

You know Google street view where you can rotate around 360 degrees? Well it's like that but shot with 2 cameras for 3D effect.

Re:360 3D

[Anonymous Coward]

By using some lightfield math or holography. I recommend the Disney Research publications on Megastereo. However, I've never seen it implemented along two axis, which means it doesn't properly work with tilt or yaw.

Re:360 3D

[grumbel5969]

It's not quite that simple. 360 with a single camera can be trivially stitched together because you can rotate the camera around a single point. With two cameras you are no longer dealing with just rotation, when you rotate a pair of cameras the individual cameras get translated as well, since they are offset from the point of rotation. If you try to stitch the images together you will get very noticable seams, especially when objects are close to the camera. There are ways around that with lots of cameras and more intelligent stitching algorithms, but so far it's still an open area of research on how to do it best. And even if you successfully stitch things together, you have still the problem that a simple 3D video can't deal with head tilt or head translation, so maybe some kind of depthmap/voxel format is needed to make 3D video really pleasent and less of a hack.

Re:360 3D

[zAPPzAPP]

If you have a camera which can record 360, then there is no need for a second camera to get 3D, provided your camera's point of view moves around in a circle about the size of a human head.
While rotating a head 360 (do not try at home), there is no point of view which is exclusive to one eye. At some point your left eye will see the same thing your right eye saw a couple degrees of rotation earlier (or later).
So you should have all the visual information you need from a single 360 recording.

Re:360 3D

[rosshalz]

True.. I suppose we'd have to add restrictions for movies where you want the action is happening at a set minimum distance from the observer (camera) and anything that gets closer will have to .... well.. disappear... yeah i see what you mean by hacky.. I do think it might be sufficient for movies/stories ....

Re:360 3D

[rosshalz]

hey that's right! so it would depend on just using the right pixels for the correct eye depending on the direction of the observers head.. recording-wise you'd just need one 360degree camera... right?

Re:360 3D

[Mr D from 63]

360 - 3D Animated would be a lot easier than real life filming. It would me interesting to see an evolution to movies that are animation models rather than video, more like a video game. Then it would not only be a 3D view from a central point, but you could watch the movie from any aspect.

Re:360 3D

[Rei]

Specifically two cameras? Why? If you're picturing simple stereoscopy, that doesn't work if the viewer can turn their head.

If we're talking turning the head on just one axis, I picture something along the lines of a single camera sweeping in a horizontal circle at 60rpm (or two cameras at 30rpm, 3 at 20 rpm, etc, or any equivalent non-rotating setup using many sensors). The rotation (or virtual rotation) would be around an offset axis, where the center of rotation is the center of rotation of a human neck and the distance to the focal point is the distance to the focal point of a human eye. This would of course be a tremendous data stream, but you wouldn't actually be saving all of that. Instead you'd be building up a z-buffer based on the parallax or another ranging method and using that to create and texture simple discontinous 3d surfaces around the user.

Displaying the resulant movie would be not quite as simple as mere billboarding, but a lot easier than full 3d (no lighting / shadowing calculations, very simple obstruction culling, etc). That the surfaces would be discontunous isn't a problem - unless the goal would be to let people walk around freely, you don't need to know what's behind the gaps that the camera can't see. You could potentially even use data from earlier frames to remove the cameraman - the cameraman would just need to be sure to stand in areas where nothing has changed (even shadows / reflections / indirect lighting / etc, to the extent that's possible).

If you want full 2-axis VR, not just one axis, I don't think actual physically rotating cameras would work, I'd expect you'd have no option but to use many sensors on the outside of a sphere roughly the size of a human head.

Re:360 3D

[Rei]

You could, if it's rotating around you at 30 or 60 or however many frames per second you want your film in. Of course, you can't just save the raw data for that. If you wanted literal direct camera data for all points and if we say a person can perceive a rotation of 0,1 degrees (it's probably a lot less than that) then a 30fps movie filmed by a rotating camera would actually have to film at 108k FPS (and a 60FPS movie at 216k FPS) to capture the view at each perceiptible point along its travel. Clearly that's an unreasonable amount of data. But as per above, you could use the stability and precisely known positions of the camera between frames to assist in quick mutual-point registration to get a measure of parallax and thus z-depth across your footage as the camera rotates, and thus could texture a series of discontinous simple surfaces around the user. The required texture data wouldn't be that much larger than simple stitched 3d stills of the same quality, and the 3d geometry resolution could probably be half an order of magnitude less than the texture resolution and still look right. And because you no longer need to film at each point the user can perceive, just enough to build up your textured geometry, you could probably film at more like a couple hundred frames per second. Or use a dozen or so stationary sensors cameras positioned around a head-sized sphere.

Re:360 3D

[Anonymous Coward]

It's not true 3d from all angles; however, the stitching technique is actually very convincing. See Carmack's samsung Keynote for full explanation :)

Re:360 3D

[Ksevio]

You know Google street view where you can rotate around 360 degrees? Well it's like that but shot with 2 cameras for 3D effect.

Or to be more exact, it's like Google street view when you right click and enable 3D mode

360 3D

[rosshalz]

This should be simple right? use 2 sets of cameras to record from 2 points of view lightly offset from each other to give the viewer some parallax and hence the illusion of 3D.... I mean 1 set(8..10..9000 how many ever necessary) of cameras for the left eye video, and another set for the right eye video...

So how is this different fro QuickTime VR?

[real gumby]

Seems the same as a product released by apple in 1994 -- 20 years ago.

http://en.m.wikipedia.org/wiki...

Re:So how is this different fro QuickTime VR?

[Begemot]

It's the same 360 but with the added 3D effect (stereo actually).

So how is the XM25 different from a hand cannon?

[dj245]

This XM25 is just a derivative. Seems like the same as a product released by the Chinese in 1271 -- more than 700 years ago.

outbound bogor

[Anonymous Coward]

thanks ask your information
  by tempat outbound

It's not true 3D and it doesn't need to be.

[PJ6]

This is not an ad. Early VR projects are interesting, and this post (even though it could be used for publicity) belongs here.

Yes, you can technically make 360 degree 3D. You can even make it work even when the viewer tilts their head, if you have a ball of cameras and the right software to correctly adjust the view for each eye in real time according to head position. Jaunt's camera is not a ball. It's a disk. Yes, it could still be accurate 3D as long as you don't tilt your head. They say it's 3D, but it's not very high quality or convincing. In fact I wasn't able to tell if it was really 3D or just seeing the same view through both my eyes.

That being said, they do not need to have it in 3D at all. VR is already *very* convincing without 3D and the effect of parallax disappears at 20 feet or so. I didn't feel it was required at all for the demo videos they showed me.

I built my own 2x4K VR camera, specifically designed for accurate 3D, and demoed alongside Jaunt in Boston. It's got 90x170 degrees FOV, which is more than enough to cover the DK2 horizontal FOV, and almost enough to cover its 100 degree vertical FOV. For one of my demo videos, I put peanut butter under the camera and recorded my dog licking it. Everyone responded the same, putting their arms up in front of them to wave him away. There were some squeals of delight. That's the point of 3D VR, in my opinion - accuracy and proximity, to make you really feel like something is there in front of you. Otherwise you might as well just go 360 and not sweat parallax accuracy, like Jaunt did.

I was getting ready to sell my cameras, make movies, and work with other people improving the rig design, but honestly I thought there'd be more talent and interest to work with. Boston really isn't anywhere close to being a Silicon Valley of the East. And I say that being an MIT graduate myself.

Re:It's not true 3D and it doesn't need to be.

[Begemot]

This is not an ad. Early VR projects are interesting, and this post (even though it could be used for publicity) belongs here.

That's exactly why it is an ad. They should have mentioned other projects in this area and maybe compare them. Then it would be a lot more interesting.