Smart Rendering For Virtual Reality

An anonymous reader writes: Researchers from Intel have been working on new methods for improving the rendering speed for modern wide-angle head-mounted displays like the Oculus Rift and Google Cardboard. Their approach makes use of the fact that because of the relatively cheap and lightweight lenses the distortion astigmatism happens: only the center area can be perceived very sharp, while with increasing distance from it, the perception gets more and more blurred. So what happens if you don't spend the same amount of calculations and quality for all pixels? The blog entry gives hints to future rendering architectures and shows performance numbers.

Comments

What if you move your eyes

[Anonymous Coward]

instead of your head?

Re:What if you move your eyes

[Anonymous Coward]

the field of vision on these VR goggles is pretty narrow, you don't want to move your eyes.

Re:What if you move your eyes

[bondsbw]

The experience would be less than ideal.

Future iterations that have better lens quality and higher field of view could make use of eye tracking to determine what location of the viewable area should be rendered optimally.

Re:What if you move your eyes

[hey!]

Eye tracking isn't all that hard to do. I worked in a lab way back in the early 80s that did it with couple of phototransistors and an IR light source to measure corneal reflections.

Re:What if you move your eyes

[Iamthecheese]

The problem is how human eyes move. We dart them quickly and unpredictably about. I think re-rendering as the user moves her eyes will take as much processing power as just rendering it all.

Re:What if you move your eyes

[Anonymous Coward]

The image is being re-rendered at 60fps anyway, so accounting for movement of the eyes will not be a problem in that sense.

Re:What if you move your eyes

[Iamthecheese]

No, it's being redrawn at that rate, an important distinction. Re-rendering implies passing a good deal of new data into video memory.

Re:What if you move your eyes

[Blaskowicz]

Ideally, it is re-rendered at that rate or higher and you have large memory bandwith for the vid card or GPU (anywhere from 20GB/s to 300GB/s)

Re:What if you move your eyes

[ShanghaiBill]

The problem is how human eyes move. We dart them quickly and unpredictably about.

You are wildly underestimating the speed of modern GPUs. They can easily generate a scene in less time than your eye can perceive it, especially if all the polygons are preloaded into graphic memory. All you have to do is run the polygons through the shaders, which are massively parallel, typically 1024 cores.

Re:What if you move your eyes

[vipw]

It's being rendered at around that rate as well.

The VR software includes some ability to shift an already rendered frame because of head tracking, the same approach could probably be used to compensate for eye motion. I'm not sure how much an eye really moves in 1/60th of a second. It also has a micro-stutter that is probably fairly unpredictable. Gross motor movement takes a while to start and stop, so the viewport of the next frame can generally be calculated with reasonable accuracy.

Re:What if you move your eyes

[smaddox]

Which is why the current crop of displays won't last long, if VR really catches on. Magic Leap is already well on the way to developing consumer-level retinal displays. I'm pretty sure Oculus and Apple are working on their own; other companies likely are, as well. There are some significant challenges, particularly with making it economical, but nothing insurmountable. Advances in MEMS and fiber-coupled diode lasers will play a critical role. I expect to see consumer-ready, variable-focus retinal displays in ten years at the latest. The question in my mind, is whether the other peripherals will be able to match the level of immersion provided by the displays. Convincing haptics may end up being more difficult than direct neural interfaces; I hope that's not the case, though, because the latter seems to be quite far off.

Of course then there's the question of how much society will be able to adapt to immersive VR. If the second or third generation consumes all the brightest minds, there will be no one left to develop the subsequent generation.

Re:What if you move your eyes

[The Raven]

Eye tracking with high latency is not hard... now please do it accurately in under 10ms, 90 times a second, so that the rendering pipeline can use that in the next frame.

A lot of 'solved' tasks become a lot harder under the constraints imposed by VR's need for speed.

God's Prior Art, we sim

[Tablizer]

So it's reinventing quantum physics: it's fuzzy until you look at it more carefully. P.S. don't look at cats.

Re:God's Prior Art, we sim

[Jeremi]

So it's reinventing quantum physics: it's fuzzy until you look at it more carefully. P.S. don't look at cats.

He's not kidding. I looked at my cat through VR glasses and saw this.

Ah...where is the foveal tracking?

[Anonymous Coward]

My eyes can still look around, Intel. Carmack mentioned the marginal benefits of this type of rendering optimization years ago. It's of very limited benefit if you are just rendering the center in high-def rather than where the eyes are actually looking.

That's not what astigmatism is - is it?

[wonkey_monkey]

According to the linked Wikipedia page:

An optical system with astigmatism is one where rays that propagate in two perpendicular planes have different focus.

What the article seems to be about, though, is the way images as viewed in a VR headset get blurrier as you move away from the center, seemingly equally in all directions.

Re:That's not what astigmatism is - is it?

[umafuckit]

What the article seems to be about, though, is the way images as viewed in a VR headset get blurrier as you move away from the center, seemingly equally in all directions.

What you're missing is that astigmatism tends to get worse as you move off-axis and astigmatism causes blurring and other similar effects. Thus, more blur at the edges of the field than in the centre. You can correct for this, but it involves a lot heavy and expensive glass.

Re:That's not what astigmatism is - is it?

[wonkey_monkey]

I got that, I just don't see that it fits the definition of astigmatism. It sounds a lot more like spherical aberration.

Re:That's not what astigmatism is - is it?

[umafuckit]

Astigmatism can behave like this. I do a little astronomy and I've seen cheaper wide-field eyepieces introduce astigmatism in the outer portions of the field (even if the objective is known to be good). The centre is good, though. You know it's astigmatism because stars turn into shapes such as crosses. Spherical aberation in the objective, on the other hand, decreases contrast throughout the field of view.

Give up!

[oldmac31310]

Apple will soon put a stop to their innovation! Hurray for patents!

Aspheric lenses !

[redelm]

If there is a optical problem, why not solve it rather than trying to get software to (maybe) compensate and eat up battery life?

Just design aspheric lenses specifically for the headsets. Yes, the injection molds might be a bit more complex, but I believe these have to be CNC anyways. A bit more setup, but no production cost increase.

Re:Aspheric lenses !

[Anonymous Coward]

It sounds like they're doing the opposite, not spending effort rendering the out-of-focus areas. I still agree with your solution, don't try to save CPU cycles but fix the problem.

Foveated rendering

[flarb936]

Regardless of lens quality, "foveated rendering" is useful for reducing rendering complexity of stuff in your peripheral vision you don't really see. But I think VR/AR Is hitting the wall with how good you can make LCD screens smashed up against your eyes look. Virtual Retina Displays seem MUCH sharper to me. Tried an Avegant Glyph awhile back and the detail was pretty great. It's super frustrating in Oculus to not be able to read text unless you move your head so that it's positioned in the center of the lens.

John Carmack already solved this.

[Anonymous Coward]

Isn't smart rendering of resolution density based on focal point proximity already technology that John Carmack said Oculus is developing as discussed at the 2014 Oculus Keynote when they debutted the VR Gear?

Solutionherbal.com || vimax,obat kuat sex

[nikoasean]

obat kuat obat kuat sex obat kuat pria obat pembesar penis pembesar penis alat pembesar penis pembesar payudara vacum pembesar payudara obat perangsang obat perangsang cair perangsang wanita perangsang permen karet