Slashdot Mirror


Ask Slashdot: Why Don't Graphics Cards For VR Use Real-Time Motion Compensation?

dryriver writes: Graphics cards manufacturers like Nvidia and AMD have gone to great pains recently to point out that in order to experience virtual reality with a VR headset properly, you need a GPU capable of pushing at least a steady 90 FPS per eye, or a total of at least 180 FPS for both eyes, and at high resolutions to boot. This of course requires the purchase of the latest, greatest high-end GPUs made by these manufacturers, alongside the money you are already plonking down for your new VR headset, and a good, fast gaming-class PC. This raises an interesting question: virtually every LCD/LED TV manufactured in the last 5 or 6 years has a 'Real-Time Motion Compensation' feature built in. This is the not-so-new-at-all technique of taking, say, a football match broadcast live at 30 FPS or Hz, and algorithmically generating extra in-between frames in real time, thus giving you a hyper-smooth 200-400 FPS/Hz image on the TV set with no visible stutter or strobing whatsoever. This technology is not new. It is cheap enough to include in virtually every TV set at every price level (thus the hardware that performs the real-time motion compensating cannot cost more than a few dollars total). And the technique should, in theory, work just fine with the output of a GPU trying to drive a VR headset. Now suppose you have an entry level or mid-range GPU capable of pushing only 40-60 FPS in a VR application (or a measly 20-30 FPS per eye, making for a truly terrible VR experience). You could, in theory, add some cheap motion compensation circuitry to that GPU and get 100-200 FPS or more per eye. Heck, you might even be able to program a few GPU cores to run the motion compensation as a real-time GPU shader as the rest of the GPU is rendering a game or VR experience.

So my question: Why don't GPUs for VR use real-time motion compensation techniques to increase the FPS pushed into the VR headset? Would this not make far more financial sense for the average VR user than having to buy a monstrously powerful GPU to experience VR at all?

159 comments

  1. Sickening by bobbutts · · Score: 4, Interesting

    I find the effect sickening on a flat TV. I'd gather it's worse in goggles.

    1. Re:Sickening by billybob2001 · · Score: 1, Insightful

      I agree, it's almost motion-sickness inducing.

      Plus it looks very unrealistic; quite eerie.

      Like interpolating pixels in the uncanny valley.

    2. Re:Sickening by Travis+Mansbridge · · Score: 1

      This. Extrapolating movement and cranking up the framerate in certain parts of the scene while other parts run at lower framerates or move out of sync is a sensation that constantly reminds me I'm watching a video and breaks the experience altogether. Being immersed in a world where this is occurring would be maddening.

    3. Re: Sickening by Anonymous Coward · · Score: 0

      Shut up, idiot.

    4. Re:Sickening by Anonymous Coward · · Score: 2, Interesting

      There are, however, things that make VR implementations easier and less error prone.

      If the VR helmet itself, which has motion sensors, detects motion, it could apply motion compensation while it awaits for a new frame.

      The approach could be limited to head movements, be truly helpful and nigh flawless if done correctly.

      So really, the poster does make a valid point.

    5. Re: Sickening by Anonymous Coward · · Score: 0

      Shut up, idiot.

    6. Re:Sickening by Immerman · · Score: 4, Informative

      I believe something related is already being done on the Oculus and probably the Vive as well - in that each frame, after being rendered, is geometrically manipulated to approximate the changes that should be seen due to head motion since rendering began.

      You can't use the same technique as used for televison, interpolating between frames, because that introduces far more lag, which I believe is the single largest contributor to nausea. After all you can't interpolate between frames until the second frame is ready, so even if interpolation were instantaneous you couldn't render frame N+0.5 until frame N+1 was finished, so you'd be inserting a minimum extra half-frame worth of additional lag in exchange for doubling the higher frame rate. Pretty sure that would be a lousy deal, especially considering the ugly artifacts such interpolation introduces.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    7. Re:Sickening by Anonymous Coward · · Score: 0

      Agreed, I get the same effect. I asked my brother "You WANT it like that?!" the first time I saw his TV setup with it.

    8. Re:Sickening by Anonymous Coward · · Score: 0

      So you have a 1-frame lag, big deal.

    9. Re:Sickening by Martin+Blank · · Score: 4, Interesting

      It reminds me of the Hobbit movies, in particular of the battle on the river. I was taken out of the movie by the splashes. They looked fake, but I knew that this was more because the movie was shot at 48FPS and so captured the motion better.

      So does it look fake because it is fake, or does it look fake because it's different from what we expect to see?

      --
      You can never go home again... but I guess you can shop there.
    10. Re: Sickening by D.McG. · · Score: 2

      Just that much lag is enough to induce motion sickness; which is why 90fps is preferable. The latency, measured from head movement to photons, needs to be under 20ms to keep one for vomitting.

    11. Re:Sickening by ChrisMaple · · Score: 1

      Extrapolate. It's at least worth a try, to determine if the resulting errors are distracting.

      --
      Contribute to civilization: ari.aynrand.org/donate
    12. Re:Sickening by Immerman · · Score: 1

      I suspect they do so. I would assume the render cycle is:
      1) Extrapolate where your head will probably be by the time the frame is done rendering
      2) Render the frame for that position
      3) Do some fast image distortion on the resulting image to better reflect what you should be seeing from where your head actually is
      4) Display final image
      Repeat.

      Bottom line is there are a *lot* of tricks being used to minimize lag-related discrepancies, it's worth looking at what's actually being done, if only to see all the tricks you didn't even think of.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    13. Re:Sickening by Immerman · · Score: 1

      Yes, a very big deal. Lag is THE main cause of nausea in VR, and every trick in the book is being used to reduce it. Add 1 frame of lag and your 90fps headset suddenly has as much lag as it would if running at only 45fps, and proceeds to make pretty much everyone sick.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    14. Re:Sickening by Anonymous Coward · · Score: 0

      Same. Every time that effect kicks in on my parents 50" screen I feel my eyes do the same thing that happens before motion sickness starts. The only time this effect is okay is on a slow panning shot (eg driving a car at 10mph.)

      On a GPU for VR this would probably have the opposite effect and amplify the motion sickness because it's not actually new frames, but "fake" inbetween frames.

    15. Re:Sickening by Trogre · · Score: 1

      That's just because you're used to watching movies at a horrible choppy 24 fps.

      Look up the Soap Opera effect; it's a real thing. And you'll be very happy once you've gotten over it. To me 24 fps movies look like poorly done stop-motion.

      --
      "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
    16. Re:Sickening by Anonymous Coward · · Score: 0

      My guess: On live sports this isn't a deal because the whole event can be time shifted back by =1 second by the TV, giving time to render the intermediate frames.

      VR shows a different image based on headset orientation which is controlled by the user. Since the computer can't know when the user is going to change the viewing perspective it can't do a time shift with introducing serious lag / stutter in the viewed image.

    17. Re:Sickening by Anonymous Coward · · Score: 0

      > Add 1 frame of lag and your 90fps headset suddenly has as much lag as it would if running at only 45fps

      Huh? 90 - 1 = 89 or am I missing something?

    18. Re: Sickening by John+Allsup · · Score: 1

      Time from movement to screen with no lag at 90hz is 1/90s. With 1 frame lag, it takes double that, so 2/90=1/45s, hence 1 frame lag means latency equivalent to half the framerate.

      --
      John_Chalisque
    19. Re:Sickening by Anonymous+Cow+Ward · · Score: 1

      I think it's the latter; as I recall, home video cameras (the ones that wrote to tape) went up to ~60 fps. That was faster than TV, and so when people watched it, it looked different and wrong to them. This actually discouraged movie studios from shooting at higher framerates because it reminded test audiences of their home videos, and thus they thought it looked lower quality, even though it was actually better.

      --
      Examine even your most deeply held beliefs. Nobody is always right.
    20. Re: Sickening by Anonymous Coward · · Score: 0

      This is close to what I was going to say. Receiving a stream of frame data and interpolating is different than generating frame data on the fly at a high frame rate.

    21. Re:Sickening by Anonymous Coward · · Score: 0

      Agreed. This effect actually makes me feel faintly sick on TVs that have it turned on (AND, it's one of the primary causes of the Soap Opera Effect, which, even if it doesn't make you sick, looks like shit). I always have to find it and turn it off on the sets I use.

      Regardless, the problem with VR and frame-rates isn't as much the frame-rate as the latency. To avoid tearing the sets must use VSync. Therefore input latency is tied directly to the frame-rate. Lower frame-rates mean delay between your action and reaction of the image on the displays. Even a tiny delay apparently makes people sick.

    22. Re:Sickening by Anonymous Coward · · Score: 0

      Pretty much any time I read a comment with someone asking "Why doesn't VR do something this way?" the answer is that it would be disorienting. I'm amazed at how many people who haven't used a VR headset insist that apps should using traditional joystick-style controls for first-person movement in VR and other things that seem ridiculous to anyone who's used one.

    23. Re:Sickening by almitydave · · Score: 1

      Agreed. This effect actually makes me feel faintly sick on TVs that have it turned on (AND, it's one of the primary causes of the Soap Opera Effect, which, even if it doesn't make you sick, looks like shit). I always have to find it and turn it off on the sets I use.

      Well, I grew up in the CRT era, where some programs (including live programming such as soap operas and evening news, and programs recorded on video tape) ran at the broadcast television standard field rate of ~60fps. Although the broadcast image was interpolated, so they only sent half a frame at a time, the updates (and apparent fps) were 60Hz. The first LCD HDTV we got (Vizio in 2008) had this interpolation feature, and although the smooth effect it created didn't bother me (it just looked like the old broadcast standard), the problem was that the TV couldn't keep up 100% of the time, and would occasionally drop frames, running at 15fps for a second. So the sporadic oscillation from 60 to 15 fps was so jarring I couldn't stand it. I'm one of those people that had to set their CRT monitors to 75Hz minimum or the flicker drove me nuts, so I might be more sensitive to this sort of thing. The wife didn't seem to notice.

      --
      my, your, his/her/its, our, your, their
      I'm, you're, he's/she's/it's, we're, you're, they're
    24. Re: Sickening by Anonymous Coward · · Score: 0

      No, lag should not affect the frame rate at all. The graphics card keeps pumping out frames at the same rate, and the display interpolates between them.

      So the display needs 2 frames to start interpolating, but after that it gets new frames at the same rate as before, just 1 frame behind.

    25. Re:Sickening by Anonymous Coward · · Score: 0

      went up to ~60 fps. That was faster than TV,

      TV has been 60 fps since the early 20th century.

    26. Re:Sickening by Anonymous+Cow+Ward · · Score: 1
      --
      Examine even your most deeply held beliefs. Nobody is always right.
    27. Re:Sickening by Anonymous Coward · · Score: 0

      Every time someone asked me "what do you think of my new expensive TV?" and it had this feature... I just had to stay silent.

      **enter flashback sequence**

      When that feature debuted, a lot of high end LCDs used it. Around that time, Pioneer and Panasonic plasmas were still the creme de la creme for image quality, black levels, motion processing, and latency, but the market was clearly choosing LCD en masse (and years later of course, many manufactures dropped plasma). Knowing I'd likely be stuck with LCDs in the foreseeable future was a bit of a pain, but seeing manufacturers trumpet garbage features (like that nasty interpolated motion) and watching consumer celebrate it... really made me worried.

      Went out an bought a Sony GDM-FW900 CRT and last-gen Pioneer plasma.

      Happy to see that LCD is pretty competent these days. They've certainly come a looooong way.

    28. Re:Sickening by null+etc. · · Score: 1

      I believe something related is already being done on the Oculus and probably the Vive as well - in that each frame, after being rendered, is geometrically manipulated to approximate the changes that should be seen due to head motion since rendering began.

      You're referring to Oculus Asynchronous Timewarp, which maps the viewport into a tessellated grid whose polygons get deformed in accordance with motion data.

    29. Re:Sickening by wonkey_monkey · · Score: 1

      They also do a warp if (on hopefully rare occasions) the GPU can't get the new frame ready in time, repeating the previous frame but warped to match the new head position.

      --
      systemd is Roko's Basilisk.
    30. Re:Sickening by wonkey_monkey · · Score: 1

      Although the broadcast image was interpolated

      Interlaced.

      --
      systemd is Roko's Basilisk.
    31. Re:Sickening by Immerman · · Score: 1

      I suspect it may even be the same technique - normally you warp to make up for lag (1 frame worth of motion), occasionally you warp even more to make up for a dropped frame (2 frames worth of motion).

      Hmm, I suppose if your warping algorithm were good enough you could theoretically double displayed frame rates by making a policy of only rendering every other frame, with warped intermediate frames. I suspect that discrepancies would become obvious if present constantly, but I could be wrong. Especially if your rendering frame rate were already adequate on it's own, you might actually be able to double from 90 fps rendered to 180 fps displayed. Could be well worth attempting...

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    32. Re: Sickening by Immerman · · Score: 1

      You're missing the point, we're only talking about the amount of lag present
      At 90 fps, under ideal circumstances, lag =1/90 s
      At 45 fps, under ideal circumstances, lag = 1/45 s

      At 90fps, with one frame extra lag, total lag = ideal lag + extra lag = 1/90 s + 1/90 s= 2/90 s = 1/45 s

      So, even though the frame rate is indeed still 90fps, the amount of lag is the same as you'd get if you were only running at 45fps.

      If frame rate itself were a major issue for nausea, then motion compensation would make sense. But it's lag that's the major issue, and the push for higher frame rates is primarily in order to reduce that lag. So a technology designed to increase frame rates at the expense of also increasing lag is counterproductive.

      --
      --- Most topics have many sides worth arguing, allow me to take one opposite you.
    33. Re:Sickening by Anonymous Coward · · Score: 0

      To me, the only thing motion smoothing is good for is when the camera pans across a scene and it removes the "judder" you get with 24fps.

      The rest of the time, it's really annoying - on maximum, you see people's faces shifting as they move their heads, like their nose is a floating blob of jelly in front of their face. Yuck.

    34. Re:Sickening by dj245 · · Score: 1

      It reminds me of the Hobbit movies, in particular of the battle on the river. I was taken out of the movie by the splashes. They looked fake, but I knew that this was more because the movie was shot at 48FPS and so captured the motion better.

      So does it look fake because it is fake, or does it look fake because it's different from what we expect to see?

      Hobbit looked bad because the CGI was bad. The piles of gold in the Smaug scenes were especially bad, and the other CGI wasn't very good.

      --
      Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
    35. Re:Sickening by Anonymous Coward · · Score: 0

      > TV has been 60 fps since the early 20th century.

      If you take fps to mean "fields per second," then yes.

    36. Re:Sickening by Martin+Blank · · Score: 1

      I'm only talking about the water on the river. That was all real, filmed at 48fps. It looked bad because it was unexpectedly too detailed.

      --
      You can never go home again... but I guess you can shop there.
  2. That doesn't work because... by Anonymous Coward · · Score: 5, Insightful

    You can't change the angle at which the scene is rendered by interpolating between frames.

    It's not the raw framerate. It's that the scene your viewing has to match where you're looking that quickly or you get motion sick.

    1. Re:That doesn't work because... by Thagg · · Score: 1

      You can't change the angle at which the scene is rendered by interpolating between frames.

      It's not the raw framerate. It's that the scene your viewing has to match where you're looking that quickly or you get motion sick.

      While the parent is Anonymous coward, please rate him up, as that is correct.

      --
      I love Mondays. On a Monday, anything is possible.
    2. Re: That doesn't work because... by Anonymous Coward · · Score: 0

      You can make small angular alterations if the frame is rendered as a fisheye during the final de-warping to rectilinear, allowing the relatively simple de-warp to run at a much higher framerate than the detail scene render.

    3. Re:That doesn't work because... by Anonymous Coward · · Score: 0

      You actually can do that. As long as the angle is small. The small angle approximation allows you to get away with a straight linear interpolation for a short time.

  3. Because frame rates vary by Anonymous Coward · · Score: 0, Insightful

    You can rely on a 24 FPS steady signal when outputting to a television and interpolating the incoming frames is safe and predictable. When your FPS is dynamic due to varying quantities of draw calls, the interpolation becomes far less predictable, and will result in artifacting.

    Also, VR headsets render the whole screen, so its not a magical '180 FPS' number when multiplied by two eyes. Two render channels each rendering at 90 FPS doesn't mean the GPU is taxed at '180 FPS', the GPU load depends on pixel density, and high end VR headsets are extremely pixel dense.

    Feels like the submitter is a child of some sort...

    1. Re: Because frame rates vary by Anonymous Coward · · Score: 0

      Safe, predictable, maybe. It always looks 'wrong' to me on the flat screen, in a way that will trigger motion sickness in many users and lots of immersion in many more. Smoother, flicker free motion is not the same as believable motion.

    2. Re:Because frame rates vary by jon3k · · Score: 1

      And even more important once we see single pass widely implemented it will truly be 90 fps with no confusion.

  4. They do. by Anonymous Coward · · Score: 3, Informative

    Look up "asynchronous time-warp". /thread

    1. Re:They do. by Lord+Crc · · Score: 4, Informative

      Look up "asynchronous time-warp". /thread

      Pretty much this. Here's a video explaining time warping, it also has some links for more details in the video description.

      https://www.youtube.com/watch?v=WvtEXMlQQtI/

    2. Re:They do. by Anonymous Coward · · Score: 1

      This is basically the answer. The important thing to note is that async timewarp has to use extremely low latency (like 120 FPS or faster) head tracking sensors in order to warp the image to match the motion of your head.

      As many people have already commented, the latency will make you motionsick, Specifically, the time delay between moving your head and what you see reflecting that movement. You need to have the raw *refresh* rate fast enough to fix this problem, and they combat the lower *render* rate by adjusting the image via async timewarp (and other methods) with fast motion sensor data and the fast refresh rate.

    3. Re:They do. by allcoolnameswheretak · · Score: 2

      The HTC Vive uses a different technology, I don't remember what it was called, but it seems to be more effective at preventing nausea, as I read in many reviews that people tend to experience less nausea using the Vive for long periods than with Oculus.

      I own a Vive, and when playing Elite I usually get between 45 and 55 frames per seconds, yet the experience seems entirely smooth to me and I don't get any nausea at all. I find no perceivable difference to playing a game running at 90 FPS.

    4. Re:They do. by MarbleMunkey · · Score: 2

      The Vive uses Interleaved Reprojection, which drops the framerate to 45 fps, and then doubles (and warps the second copy of) each frame. This is enabled when the fps drops below 90. More info 'here.

    5. Re:They do. by wonkey_monkey · · Score: 1

      That's not motion compensation - not the kind the submitter is talking about, anyway.

      --
      systemd is Roko's Basilisk.
    6. Re: They do. by Anonymous Coward · · Score: 0

      its basically the same thing but with a twist.

      ita updating what you see(angle) more than 55fps.

    7. Re:They do. by wonkey_monkey · · Score: 1

      Pretty much this.

      No, not this at all. A lot of people are making this mistake. It's not what the submitter is talking about. He's talking about what is usually called "motion compensation" but should, in this case, probably be called "motion compensated interpolation."

      It has nothing to do with the motion of the viewer's head - which is what Async Timewarp is all about - but is about the "motion" between two frames of video, and using the detected motion to create an interpolated frame between each "real" frame to increase framerate.

      --
      systemd is Roko's Basilisk.
    8. Re:They do. by Lord+Crc · · Score: 1

      No, not this at all. A lot of people are making this mistake. It's not what the submitter is talking about.

      Of course this is not what the submitter is talking about, I never said it was.

      The need for high framerate in VR is mainly due to rotation of the head, which is exactly what time warping is good for.

      If one were unable to rotate your head and could only translate it, VR wouldn't have the same need for 90+ FPS.

      I can't see how motion compensation would do a better job, in fact I don't quite see how it would be any good at all.

      So yes, very much "this" indeed.

    9. Re:They do. by Anonymous Coward · · Score: 0

      Here's the one I liked https://youtu.be/WvtEXMlQQtI?t...

  5. Latency by Kjella · · Score: 4, Informative

    Broadcast TV with this stuff on has considerable latency. It looks at the next frame and then interpolates its way there, might work reasonably well to watch a recorded show but would be pretty horrible for real time motion from VR. Next question?

    --
    Live today, because you never know what tomorrow brings
    1. Re:Latency by ndnet · · Score: 5, Informative

      This!

      Even with a stable framerate, this technique intentionally delays the next frame to add compensation frames.

      As an example, let's have a magic VR helmet running at 120Hz and instant processing (ie, 0ms GTG time, which doesn't exist) and a video card capped at a perfectly stable 30 FPS (aka 30Hz).

      We will split a second into ticks - we'll use the VR helmet's frequency of 120 Hz, so we have 120 ticks, numbered 1 to 120. (Just to annoy my fellow programmers!)

      We therefore get a new actual frame every 4th tick - 1st, 5th, 9th, etc.

      Without motion compensation, we would display a new frame every 4th tick - 1st, 5th, 9th, etc.
      With ideal (instant) motion compensation, we can't compute a transition frame until we have the new frame. So we could, theoretically, go real frame #1 on 1st tick, computed frame based on #1 and #2 on 5th tick, real frame #2 on 6th tick, computed frame based on #2 and #3 at 9th tick, etc.

      This would also be jerky - 2 motion frames then 3 at rest? We could push the frames back a tick and fill the interval with three compensation frames, but then we increase the delay, which is always higher than this example (and is multiplicative). So we'd have frame #1 at 5th tick, computed frames at 6th/7th/8th, frame #2 at 9th tick, etc. You've now introduced a minimum 4 tick delay, which at 120Hz is 1/30 of a second, or 33ms! To an otherwise impossibly-perfect system!

      What about historical frames instead to PREDICT the next blur? Well, then, when something in-game (or, really, on screen) changes velocity, there would be mis-compensation. (Overcompensating if the object slows, undercompensating if it speeds up, and miscompensation if direction changes).

      There's more problems, too:
      - This doesn't help when the video card frameskips/dips.
      - Instant GTG and instant motion frame computation do not exist. At best, they're sub-tick, but you'd still operate on the tick.
      - Input delay already exists for game processing, etc.
      - Increased input delay perception would be exponential to the actual length of the delay. For example, 1-2ms between keypress and onscreen action? Hardly notable. 50ms delay just to start registering a motion on screen and course correct? Maybe OK, maybe annoying. 150-200ms? Brutal.

    2. Re:Latency by ShooterNeo · · Score: 1

      It is an unsolvable problem. In order to create an intermediate frame you must know the future frame.

    3. Re:Latency by windwalkr · · Score: 1

      They already know the camera position for the future frame, which is enough to make some minor adjustments. The result isn't as good as having the actual future frame, but it's also better than using the past frame.

    4. Re:Latency by Bite+The+Pillow · · Score: 1

      Wait, an actual explanation instead of "because profits, and also fuck you"? What site am I on?

      Ah yes, it is a rather narrowly targeted question about technical details, instead of business, politics, or sociology. Therefore, the guy who thinks he's right probably will be.

      Well done. The rest of youse guyse, still on notice.

    5. Re:Latency by Anonymous Coward · · Score: 0

      Exactly

      Latency from motion detection is the issue for a good (not sickening) VR experience, not so much FPS. Adding latency to increase FPS is a bad idea.

    6. Re:Latency by rocket+rancher · · Score: 1

      Thanks for the informative post. VR can't happen with current or foreseeable technology for precisely the reasons you mention. Sadly, VR has become just another religion that exists only to siphon money from the faithful. AR is a much more rational approach to an immersive experience. Why create a fake world, when the real world is already rendered for you, and has a perfect physics engine, as well? Just tag the parts you are interested in and use sprites if your tags need to move.

    7. Re:Latency by thegarbz · · Score: 1

      but would be pretty horrible for real time motion from VR. Next question?

      Motion VR? It's horrible enough for playing standard XBox games. It's the first thing I turn off on my HDMI inputs when I get a new TV.

    8. Re:Latency by Anonymous Coward · · Score: 0

      The higher we can push the output of new frames, the shorter the time shift gets. In principle a gpu rendering 360 fps, could be shown at 1440 fps with only a teoretical time shift of 2.7 miliseconds.

      Whether extra calculated frames would help at this point is doubtful, but it might be used for advertising purposes.

    9. Re:Latency by HalAtWork · · Score: 1

      Certainly it takes much more computational power, and therefore greater potential latency, to actually render the raw frame than to do motion compensation?

    10. Re:Latency by Anonymous Coward · · Score: 0

      The issue is in order to interpolate you need both the old frame and the new frame. Great for watching a football match, because being an extra few frame behind is not an issue. Adding extra response time lag is terrible for VR though.

    11. Re:Latency by wonkey_monkey · · Score: 1

      VR can't happen with current or foreseeable technology for precisely the reasons you mention.

      Err, no. The reasons mentioned have nothing to do with why VR "can't happen." The motion compensation trick is entirely unnecessary if the GPU can render frames fast enough. And it runs counter to the aims of VR, anyway.

      In case you hadn't noticed, VR is already happening. It might fizzle out, it might not. It's too early to say.

      It sounds like you just have a beef with VR, saw an opporunity to air said beef, and took it without really reading what was being said.

      --
      systemd is Roko's Basilisk.
    12. Re:Latency by wonkey_monkey · · Score: 1

      They already know the camera position for the future frame

      How? Player input and/or head position can change before the future frame is to be rendered.

      --
      systemd is Roko's Basilisk.
  6. Because: Latency by Anonymous Coward · · Score: 0

    That's the other side of multi-sided coin they are fighting. Motion compensation has to have two frames already processed, then add one in between so it is 2 frames behind. Adding 1 frame is already too much of an issue. Same reason you don't use motion compensation for gaming on TVs.

  7. It's not throughput, it's latency. by interiot · · Score: 3, Informative

    It's not throughput that matters, it's latency. If there's more than a tiny delay between turning your head and your eyes seeing the viewport move, then many people get bad motion sickness.

    1. Re:It's not throughput, it's latency. by Anonymous Coward · · Score: 0

      Also, what the OP seems to miss is that between the two ways of increasing the frame rate, organically rendering faster actually *reduces* latency, while interpolation *increases* latency. Hence why GPU coding is focusing on things like multi-viewport rendering as potential methods to reduce the strain of VR render loads.

  8. Because it works by delaying frames by Qzukk · · Score: 4, Insightful

    In order to add a frame between 1 and 2, you have to have received both frame 1 and frame 2. People are already getting sick because what they see and do don't match, you're going to make it worse by making what they see lag further behind what little the headset picks up.

    --
    If I have been able to see further than others, it is because I bought a pair of binoculars.
    1. Re:Because it works by delaying frames by Anonymous Coward · · Score: 0

      Would it really be noticeable to step one frame backward in time when we're already talking 30-120 fps (i.e. 1/30th to 1/120th of a second delay)? I would think that the overall improvement might still be worthwhile if what OP is saying is as simple as it's stated.

    2. Re:Because it works by delaying frames by Anonymous Coward · · Score: 0

      In order to add a frame between 1 and 2, you have to have received both frame 1 and frame 2. People are already getting sick because what they see and do don't match, you're going to make it worse by making what they see lag further behind what little the headset picks up.

      Exactly true. With a broadcast, unless you are at an actual sporting event live at the same time you are watching the tv, you have no frame of reference as to when something actually happens and when it is displayed. Even if you are an actual event and become aware of the fact the event on tv are not real-time down to the exact millisecond, your brain still changes focus to the tv when watching it and accepts the delay.

      With VR, events such as head tracking are expected to be acted on immediately to seem real. 90FPS is not just about how smooth things are between frames but also bringing the response time closer to 11 milliseconds. Throwing in a motion compensation chip to buffer frame 1 and frame 2 to add a seamless frame 1.5 will add at least one frame of delay. So, 90FPS may now stream like 135FPS but it is now 22 milliseconds of latency instead of 11. So from a latency perspective, the motion compensation applied to 90FPS produces the same responsiveness as 45FPS as both the 135FPS via motion compensation and raw 45FPS will have 22 milliseconds from the time you move and the time the images reflect the movement. At best this will just make you feel a little like your drunk. More likely this will cause the content of your stomach to become external.

    3. Re:Because it works by delaying frames by windwalkr · · Score: 1

      This isn't really accurate; you can (and they do) use frame 1 only, along with knowledge of the camera motion through the scene. It's not perfect, but it's better than missing a frame.

    4. Re:Because it works by delaying frames by Anonymous Coward · · Score: 0

      The human mind while it can't see the significant differences once you get above 60fps for most people, it can though distinguish inconsistencies in frame rates, artifiacting or other anomalies, usually sub consciously. The net effect is discomfort and nausea.

    5. Re:Because it works by delaying frames by wonkey_monkey · · Score: 1

      Yes, it really would be noticeable.

      This is why modern TVs have game modes to turn off motion compensation and other processing. Otherwise there's too much latency and gameplay suffers.

      Even with game mode, I still find playing on a CRT or other truly-dumb monitor to be much more responsive for gameplay.

      --
      systemd is Roko's Basilisk.
    6. Re:Because it works by delaying frames by wonkey_monkey · · Score: 1

      Everyone is conflating two entirely different concepts.

      First, there's motion compensation - perhaps better termed "motion compensated interpolation." That's what the GP and the submitter are talking about. For that you need the future frame.

      But there's also what Nvidia call Asynchronous Timewarp, which compensates for the motion of the player (but otherwise has nothing to do with the other kind of motion compensation) by warping a single frame, either because it has the extra time to do so before display is required, or because the new frame is not yet ready, so gets dropped and replaced with a warped copy of the previous frame to mitigate stutter.

      --
      systemd is Roko's Basilisk.
  9. Won't that increase the lag/latency? by Anonymous Coward · · Score: 0

    You would need to draw the next frame before you could interpolate. You may as well blast the next frame instead of waiting to draw one in between. Latency reduces immediacy, which induces motion sickness.

  10. Short Answer by Anonymous Coward · · Score: 0

    The FPS requirements on VR are less about the number of frames rendered and more about the latency of computing the frame, so that the position that your head was in when the frame started rendering is as close as possible to the position it is in after rendering completed. However, if rendering a frame only takes 10 ms, then there is no reason not to render 100 FPS. On the other hand, the pipeline techniques will tend to result in subsequent frames having started rendering while the previous frame is still in progress, which means that 90 FPS doesn't strictly guarantee a latency of 11 ms for the frame.

  11. Latency by Anonymous Coward · · Score: 0

    Creating extra frames causes latency in the video feed. That would be worse than low frame rate.

  12. Not the right solution by Anonymous Coward · · Score: 0

    The televisions are interpolating between two frames. The issue with the graphics is that you need a new frame reflecting the more recent position of your head. Since that data isn't available until you render the scene from the updated camera position, you would have to use stale data, which is out of sync with how you moved your head, and is more likely to make you sick.

    You could render a lower resolution image and just pixel double it, or just have a simpler scene if you are concerned about the GPU cost.

    Some of the more modern cards also include optimizations to be less accurate on the outer pixels, which can make things more efficient. This is because unlike a monitor, where you look at all parts of the screen, the head mounted displays generally have whatever you are looking at in the middle, and the outer pixels are for your peripheral vision. Of course, you'll have to wait a generation before those are in the entry level cards.

    Another optimization is that since the scene contents don't change between the eyes, you should be able to draw from both camera positions (one for each eye) without clearing the card's information about the scene, but again, this is something that won't be on entry level cards for a bit.

    1. Re:Not the right solution by Anonymous Coward · · Score: 0

      You could also use the depth information to recompute what the scene should look like from a slightly different angle. If you did that, you'd probably render to a buffer that is larger than the screen, and then when the user's head moves, you'd just slide the image around. Unfortunately, that makes everything move as though it's the same distance away. You could go further, and incorporate the depth buffer information (so things that are close move more than things that are far), but that runs into an issue where you should be able to see things that were behind other things, so you have to make up what goes in those spots (probably averaging the foreground and background). It might work for every other frame at 60Hz, but it's probably not worth it.

      -Same AC

  13. Latency by PhrostyMcByte · · Score: 2

    High frame rate is important for motion data, but it's also (and perhaps primarily) about latency. It is *much* more jarring in my Rift to experience latency -- akin to rapid sea-sickness -- than to have a lower frame rate.

    Getting more motion information would be great but we can't sacrifice latency for it, and those TVs tend to have a very noticeable amount of latency. Not that this is an unsolvable problem -- I just haven't seen it yet.

  14. They do something similar. by Anonymous Coward · · Score: 0

    As people have been saying, motion compensation introduces unacceptable lag. There is however a similar feature that does work with VR, called Asynchronous Timewarp which slightly warps the image to match the most recently sampled head position to compensate for motion between render and display time.

    https://developer.oculus.com/blog/asynchronous-timewarp-on-oculus-rift/

  15. Huge summary for nothing story. by Anonymous Coward · · Score: 0

    Slashdot used to have interesting topics and wise comments. Now it's just wack-a-mole.

  16. Timewarp and Reprojection by Anonymous Coward · · Score: 0

    Aren't they already doing these things?

  17. Latency by maorb · · Score: 2

    The biggest reason is latency. People start getting nauseous when they turn their head in a VR headset and their view doesn't change quickly to match the head movement. Motion Compensation on TVs relies on having at least two frames (or 22ms worth of frames at 90fps if this were a current-gen VR headset) already at the TV in order to do the calculations for a frame in between, and in practice they could be buffering 2-3 seconds worth of frames for their calculations and you'd never notice that your TV is displaying your channel 2-3 seconds behind everyone else because it's not interactive.

    Unfortunately even 22ms of desynchronization between our heads and our eyes is uncomfortable for most people.

    I imagine that at a high enough refresh-rate (180hz?), you could get away with the added delay (11ms at 180hz, 6ms at 360hz), but at that point the image would already be very smooth so you wouldn't be gaining anything of value.

  18. foveated rendering by Anonymous Coward · · Score: 0

    I'm more concerned about when fove ate rendering will become standard...that will reduce card spec requirements significantly.

  19. Also because it is artifact-prone by gweilo8888 · · Score: 1

    It doesn't help that the technique is also *very* prone to artifacts, and those artifacts would differ per eye, making them even more noticeable in 3D.

    1. Re:Also because it is artifact-prone by wonkey_monkey · · Score: 1

      It doesn't help that the technique is also *very* prone to artifacts

      But the submitter said "no visible stutter or strobing whatsoever"!

      Do you mean to say he may be mistaken?! I just don't know what to believe any more.

      --
      systemd is Roko's Basilisk.
    2. Re:Also because it is artifact-prone by gweilo8888 · · Score: 1

      Stutter and strobing are different things entirely to artifacts.

  20. frame rate vs latency by Anonymous Coward · · Score: 0

    VR is Virtual reality - everything you see is rendered.
    You don't particularly need a super high frame rate, and overall system latency is the real killer (latency is the overall time it takes for a render operation to occur and finish displaying the image). So real time motion compensation is not really necessary.

    AR is Augmented reality - virtual objects are overlayed on real world data.
    Here latency is is even more crucial, as without super-low latency, objects appear to jitter and swim around chasing your view (since the virtual object and the real life objects are moving differenetly). Again, frame rate is not sure crucial.
    All sorts of corrections are done to minimize the latency, including using motion sensors and dedicated hardware. Microsoft's Hololens actually does an amazing job at this, and their engineers deserve major kudos for making it possible to anchor objects so well.

  21. Please make this room stop spinning by Anonymous Coward · · Score: 0

    50% more input lag, coming soon!

  22. Huh? by Nemyst · · Score: 2

    you need a GPU capable of pushing at least a steady 90 FPS per eye, or a total of at least 180 FPS for both eyes

    Um, what? That's entirely wrong. You need a steady 90 fps, that's it. There's no doubling because of eyes, this isn't 3D TVs where you need to alternate frames. The only other concern is that the resolution is higher than 1080p.

    1. Re:Huh? by ShooterNeo · · Score: 1, Informative

      No, it's correct. For a single GPU solution without Nvidia's new tech (that won't be widely used until AMD releases an equivalent) you render the frame for one eye, clear buffers, render it for the other. That means if you sent all frames rendered to a single monitor you'd get 180 FPS.

    2. Re: Huh? by Miamicanes · · Score: 2

      Actually, it's worse than that. 90fps is the rock-bottom MINIMUM. If you want a frame rate that satisfies the Nyquist minimum for high-contrast peripheral vision, even 400fps is on the low side.

      The human eye is really hard to fully satisfy with immersive video, because trying to define the eye's "resolution" or "frame rate" is kind of like trying to define the resolution of an Atari 2600 or an Apple IIe. The fovea has relatively high color resolution, but comparatively poor luminance resolution. Peripheral vision has poor color perception, but is extremely sensitive to motion.

      That's why 60hz CRT multisync monitors looked OK when you looked directly at them, but visibly flickered if you saw the screen in peripheral vision while focusing on something else, like a book in your lap.

    3. Re:Huh? by Anonymous Coward · · Score: 0

      Is that really how it's done? The viewpoints are not that far apart, the scene is almost exactly the same, certainly the geometry and lighting are the same. Wouldn't it be more efficient to render them simultaneously?

    4. Re:Huh? by Anonymous Coward · · Score: 0

      This argument is so stupid.... but I'll bite: VR IS 3D. Those frames are alternate because of TV/glasses design, but from GPU POV there is no difference. It still computes 2 scenes, so in comparison to non-3D play, at same (FullHD) per-eye resolution, you need doubling of performance. Now, those smarphone based solutions, those actually split the 1920x1080p image into 2 960x1080 images, thus the actual pixel count is the same as non 3D.

    5. Re: Huh? by Anonymous Coward · · Score: 0

      It really is rendered separately now. That is why nvidia's single pass multi-viewport tech is a big-ish deal.

    6. Re:Huh? by frinsore · · Score: 0

      No. It's not 180 FPS. You're doing the math wrong. A few examples: My monitor refreshes at 60 fps. I've got a chrome window open; you can say that chrome is rendering at 60 fps. Now I open another chrome window. Does this mean chrome is rendering at 120 fps? No. It means that chrome is rendering 60 fps in two windows. Split screen games where one player is on the top half of the screen and the other is on the bottom half, does that double the fps on the game? No. It's simply rendering two cameras at half the resolution at the normal frame rate. (performance of this mode is usually poor for a lot of reasons such as larger world simulation area and two separate culling passes neither applies to vr). Google cardboard can render simple vr. When watching a 360 youtube video just because a you tube video is displaying two slightly different perspectives side by side inside the same video doesn't change the frame rate of the video. Or how about game's that include mirrors that show actual gameplay? The way a real time mirror works is that a camera is placed from the mirror's perspective, renders the scene to a texture and then that texture is placed at the mirror's location. Does the fps suddenly double when a player walks into a bathroom just because there's a mirror rendering a different perspective? In the end the ultimate factor for graphics is: How many pixels and how often a second. If you're rendering the left half of the screen, throwing everything out, then rendering the right half of the screen, throwing everything out, & repeating then you're doing it wrong. There is no reason why the single frame needs to be rendered twice but with half content. Just like the hardware render the two cameras in parallel like it was designed to do.

    7. Re:Huh? by ShooterNeo · · Score: 1

      That's not how it works.

    8. Re:Huh? by Anonymous Coward · · Score: 0

      That's not what it does currently. For stereo rendering you just double the draw calls and modify the projection matrix for the second eye in between. All that is done in one frame.

      Also Nvidia's new tech is not required to do that in one pass, it has been done already for S3D in 2010 with instancing.

    9. Re:Huh? by Nemyst · · Score: 1

      But then you're "only" rendering on a 1080x1200 screen, which is quite a bit smaller than a 1080p screen, whereas I'm pretty sure from the "high resolutions" part of the submitter that they expected the full 2160x1200 at 180 fps.

      Besides, you don't need particularly novel tech to render on two buffers simultaneously, reusing as much of the work as you can. Stereoscopic rendering is just a change of matrices (ignoring the final projective steps), so you can reuse a lot of data and processing. Just rendering each eye independently is the dumb/brute force way, not to mention that one eye would always be a little late compared to the other.

    10. Re:Huh? by ShooterNeo · · Score: 1

      Apparently that "just" bit requires new GPU hardware, and at this instant in time, only 2 GPUs (the 2 new Nvidia 1000 cards) even support it at all. Any VR developer who wants to eat has to support AMD hardware as well because it's the chip in the PS4 and probably the chip used in the new generation of VR supporting consoles.

    11. Re: Huh? by Anonymous Coward · · Score: 0

      You're just trying to sound smart by saying "Nyquist". Truth is, during saccade the eye can distinguish about 20 fps or less.

    12. Re: Huh? by Miamicanes · · Score: 1

      Bullshit. Look at any demo that animates a high-constrast white object horizontally on a black background at a rate of 120 pixels per second on three monitors side by side... one moving it by one pixel every 120th of a second, one moving it by 2 pixels every 1/60th of a second, and one moving it by 4 pixels every 1/30th of a second. I can absolutely GUARANTEE that nearly everyone can accurately and easily tell them apart (though some users might need to be told what to look for).

      Yes, to some extent, motion blur can compensate for framerate (because you're replicating in source video what the eye itself would see with hard-edged motion at a high framerate), but unless you get it EXACTLY RIGHT (or more precisely, "at least twice as good as exactly right"), the brain is still going to notice that something isn't quite right & you'll crash-land into the uncanny valley.

      I've worked with VR. In the medium term, at least, high-framerate is likely to be a much easier way to bridge the uncanny valley than attempting to perfectly simulate motion blur.

      15 years ago, we had multisync CRTs that could do 120hz or more without breaking a sweat (at least, as long as you reduced your horizontal pixel clock and blurred it a bit to remain within the CRT's bandwidth limits), but unfortunately we didn't really have graphics chips, memory, or CPU horsepower to really put it to good use. Then, right around the time the graphics hardware started to become powerful enough, LCDs became dominant almost overnight (something that took a lot of people by surprise, because LCDs looked better than shit CRTs, but the best of the best flat-front CRT displays totally spanked them, even though they weighed a ton and took a huge amount of desk space... we HAD those best-of-breed CRTs, and underestimated just how awful mainstream CRT displays were for the unwashed masses). Simply put, the LCD market stagnated in terms of resolution and framerate, and focused almost entirely on cost & size (with a tiny bit of work on improving color fidelity and response times). Many people forget that there was a period around 2008 when it was nearly IMPOSSIBLE to buy a laptop (or monitor) with a resolution besides 1920x1080 or 1366x768, and if you did, you'd have paid prices that would have made Apollo-era NASA gasp.

      Thankfully, we're finally on the cusp of monitors that can be updated (at least partially) at effective framerates approaching 300fps (though we're probably still at least a decade away from seeing affordable monitors that can literally change 100% of their pixels from frame to frame at 300fps, partly because the industry has now fetishized "UHD" at the same slow external framerates [hence, the current glut of supposedly 240hz TVs that can't even be directly driven at 120fps]).

      Is 300fps overkill for foveal vision? Mostly, yeah. The problem is, without putting a dedicated display in front of each eye and using gaze tracking to figure out WHERE you need to sacrifice resolution for raw framerate in realtime, your only alternative is to satisfy the most demanding requirements for both peripheral-motion and foveal-detail across the entire screen, since the user could conceivably be looking anywhere.

  23. Because there is no such thing as magic by BlueCoder · · Score: 1

    It works in a TV broadcast because it's a stream. It will work in a 3d streaming video to a headset but that's without head tracking.

    It can't work for interactive head tracking. It's a thing called latency and it's the reason people get headaches from VR. Your brain does it's own head tracking and when what you see doesn't match you get vertigo and or a headache.

    You also get vertigo from confusing your brain by spending too much time in zero G. A ride in an elevator can make you lose your cookies. That's because it remembers both what it expects with gravity and without. You feel motion but you don't see it and your brain is drawing two different opposing conjectures.

    1. Re:Because there is no such thing as magic by Jeremi · · Score: 2

      You feel motion but you don't see it and your brain is drawing two different opposing conjectures.

      ... which is actually kind of amusing when I think about it. There's a watchdog circuit somewhere in your brain dedicated specifically to checking whether or not your sensory inputs match up, and when it detects that they don't, it assumes that you are drunk or high (or otherwise somehow poisoned) and initiates the upchuck routine. How many generations of questionable-quality-alcohol drinkers did it take to evolve that?

      --


      I don't care if it's 90,000 hectares. That lake was not my doing.
    2. Re:Because there is no such thing as magic by Shinobi · · Score: 1

      "It's a thing called latency and it's the reason people get headaches from VR"

      No, it's ONE of the reasons people can get headaches from VR. Another cause of VR-related migraines is that up to around 25% of all humans has some degree of problem with depth perception, and using VR helmets will trigger that, the more problems you have with it, the faster it will trigger. Resolution, FPS, motion latency has got nothing to do with that problem

    3. Re:Because there is no such thing as magic by wonkey_monkey · · Score: 1

      And another is that there is no difference in focus in a VR world. Your eyes will strain to make sense of that, just as they do when you wear someone else's glasses.

      --
      systemd is Roko's Basilisk.
  24. gaming needs the current video, not old video by frovingslosh · · Score: 1

    Actually, motion compensation requires both the current frame and one or more future frames to be able to compute intermediate frames. With a media player where the full video already exists it is simple enough to access "future" frames. With a TV showing video inbound from a broadcaster, cable company or media device, you can delay the output by 1/30th or 1/24th of a second, delay sound similarly, and no one is the wiser. P>

    But in video gaming, delaying the video by 1/30 or 1/24 or even 1/20 of a second has serious effects on game play. You really don't want a gaming device that delays the video just so that it can insert extra frames, you need to see the most recent frame as soon as you can.

    --
    I'm an American. I love this country and the freedoms that we used to have.
  25. They already have this by Anonymous Coward · · Score: 0

    They already have this for VR. It's called Asynchronous Time Warp. If a frame isn't delivered in time then the previous frame is rendered with the new head position and warped around. It's not ideal as for me even at 90fps on the consumer rift I can tell when timewarp is occurring for a dropped frame.

  26. Not Real Time by Anonymous Coward · · Score: 0

    As others have said, its not real time. The frames are delayed before being displayed and any delay in VR causes sickness in VR. So as you turn your head, there is latency added for motion compensation before it is displayed.

  27. They actually do.. by Anonymous Coward · · Score: 0

    Though not in the way you are asking.

    First what the TV is doing is doing an intelligent interpolation of two frames to create something that is *usaully* a decent approximation of what the frames missed. Not in animation, it frequently does a bad job.

    In 3D gaming, the closest analog would be motion blur (actually it's the exact opposite). 30 FPS is very low compared to what the eye can make out, but we generally don't notice it because exposure time on the captured frame is long enough to do a bit of blurring. 3d engines can imitate this so the sudden 'stutter' is less noticable.

    What VR does is render a larger scene. Between the actual rendering engine updating the world, the users head movement is used to move around the most recently available scene. Valve does not do this, Oculus does (called time warp) at one point they were supporting both orientation and positioning (which was quite awesome, you could render at ~40fps and be none the wiser), but have since stopped doing positional time warp, but still does the rotational, which can smooth over gaps,but not nearly as seamlessly. It was intended as a late-stage correction to mitigate the mal effects of unavoidable latency, but it is able to mitigate lost frames as well. Translational movement will undo it pretty quickly.

    I never did hear why Oculus stopped doing positional, it was quite awesome.

  28. Asynchronous Time Warp by Anonymous Coward · · Score: 0

    The Oculus Rift already does something to a similar effect, asynchronous time warp:
    https://developer.oculus.com/blog/asynchronous-timewarp-examined/

    Basically, for missed frames, the software compensates by distorting the image based on the difference between your head's current rotation and the rotation when the image was rendered. Effectively, imagine the game rendering out a quad for each eye that gets fixed in space, and then that quad gets fixed in space some distance in front of your eyes. You can rotate your head, and even without new rendered frames, your eyes will see something reasonable (until you turn too far, or if your head is translating). The texture and position of the quad then update once a new frame actually completes, and repeat. This is a lightweight operation that runs at 90Hz, regardless of the game's framerate. It's not as good as actually hitting framerate, but it's a solid heuristic to smooth things out.

  29. They do... by Anonymous Coward · · Score: 0

    Valve, occulus, and sony are doing research on this sort of thing. It doesn't work very well at anything below 60 fps.

  30. Idiot by kbg · · Score: 0

    Because you are an idiot?

    In order to generate extra in between frames you need at least 2 frames. For TV this means that the output is delayed for a few milliseconds which isn't a problem because TV is not interactive but for a VR which is interactive you just added more delay into the controller response system.

  31. Latency not FPS by Anonymous Coward · · Score: 0

    Its all about latency. Your head needs to move in game at near the exact speed in real life. Smoothing wouldn't do anything to increase the latency, only the framerate.

  32. You said it by Anonymous Coward · · Score: 0

    Feels like the submitter is a child of some sort...

    I thought oooh, someone saw a term wanted to pretend smurtness .

    Amazing that someone can ask such a question when a simple understanding of motion tracking exists and has been around for.. uh, well over half a century. I know.. easier to get the cheat sheet than do any homework at all.. but you give away your ignorance when you try to convince people that you did some homework.

  33. Some graphics card already do by Yakust · · Score: 1

    NVIDIA does exactly that : warps the image according to most recent (or even predicted) head position.
    Works on their recent high end cards
    http://www.slashgear.com/nvidi...

    1. Re:Some graphics card already do by wonkey_monkey · · Score: 1

      Not what the submitter is talking about.

      There's "motion compensat[ed interpolation]" and there's "compensating for player motion." Two entirely different things.

      --
      systemd is Roko's Basilisk.
  34. ATW FTW by WaffleMonster · · Score: 2, Insightful

    Oculus 1.3 runtime for the Rift was released with async timewarp. When it was released DK2 users used to earlier runtimes without it were all over the boards with phrases like "holy shit" and "DK3" to explain how ATW changed everything for them. Jitter issues magically disappeared overnight with only a simple software update.

    More generally there is one and only one "trick" for improving VR quality going forward and that is foveated rendering. This technology is absolutely critical to any serious vision of future HMDs.

    To provide some context cones of our eyes cover a massive (cough cough) 15 degrees of arc. That's it. You can't even lean back and read 1/4 of what is on your monitor without moving your eyeballs around to do it. 4k is overkill.. 1080 is overkill... The future in VR is entirely locked up in sensing eye orientation and optical and or electronic steering of relatively low resolution displays in response.

    1. Re:ATW FTW by Anonymous Coward · · Score: 0

      The future in VR is entirely locked up in sensing eye orientation and optical and or electronic steering of relatively low resolution displays in response.

      That still requires re-rendering the frame very, very quickly with the higher resolution area relocated to where you look now. Latencies will have to be significantly below 10ms to not make Level of Detail irregularities pop up all the time.

    2. Re:ATW FTW by jeffb+(2.718) · · Score: 1

      Yes, very quickly -- even more quickly than with other approaches, because your eye saccades really fast. (Although you probably don't have to render intermediate frames during the saccade, because IIRC your brain sort of ignores incoming detail during the movement itself.)

      The win is that you have to render a much smaller patch of high-resolution detail, which saves computation in some parts of the pipeline. You've still got to do all the intersection, bounding-pyramid, depth-sorting and whatnot for your whole model (I assume -- it's been close to thirty years since I've done much in 3D graphics), but you don't have to crank out as many pixels. To get "retina" resolution, you'd need a foveal patch no more than maybe 500 pixels in diameter, and the rest of the visual field could probably get by with another 500 pixels on a side. That's a lot less painting than the stack of 3840x2160 screens you'd need to fill the whole visual field at that resolution -- probably between one and two orders of magnitude smaller.

      As far as the rest of the pipeline, I guess you win because those rapid saccades don't significantly change the viewpoint (head location), so there won't be a lot of changes to object order or occlusion. (Your pupil does shift its position, since the eye's center of rotation is obviously near the center of the eyeball, but any virtual object that close to your eye ought to be blurred anyhow, unless you're playing the final levels of Quest for Nearsightedness.)

  35. Re:asked and answered by Lunix+Nutcase · · Score: 1

    It wouldn't do the job unless increased motion-sickness is the intended goal.

  36. Because it would be cheating by Anonymous Coward · · Score: 0

    Give gamers a frame rate of 100 to 200 FPS (more accurately a latency of 5 to 10 mS) to track head motion. But only refresh the scene at 30 FPS. And pretty soon, somebody will figure this out and go all autistic about how the high refresh rate/low latency isn't legit. And their subconscious will convince them that they should be getting sick. So they will.

    This actually happened some years ago where, due to a driver glitch, a new graphics card wasn't actually living up to its advertised specifications. Everyone read the numbers on the box and figured that this was the best performing h/w on the market. And it's performance demonstrably better than older models. People didn't start to get headaches from the flicker until the news broke about the driver glitch.

    Setting the latency issue aside, the human visual system appears to be evolving at exactly the rate that graphics vendors are coming out with new hardware. Or I must have missed the 20 to 30 percent of moviegoers that ran out of 24 FPS theaters 25 years ago with headaches, vomiting. Latency IS a legitimate issue with systems that have to track head motion. So a system that could rotate or shift a scene at 200 FPS would seem to have benefit. Even if the scene content were rendered at a slower rate, like 30 FPS.

  37. Several reasons by guruevi · · Score: 1

    a) What you're describing is frame doubling or similar tricks (every manufacturer has it's own name for it). It only makes the system APPEAR better and brighter. This is often done by simply doubling the frames or turning the backlights on and off between changing frames. This makes it appear "crisper", brighter and smoother, it's only a trick though and most displays already do it, it doesn't actually improve the actual experience, only tricks your brain into thinking it is at first glance and often when you do have a real 60 or 120fps source, it interferes with it (and thus needs to be disabled, either manually or automatically). It does increase fatigue.

    b) VR needs to be as close as realtime as possible. When you move your head, your brain expects the world to move right away (most current research pegs the human eye's temporal resolution to be at about 10ms or ~100Hz in bright environments). So when you make your 60Hz game (~16ms latency) into a VR game, you WILL get some issues because you are basically lagging an entire 5-10 frames behind the whole time (your input latency on very high end systems is typically ~10ms, processing latency and video card latency add another 20-50ms and then LCD latency (the time it takes for an LCD to go from black-to-white, not gray-to-gray) is typically another 10ms).

    c) Most frame doubling that uses a 'smoothing engine' requires the next frame to be available. You're now adding 32ms of latency (on a 60Hz system or ~3 "frames" for your eyes) just to get a smoother image so that is entirely useless.

    A 'good' VR system would require end-to-end control over the individual pixel arrays of the LCD displays as well as direct access to any sensor inputs (which all have inherent latency as well). The second you say "USB" you've already lost 10-100ms and when you say HDMI/VGA/DVI, you've lost another 8-16ms (because those are built for streaming images to "TV's". Hell, most video cards these days also use some form of double or even triple buffering (meaning your image comes out 1-2 frames too late already) in order to do all sorts of post-processing tricks (aliasing etc).

    --
    Custom electronics and digital signage for your business: www.evcircuits.com
  38. Total Misunderstanding... by barfy · · Score: 1

    These are not the same problems at all. Look at gaming monitors and Ping times to better understand the problem. (These aren't the same thing, but at least the underlying reasoning is closer). The VR problem is one of synchronicity. If your sitting looking at a screen of a movie or something, you don't need to change the image very often, and your brain will work it out. If there is something you are carefully paying attention too, like a ball in a sporting event. 30 FPS is not smooth enough (you will see gaps.), this also is worse with Tv's without phosphor display. And the ball will noticeably skip. So this trick gives you more frames with less time between them, so it won't skip. It's not perfect and some people get weirded out by the effect. BUT this has NOTHING to do with the VR problem. In this case it's synchronicity. Think about it, real life is relatively continues, there is no 30 fps per second, and there is reasonable belief that synchronicity has a much much faster ruler. You have a near perfect understanding between doing something and watching what happens when you do it. And if it seems real, but it's not. Turns out a large part of the population will get weirded out. And in the ol venn diagram way of thinking about it, that population and influential gamers is pretty damn high. So no it has to be fast ass cards. (there is a little bit of similarity, but one of the other problems has to do with speed of action. If your fast enough you solve by far the largest of synchronism issues. (remember when the rift first came out. Everyone was excited because they largely solved the synchronicity issue, but on low-res screens. The low res screen is problematic with moving input that is trying to reflect reality.If you move at the right speed you will get stuttering. While the timing will be right,but resolution stuttering will weird some people out. So it has to be fast enough to allow synchronicity enough, it has to be pixel dense enough that it doesn't skip unexpectedly differently from real life. None of this can be cheated, because we are not watching movies in real life. We don't ADAPT like when watching a movie.

  39. my nexus player does this by Sark666 · · Score: 1

    I have a nexus player which I like, but if I cast anything to it, or play local content with MX player I get the soap opera effect making a 24 fps movie look like it's at 60. Is it the Nexus player doing this or my TV? I've posted elsewhere and some said it's my TV but if so, wouldn't it do it with everything? Like Netflix YouTube etc.

    Drives me nuts.

    1. Re:my nexus player does this by wonkey_monkey · · Score: 1

      Is it the Nexus player doing this or my TV?

      It probably is the TV. It may have different settings for different sources. Look for "judder reduction" or something like that.

      --
      systemd is Roko's Basilisk.
  40. This has nothing to do with VR by Anonymous Coward · · Score: 0

    The proposed technology has nothing to do with why VR needs to be at 90fps.

    There's a lot of reasons but the simplest is that VR must, absolutely must, keep up with your head movement. If you move your head and the frame takes too long to catch up you're going to barf. And it turns out people move their heads pretty damned quickly while turning side to side. Because you don't know exactly where the players head is going to move (and yes there are prediction algorithms already) you don't have any data as to what is at the the area the player is moving their head towards until your render it. So there's no interpolation possible. You either miss the frame, and the head's movement doesn't correspond to their vision and they barf, or there's just a flicker of giant black nothingness which takes you out of the experience.

    Considering there's already a ton of other things that still make you barf in VR that need work, dropping frames just isn't really possible right now.

  41. System approach by ChrisMaple · · Score: 1

    Many posters have pointed out that latency in the graphics card is an inherent problem.

    There's also latency in the monitor, and latency in the computer which creates each frame. Merging the functions of the monitor and the graphics card should allow a latency reduction of 1 frame (at the display rate). (This brings up new problems, of course. Interfaces change, and the monitor has to handle the weight and heat of the graphics card function - a problem if the monitor is a VR headset.)

    --
    Contribute to civilization: ari.aynrand.org/donate
  42. Asynchronous Timewarp by NoZart · · Score: 1

    ... does exactly this, and it works a treat. Only difference to TV's is, it is only used when your framerate goes below 90 fps.

    It does look weird when it goes on for longer periods, but the alternative would be juddering which is insta-puke-city.

    1. Re:Asynchronous Timewarp by wonkey_monkey · · Score: 1

      Asynchronous Timewarp is something different than what the submitter is talking about.

      The submitter is talking about interpolating between rendered frames to create smoother video - i.e. only rendering every other frame fully, then just interpolating between them to fill the gap.

      The obvious problem is that it means you've added an extra frame's delay to everything. Also, motion compensated interpolation isn't perfect.

      --
      systemd is Roko's Basilisk.
  43. Interactive vs. Passive by XNormal · · Score: 1

    A good VR experience (and preventing motion sickness) requires fast response time. This requires low latency of the entire chain from the motion sensing device, through the USB connection, OS process scheduling, scene calculation and rendering and any buffering in the video card and display.

    A system that is able to respond quickly can obviously produce more frames per second. But just creating more frames per second without reducing latency will not help the experience feel more convincing (or prevent you from feeling queezy). It will just look a bit smoother.

    In playback of canned video latency does not matter much. In fact, generating these in-between images actually increases overall latency as the system has to delay the next image while calculating and displaying the in-between image. As long as an equivalent delay is inserted into the audio nobody notices this. But it won't work for VR.

    There is, however, a method to produce faster response without calculating more images per second. The most critical movements are those of the head and the change in the scene from such motions can be approximated by simple panning. It's not perfect, but does work to reduce motion sickness.

    --
    Stop worrying about the risks of nuclear power and start worrying about the risks of not using nuclear power.
  44. Why do that now? by iamacat · · Score: 1

    There are already GPUs that can do 90FPS without interpolation techniques which would not provide as good of a latency or eye tracking. In a couple of years the same level of performance will be cheap and widespread. Why would manufacturers focus on short term tricks and not the real thing?

  45. Because it doesn't work by Anonymous Coward · · Score: 0

    It's great if the objects on screen are moving in predictable patterns, but if they're not then it looks horrific.

  46. PSVR by Anonymous Coward · · Score: 0

    Isn't the PSVR actually using this? Since it's doubtful the internal PS4 hardware is capable of pushing 90 fps as sometimes even normal games have a hard time maintaining 60 fps.

  47. Because they're rich hipsters by Anonymous Coward · · Score: 0

    VR people are the rich hipster dicks who run out and buy a new processor or video card when it comes out because the old one is "obsolete". The reason that you're asking this is because you're one of the poors.

  48. because Latency, also you are a noob by citizenr · · Score: 1

    what good would extra 60 frames _from the past_ do to your VR experience? other than make you puke

    BTW Microsoft once championed the idea of generating 3D from a bunch of 2D tiles, this would enable tricks like motion compensation for tile reusing. It went pretty well for everyone involved (read billions invested/wasted by all MS 'partners').
    https://en.wikipedia.org/wiki/...

    --
    Who logs in to gdm? Not I, said the duck.
  49. Soap Opera Effect by Anonymous Coward · · Score: 0

    The first step in almost any TV calibration guide is to turn of the soap opera effect. The only reason TV manufactures still both with this junk is so that they can boast(lie) 120hz from a 60hz panel. All they are doing is averaging every two frames and inserting it in-between them. So the content still is not actually being updated at 60 fps. Further this takes so much computation that it adds a significant amount of input lag, over 100ms on many TVs, so it would completely unusable for games.

  50. Quality of articles.. by Anonymous Coward · · Score: 0

    Seriously. This is way below any acceptable standard for interesting articles.
    Parent doesn't understand the technology in question, despite mentioning it's an old and well known technology. Then produces a fantasy about how he/she thinks it should work for vr and asks the world "WHY!?".

    First of all, it's not 'real-time motion compensation. It's 'after the facts frame interpolation'.
    So immediately several problems become clear.
    1. since it's an interpolation, you need to delay the output because you can't create an interpolated frame if you don't know what the next frame is going to be. So the output is delayed by at least an extra frame, more if you want to do better interpolation.
    2. even with multi-frame interpolation these algorithms regularly suck ass and do all kinds of weird stuttering if the images are complex. THat is because it's a predictive algorithm and you can't possibly predict everything just from the pixels. This would be super ultra shitty for VR and would make you seasick in minutes. And not just if you're sensitive to it. It would be a positively puke-inducing experience. Besides this you would also get all kinds of artefacting going on between the eyes that would be super jarring.
    3. there is already a far superior technique that fits the problem much better. I think they call it Time Warp or something like that.

    So the writer of this article is just lazy and didn't do his homework.

  51. THEY ALREADY DO by Anonymous Coward · · Score: 0

    This is a really dumb question, because the answer is "they already do". Whoever asked this knows nothing about the state of VR.

    Carmack programmed time-warp to do exactly this like over a year ago. This is old-hat technology in VR. We've been doing this forever.

    PSVR even does this. Games render at 60-90hz and are interpolated up to 100-120hz.

    Clueless post.

    1. Re:THEY ALREADY DO by wonkey_monkey · · Score: 1

      Time-warp is not motion compensation, not in the sense used in the question.

      It does compensate for motion, yes, but "motion compensation" is a different thing in video terms (it should probably be called "motion compensated interpolation.")

      --
      systemd is Roko's Basilisk.
  52. Stupid question. by ChaoticLimbs · · Score: 1

    You ask why we don't generate a video frame that's halfway between the current video frame and ONE THAT WILL HAPPEN IN THE FUTURE. That is quite probably the stupidest question I've heard a nerd make.

  53. Sickening without, you mean by HalAtWork · · Score: 1

    I find it sickening when it's disabled, as motion is much more jittery and stutters while panning on today's blur-free LCD panels.

    On earlier TVs that would mask this with blurring or phosphor fade between frames, this was not an issue. Today's sets have high refresh rates with little artifacts and blur, so pans and motion appear full of stutter, especially on a large set where an object/person in motion travels larger physical distances between frames

  54. No, it's just 90 FPS by monkeyporn · · Score: 1

    FPS isn't cumulative like that. 90 FPS "per eye" is just 90 FPS overall. It's the draw area (number of pixels) that increases. And, with what we currently have, it doesn't actually increase. Both Oculus and Vive are 2160 x 1200, 25% more than the usual 1920 x 1080.

  55. obvious by Anonymous Coward · · Score: 0

    Would this not make far more financial sense for the average VR user than having to buy a monstrously powerful GPU to experience VR at all?

    awnsered your own question, capitalism at work

  56. Slanty eye Chinese cunts by Anonymous Coward · · Score: 0

    Stealing all our interpolation.

  57. they got better ideas already. by Anonymous Coward · · Score: 0

    gear vr and oculus and i think vive too already support changing the angle "midframe" basically, render a slightly bigger imagethan fits the display(s) and as it comes out of the render pipeline adjust it based on movement that happened since the frame went into rendering. you can notice it on gearvr by turning 180 quickly and seeing the black areas.

    i think the question asker isnt up to date on how the current gen vr works.. its not important if the content is 90fps. its important that the angle changes 90fps+.

    gearvr also uses trickery to time the blitting perfectly with this scheme, which makes the screen look weird if you look at it all at once. shame its such a wanabe closed ecosystem. oculus is really fucking itself up with the closed swhw combo angle they are trying to go for.

    also next android should have something similar support built in(mid-frame head movement compensation).

  58. Because you can't interpolate into the future by wonkey_monkey · · Score: 1

    You need to know what the next frame will be before you can interpolate towards it. This means delaying everything, That's a big no-no in VR.

    Also, motion compensation is not perfect. That's why all these "60fps!" trailers people like to put on YouTube look quite underwhelming.

    Real 48/60fps video looks a lot nicer.

    --
    systemd is Roko's Basilisk.
  59. The simple answer... by Anonymous Coward · · Score: 0

    Latency...all that motion interpolation takes GPU time, which just means there's more lag between the actual signal input and when the image gets drawn on the screen. The main reason VR is attempting to push 90FPS all the time is to minimize latency as much as possible between when the system detects your head moving and the image is drawn on the display.

    TV's have to do it because they only get 30 FPS to work with, so they're essentially tweening in frames generated out of thin air (I mean computed from the previous and next frame). However, if you try playing a videogame that requires precise timing using one of these so-called 120-240Hz modes, you'll find that there's input lag on the order of 100-150ms or so, depending on the TV. This is why many TV's have a specific "Game" mode that disables all of the TV's post processing to minimize input lag.

    1. Re:The simple answer... by Anonymous Coward · · Score: 0

      > Latency...all that motion interpolation takes GPU time

      No, submitter is not talking about computing more frames in 3D, they're talking about the DISPLAY unit interpolating between frames at a higher rate.

      The output from your GPU is exactly the same as it was before--just like your TV receives its signal at the same frame rate whether interpolation is turned on or off.

  60. Motion tracking lag by Anonymous Coward · · Score: 0

    Can't do that as it creates lag between perceive image and head tracking position, this will make you feel drunk and likely get sick the frame rate needs to be as close to realtime information not what was buffered and postprocessed after the act.