Slashdot Mirror


Human Eye's Oscillation Rate Determines Smooth Frame Rate

jones_supa writes: It should be safe to conclude that humans can see frame rates greater than 24 fps. The next question is: why do movies at 48 fps look "video-y," and why do movies at 24 fps look "dreamy" and "cinematic." Why are games more realistic at 60 fps than 30 fps? Simon Cooke from Microsoft (Xbox) Advanced Technology Group has an interesting theory to explain this all. Your eyes oscillate a tiny amount, ranging from 70 to 103 Hz (on average 83.68 Hz). So here's the hypothesis: The ocular microtremors wiggle the retina, allowing it to sample at approximately 2x the resolution of the sensors. Showing someone pictures that vary at less than half the rate of the oscillation means we're no longer receiving a signal that changes fast enough to allow the supersampling operation to happen. So we're throwing away a lot of perceived-motion data, and a lot of detail as well. Some of the detail can be restored with temporal antialiasing and simulating real noise, but ideally Cooke suggests going with a high enough frame rate (over 43 fps) and if possible, a high resolution.

28 of 187 comments (clear)

  1. Bring on HFR by Anonymous Coward · · Score: 5, Interesting

    I, for one, am sick of slow (seconds-long!) pans across scenery that *still* end up with judder and motion blur.

    HFR isn't a gimmick like migraine-inducing stereoscopic "3D", it's more akin to adding color instead of relying solely upon greyscale for film presentation.

    Like all tools, I'm sure it can be used for both good and evil. Blame evil, jump-cutting directors if the dark side is channeled.

  2. It's in the image by Diddlbiker · · Score: 5, Interesting

    Movies tend to be shot around 1/50" shutter speed, and that creates motion blur. The motion blur actually helps us see the animation as smooth, even at "only" 24 fps. Games on the other hand are razor sharp and will hence look much more like a staccato sequence of images than as an animation.

    Or so I was told by a moviemaker

    1. Re:It's in the image by PPalmgren · · Score: 3, Insightful

      Because in games, you're interacting with the content. Any delay in interaction is extremely jarring. Movies and such don't have this issue.

    2. Re:It's in the image by beelsebob · · Score: 2

      Suggest a technique for doing it then.

      Hint: there are bad solutions to this, like blending with the previous frame, and there are really really expensive solutions to this, like figuring out the paths things moved over during the frame, and rendering along those paths.

      Aside - games have a much bigger problem than motion blur. Movies run at a consistent 24fps, no matter what. Games meanwhile takes varying amounts of CPU and GPU time per frame, and have fixed time windows in which a frame must have been prepared. If that fixed time window is missed, the same frame gets presented twice (or at frame rates like 24fps 3-4 times), the effect of showing some frames form 16ms others for 33ms, still more for 50ms, and still more for 66ms is much more jarring than the effect of consistently showing all frames for 42ms.

    3. Re:It's in the image by Just+Some+Guy · · Score: 2

      Also, video games tend to have no depth of field. How could they? The whole screen needs to be in focus so that you can look at any part of it. Movies have the luxury of deciding what you should pay attention to and focusing (literally) on that.

      --
      Dewey, what part of this looks like authorities should be involved?
    4. Re:It's in the image by zippthorne · · Score: 2

      Movies don't look smooth. They look like a staccato of motion-blurred still frames. 24fps was simply the minimum (read: cheapest) frame rate at which most of the population would perceive as mostly motion-like. Motion blur helps, but it hardly makes up for the deficiency.

      Technology has advanced quite a bit since the advent of motion picture cameras, to the point that the "film" is pretty far from the most expensive line item in the budget. Why not record at a more natural frame rate?

      The conceit of the movie industry is conditioning he movie watching public to believe that 24fps looks "more cinematic." How convenient for them that it is also "less expensive." But how disappointing, too, when those hard to-obtain establishing shots from high over the countryside don't really show any of the beautiful detail to the viewers?

      --
      Can you be Even More Awesome?!
    5. Re:It's in the image by Forever+Wondering · · Score: 2, Interesting

      The mayonaise you like is the mayonaise you grew up with ...

      Films are shot at 24 fps, but displayed [in theaters] at 48 fps, each frame is displayed twice: f0, black, f0, black, f1, black, f1, black, f2, ...

      According to one study, when test audiences were shown true 1-to-1 48 fps film, they actually preferred the 24 fps.

      The same is true for audio. Those that grew up on 128 kbps .mp3's preferred that over higher fidelity formats.

      The human optic nerve has [surprisingly] low bandwidth. I worked for a company that developed a [now shipping] video product that models the human optic system and removes detail that the human eye would not see. This allows better compression without sacrificing video quality. In A/B testing of original [uncompressed] video sources vs. the detail reduced video, test audiences preferred the detail reduced video. It was considered "cleaner" and "more pleasing".

      --
      Like a good neighbor, fsck is there ...
  3. Re:I RTFA'd, and I still want to know... by Rashkae · · Score: 2

    I don't really know, but I can throw a couple educated guesses from experience. There are two reasons:

    1: Motion Blur. This is even simulated in high end animated movies. (look at a scene in a movie like Shrek of How to Train your dragon, and watch frame by frame where there is motion.

    2. Conistency. 24fps looks ok so long as it is consistent, either because of how the brain receives the image naturally, or just a matter of conditioning since we've been watching movies at 24fps for so long. I know when I watch video that is not properly de-telecined, (29fsp, but every 4 frames there a repeated frame) it immiediately looks very jerky and unwatcheable to me. Video game frame rate tend to swing wildly.

  4. Re:The human eye is proof God exists by binarylarry · · Score: 2, Insightful

    Evolution doesn't push "purpose" or "meaning" it's simply an ongoing discovering of how life exists and changes over time.

    Religious zealots find it to be an intrusion into their belief system and thus automatically try to make it into a religion.

    What we know about the process of evolution is constantly changing and as a science NOTHING we think we know is sacred. Like the Sith, religion is the only thing dealing in absolutes.

    --
    Mod me down, my New Earth Global Warmingist friends!
  5. Movie FPS by meustrus · · Score: 4, Interesting

    why do movies at 48 fps look "video-y," and why do movies at 24 fps look "dreamy" and "cinematic."

    For the same reason children are picky eaters. They say that people have to take three bites of a new flavor to really know if they like or dislike it. I have personally experienced that, going from "wtf this is so wrong" to "ok it's not so bad and I might actually like this" between bite 1 and bite 3. Well, we all grew up consuming 24 fps movies, and anything higher is new and different. Rather than "take three bites", though, so many of us recoil from the different experience and immediately start talking to all our friends about how it looks wrong, concluding that high FPS just looks bad. Try. Three. Bites.

    --
    I sometimes ask revealing, often ignorant-seeming questions. Maybe they're harder to answer than you think.
  6. Re:"Your eyes oscillate"?? by sribe · · Score: 3, Insightful

    What about our eyes is oscillating?

    The whole eye. Our eyes actually cannot detect a static edge, only transitions. The reason we can see non-moving objects is that the oscillations of the eye provide the transitions. There's a simple experiment from long ago which illustrates this vividly: put a black square on a white background, track a subject's eye motion and move that target with the eye motion so that the image is always hitting the retina at the same location, and voila, the subject cannot see that target.

  7. Re:The human eye is proof God exists by jeffmflanagan · · Score: 5, Insightful

    Smart people don't choose to believe things because they want them to be true like you do. An idea doesn't have to give you a warm fuzzy feeling to be true.

    If religious people had any proof, it would no longer be religion. Of course they don't, because the supernatural is imaginary. Too many people fail to grow out of their childhood superstitions, and never develop an evidence-based adult worldview.

  8. Re:The human eye is proof God exists by morgauxo · · Score: 4, Interesting

    That would be Pascal's wager. The problem with it is that an all-knowing God would know the difference between you telling yourself that you believe in him in order to secure your salvation vs you actually believing. So... you go through the motions all your life just to burn in hell anyway.

    I don't know about you but for me belief is a conclusion I come to based on the evidence I know about, not a decision that I make. Anything else would just be lying to myself. The evidence I see and know about overwellmingly supports evolution. If the reality around me is just an illusion planted by Satan to test me or a corruption resulting from the fall or planted by God to test me then I guess I am just screwed because what I see does not match up with any supernatural creation myth I have ever heard of.

    By the way, evolution has nothing to do with purpose, progress or meaning. You have to make that for yourself. Evolution is just change and an explanation of why the change hapens the way it does.

  9. Motion blur is temporal AA by tepples · · Score: 3, Interesting

    There are several ways to apply temporal antialiasing or "motion blur", each of which is analogous to a well-known spatial antialiasing method. One is to render the scene twice at a slight time offset and average the two; this is the temporal counterpart to FSAA. Or find the motion vector around the frontmost mesh in each 8x8 pixel section of the screen and add a local blur filter; this is more like MSAA. But in the march from 240p (PlayStation and Nintendo 64) to 1080p (current consoles) and higher (PC master race), the preference has been for more detail in each frame rather than a better illusion of motion within a frame.

    1. Re:Motion blur is temporal AA by Nemyst · · Score: 2, Informative

      Current motion blur techniques are smarter than you seem to imply, but we're still really far from anything resembling movies. Most implementations use per-pixel blurring (not blocks of a certain size), sometimes across multiple velocities per pixel, but they're still gross approximations which lack information (such as what's behind the frontmost objects). Re-rendering the scene multiple times at different time points, on top of being extremely time-consuming, looks really bad. You need something like 50+ images per frame to create the illusion of smoothness, and at that point you're better off simply presenting 100 frames per second and letting the human eye apply blur.

      Offline techniques use stochastic sampling instead, so you're still effectively re-rendering the image a certain number of times at different time positions, but the difference is that each pixel uses its own set of unique time positions, which trades banding (i.e. being able to clearly see the different time steps) for noise. This sort of technique can be applied in real-time using algorithms like stochastic rasterization, but we're quite far from seeing that used in actual games.

  10. Re:The human eye is proof God exists by Yosho · · Score: 2, Insightful

    The appeal is that it's the truth. People want to discover the truth about how our world works and how we came to exist in it, and science is how we do that -- and every bit of science we have indicates that evolution is the only plausible explanation for the current diversity of life on Earth.

    you're pushing the idea that life has no true purpose and random death means progress

    No, we're not, and I think the whole "you can't have meaning without religion!" bit is one of the most insidious lies the Christian church has pushed. Not following the dogma laid out in 2000-year-old books means you can make your own purpose in life. You can decide what gives you meaning and what you consider progress. I promise that if you go to any Humanist gathering, you will see plenty of people who have meaning and purpose without religion. Talk to them a bit, even -- I'll bet they'll be perfectly friendly if you say, "Hey, I'm a Christian but wanted to see what you guys are all about." Learning first-hand what life without religion is like is better than taking your pastor's word for it (after all, do you think maybe he has his own agenda?).

    I'd rather bet on a .001% chance that Jesus is Lord than 99.999% chance that life is based on nothing but random chance and death.

    Here's the thing, though: there's not a 0.001% chance of that, there's a 0% chance of it. There is zero evidence than any sort of supernatural being exists at all, and it's a huge leap from there to "Christianity is true," with just as much evidence. You are believing it purely because it makes you feel good. That's your choice, but maybe you should find out what the alternative is actually like before dismissing it.

    --
    Karma: Terrifying (mostly affected by atrocities you've committed)
  11. Re:I RTFA'd, and I still want to know... by beelsebob · · Score: 2

    Simple - game frame rates aren't consistent.

    The effect of showing some frames for 16ms and others for 33ms (what will happen if your game is running at somewhere between 30 and 60 fps) is much more jarring to you than the effect of showing all frames for 42ms.

    This is why hitting 60fps (or slightly above) is the magic number at which it all looks smooth again - at that point, all frames are rendered before the screen refreshes, and you get an absolutely smooth 60fps with 16ms frame times across the board. The reason you ideally want to be slightly above 60fps, is because some frames will always takes slightly longer to render than others, you want even those to be rendered before the screen refresh.

  12. Re:Mushrooms (may work with other drugs) by itzly · · Score: 5, Funny

    Unless of course, the mushroom just make you think this is happening.

  13. Re:"Your eyes oscillate"?? by tlhIngan · · Score: 5, Informative

    The whole eye. Our eyes actually cannot detect a static edge, only transitions. The reason we can see non-moving objects is that the oscillations of the eye provide the transitions. There's a simple experiment from long ago which illustrates this vividly: put a black square on a white background, track a subject's eye motion and move that target with the eye motion so that the image is always hitting the retina at the same location, and voila, the subject cannot see that target.

    The other reason is the "sensors" we have are quite poor - the eyeball itself is actually a very low resolution device - the high resolution center part of the eye covers such a narrow field of view that it's practically useless if it was a fixed camera, while the peripheral vision is so low res it's unusable.

    Instead, what happens is we evolved a gigantic amount of wetware to process the image into a high-resolution image we perceive - the brain does a lot of visual processing, and the eyes rapidly move (or oscillate) to move the sharp high-res center vision around to give you a much higher "virtual resolution" than the actual Mk. 1 Eyeball can achieve.

    Of course, this visual processing comes at a price - optical illusions abound because it's very easy to trick the wetware into seeing things that aren't there, because the information is often interpolated, shifted in time, etc.

  14. Re:24 fps are (or at least were) projected at 72 f by wonkey_monkey · · Score: 2

    It's not about flicker, it's about the smoothness of motion.

    --
    systemd is Roko's Basilisk.
  15. Re:Get rid of Frames!!!! by Megane · · Score: 4, Informative

    That's like saying laserdisc is digital "because it's got pits and non-pits". Except that the length of the pits and non-pits is very much analog. (It's a full-bandwidth FM signal driven to maximum overmodulation. VHS does a similar thing.) In other words, the digital-ness becomes analog if you look even closer.

    --
    #naabhaprzrag, #sverubfr-000, #agi-fcbafberq, negvpyr[pynff*=' negvpyr-ary-'] { qvfcynl: abar !vzcbegnag; }
  16. Re:24 fps are (or at least were) projected at 72 f by jones_supa · · Score: 2

    It still good to ponder what flicker could tell us. For example, the 200 Hz LCD backlight PWM frequency causes sore eyes and headaches to many people, so certainly the eyes and/or nervous system are sensing something? Put some white text on a black background on a computer like that and you can see multiple images of the text when rapidly turning eyes horizontally. Of course, as you say, the picture appearing and disappearing is largely a different discussion when compared to motion sensing, but there might be a bridge which can combine these discussions for some benefit.

  17. Re: Get rid of Frames!!!! by kwbauer · · Score: 2

    The poster to whom you replied mentioned recordings and while not exactly correct he is closer than you are about recordings. What he was pointing out is that the recordings he mentioned, analog tape and vinyl records, operate using discrete sampling, the same as digital does. This is because the recoding/reading process of analog tape has limits and so does the groove on vinyl records. Listening to a concert with all acoustic instruments is about the only way to guarantee no discreteness. Even analog amps can introduce discreteness at the edges and even the speakers can introduce a level of discreteness over the compressed air waves coming out a trumpet or of off violin or piano strings. The problem is about getting the level of discreteness below your ability to notice and there is no reason why digital can't do that as well as analog.

  18. Re:The human eye is proof God exists by William+Baric · · Score: 2

    Whether believing in a god is a sign of psychosis or not depends on why you believe.

    If you believe because you want to be part of your cultural group or because you find it useful (either from a social point of view or a personal one), I agree it has nothing to do with psychosis. But if you truly feel there is a god, then obviously your sense of reality is wrong and this is what psychosis is about.

  19. Re:Get rid of Frames!!!! by kwbauer · · Score: 2

    or the analog becomes digital if you turn it around. The point is that the only faithful reproduction of a how a trumpet causes waves of compressed air is a trumpet. No series of microphone -> vacuum tubes -> magnets -> flexible diaphragm will produce exactly the same set of waves as the trumpet (or piano strings or vocal cords, etc.) You get to the point that the difference is not perceivable by whatever instrument you choose to measure with and then call it perfect. A great number of people choose to use their own ears as the instrument with which they measure and when they no longer perceive a difference, then the reproduction may be considered as perfect for them.

  20. Re:The human eye is proof God exists by William+Baric · · Score: 2

    If religion had any kind of proof it would be called "science".

  21. Re:"Your eyes oscillate"?? by Solandri · · Score: 4, Interesting

    Some representations of what our eyes really see.

  22. 48 fps may be TOO clear? by MtViewGuy · · Score: 2

    I think the problem is that because we're so used to 24 fps on theatrical motion pictures, going to 48 fps can be quite jarring, since everything looks so much "clearer" that you have to rethink set design, costume design and even the use of special effects to be less obtrusive at 48 fps. (Indeed, this became a huge issue with Peter Jackson's "Hobbit" trilogy because everything looked TOO clear.)

    The late Roger Ebert liked the 48 fps "Maxivision" analog film format, but that idea never took off due to need to use a lot more physical film and the increased stress of running a film projector at twice the speed of regular projectors. But with modern digital movie cameras, 48 fps is now much more viable.