Measuring Input Latency In Console Games

The Digital Foundry blog has an article about measuring an important but often nebulous aspect of console gameplay: input lag. Using a video camera and a custom input monitor made by console modder Ben Heck, and after calibrating for display lag, they tested a variety of games to an accuracy of one video frame in order to determine the latency between pressing a button and seeing its effect on the screen. Quoting: "If a proven methodology can be put into place, games reviewers can better inform their readers, but more importantly developers can benefit in helping to eliminate unwanted lag from their code. ... It's fair to say that players today have become conditioned to what the truly hardcore PC gamers would consider to be almost unacceptably high levels of latency to the point where cloud gaming services such as OnLive and Gaikai rely heavily upon it. The average videogame runs at 30fps, and appears to have an average lag in the region of 133ms. On top of that is additional delay from the display itself, bringing the overall latency to around 166ms. Assuming that the most ultra-PC gaming set-up has a latency less than one third of that, this is good news for cloud gaming in that there's a good 80ms or so window for game video to be transmitted from client to server."

Comments

Transfers to PC Game Ports too...

[TheSambassador]

Only in the ports that the PC gets from the consoles (or even ones that happen to be released on both systems) do I notice the horrible latency. It's awful in Oblivion, Fallout 3, Bioshock, and plenty of others. Part of it has to do with V-Sync, but turning that off doesn't eliminate all of it. I can't believe that 133ms is the norm. I've grown up a PC gamer, and that's definitely one of the top reasons I *hate* console FPS games.

Re:Transfers to PC Game Ports too...

[KulSeran]

1) Input is often sampled only once per frame. That is why quake at 120fps feels more responsive, the time between you pressing a button and the game noticing you pressed the button is reduced.

2) Input and actions are often determined on a per-frame basis. Meaning the fastest delay you can get is a single frame. Consoles tend to have games that run at a target frame rate (30, 24, 60) that determines how much visual flavor the game can have (60hz leaves less time to draw and update stuff than 30hz). So, at 30Hz, the fastest you can hope to see an action is 1 frame after the game detects it. That amounts to 33ms-66ms depending on your timing of the press in relation to the frame processing (we are asynchronous to the technology after all).

3) After a game renders a frame, it is usually buffered, so it has to swap and display the buffer. With V-Sync (consoles tend to v-sync automatically) that means it has to wait the 16ms for a 60Hz screen to refresh. But it only attempts to swap once a frame, introducing an aditional frames worth of delay in the display.

This is where the 99ms minimum response for a 30FPS game came from. 48ms for a 60FPS game.

Then you have to take into account that because of threading and networking, there are often more buffers in the game that only swap once a frame. This can introduce additional frames worth of delays as developers attempt to use the hardware to its limits.

Reality check

[girlintraining]

...average lag in the region of 133ms. On top of that is additional delay from the display itself, bringing the overall latency to around 166ms.

Considering that until very recently all displays had an inherent lag of about 70ms -- and that new [LCD] technology has pushed that higher. But we're only considering half the equation: The average human response time for auditory or visual input is 160--220ms. This increases as we age. We are also part of this system and we're a helluva lot more lagged than our technology is.

I want an upgrade.

Re:Reality check

[Mprx]

The only inherent display latency of a CRT is the time taken for the beam to arrive at any particular part of the screen. In the worst case this is one frame, which at a reasonable refresh rate (100Hz+) will be only 10ms or less. A good LCD (there's only one on the market, the ViewSonic VX2268wm) updates in the same line by line fashion as a CRT, and will add only a few more milliseconds switching time latency.

Of course you still have the latency in the input/processing/rendering stages, but this doesn't have to be very high (increase input sampling rate, avoid any interpolation, disable graphics buffering, etc). The only reason most modern console games are unplayable is because reviewers all ignore latency, and low latency can be traded for higher graphics detail which the reviewers pay attention to.

Perceived latency has nothing to do with reaction time.

Re:Reality check

[sahonen]

Actually, raw reaction time, which doesn't even change too much between 20 and 30, is not the primary element of skill at first person shooters. I've looked at the raw reaction time (i.e. click your mouse when you see a light turn on) of many gamers, some who absolutely dominate me and some at or below my level, and there was no real correlation between that reaction time and skill. From what I've gathered, I've determined that skill at FPS games is more a function of experience and training rather than raw reaction time.

The basic categories that set an elite gamer apart from an average or newbie gamer go something like this:

Predicting your opponent and being unpredictable yourself: Knowing where your opponent is going to be, and acting in a manner that your opponent can't predict. If you can put your crosshair where you know your enemy is going to be, and he can't do the same, you're going to win even if he has better raw reaction time than you. This is a function of experience with the game.

Decision making: Evaluating the importance of the various high-level goals in the game, deciding which ones to prioritize, and acting on that decision. Making better decisions, making them faster. Again, a function of experience with the game.

Aiming skill: If an enemy appears on your screen away from your crosshair, how quickly and accurately you can move your mouse to put the crosshair over him. This is a function of training, learning exactly how much mouse movement corresponds to how much movement on screen, and being able to precisely produce that movement with your hand. This is often confused for reaction time when watching people play, but really, the reaction time component is only in seeing the enemy and deciding to shoot him. The rest is muscle memory.

This is where input lag really hurts, it's very very important that your field of view appears to correspond to your mouse movements with absolutely no lag. Console games don't suffer from this because aiming with console controllers is far less precise than using a mouse, so the input lag "hides" behind the imprecision of the joystick. When the game meets the PC where people are using mice, the lag between moving your mouse and your on screen view changing becomes perceptible.

Movement skill: The ability to manipulate your controls to allow you to travel faster. Not just finding the most efficient routes, but being able to use quirks in the game's movement code to give yourself more velocity. Another function of training, getting the control inputs just right can be difficult to master.

Teamwork: In team-based games, communication, chemistry, planning, and effective group decision making.

DDR?

[koinu]

Anyone can make a comment how the lags affect gameplay on DDR? I still hesitate to buy an LCD TV and stay with my CRT, because I am not sure about it. When playing DDR, I usually listen to the music and the rhythm, so I really don't know exactly what would happen with a LCD TV.

I've seen people playing DDR with Samsung LCD TVs on Youtube. It seems it's working well.

Re:DDR?

[bjorniac]

DDR or any rhythm/timing based game will be perfectly fine with a fair amount of lag so long as the lag is consistent. The game isn't based much on reaction times, more hitting the pads at the right intervals. Once you get accustomed to the lag (which should happen naturally as you dance) the actual amount won't matter so much - you just have to move 160ms before the arrow hits the circle or whatever, something you will have been doing already, moving to land on the beat, rather than waiting for the beat and then moving. This differs from, say, a shooter like counter-strike, where you have to react as fast as possible to what is a non-rhythmic, supposedly non-predictable event (unless the opposing team comes out in synchronized swimming formation).

Inconsistency in lag would be a killer here, as it is everywhere, as it would be essentially adding a random component to your timing that you have no control over. But any time you do rhythmic work you're doing predictable lag compensation already - eg clapping on the beat requires you to start the motion before the beat happens rather than react to it.

Re:DDR?

[Anpheus]

One thing Rock Band has done, and presumably this came from somewhere else or has propagated to Guitar Hero and other rhythm games, is that you can set the video latency and audio latencies separately and finely tune the system so that it looks and sounds like you want it to be.

Rock Band 2's guitar controller actually has a tiny light sensitive component and a cheap microphone, so that you can auto-set your game. It's really very handy, and took only fifteen seconds or so. The result was that when a note crosses the "active line" of the game is when I should both strum it / hit it / sing it and hear the result.

Are you certain there is no way to do the same thing with DDR?

Re:DDR?

[mwvdlee]

I'm sorry, perhaps I'm misunderstanding you, but in the world of music 500 BPM is far from "low". Most "danceable" music generally is somewhere between 120 and 130 BPM, Drum-and-bass (which most people would consider quite fast) is about 170-180 BPM. Finding anything over 200 BPM is uncommon and usually for novelty sake. Perhaps the measurement you're talking about is something else than Beats Per Minute?

Musicians can detect very small amounts of latency

[silverspell]

It may be that console gamers have learned to expect around 100-150ms of input latency, perhaps thanks to visual cues that help to justify the latency on some level. (If I decide to jump, it takes a certain amount of time to react to my thought and make that happen; if I tell Mario to jump, maybe he takes about the same amount of time to react to the stimulus. It makes a certain kind of sense.)

But I assure you that musicians find that level of latency unacceptable. When you're playing a software synth live, performing with other musicians, even 75ms of latency is very noticeable and makes you feel like you're playing through molasses. Same thing with recording -- if it takes longer than 25-30ms to hear my own sound coming back at me, I definitely notice it. Virtuosic music regularly exceeds an input density of 50ms per event!

Atari 2600 has less latency

[Visoblast]

On the old Atari 2600, the game has to be written around rendering fields (half frames) of video. On NTSC, that is 59.94 fields per second, or a little under 16.7ms. Input is usually read during vertical blanking between fields. That makes for not much more than 33.3ms latency in the worst case of input change just after vertical blanking.

Maybe new isn't really better.

Re:Shooting Pause...

[Jeek Elemental]

I think what you see is simply hitting the max number of inflight bullets? Software limited yes but probably based on what the hardware can handle.
If the game uses hardware sprites (quite possible) it may be limited by the total number of sprites on screen.

So when you hit this max number you wont be able to fire any "new" bullets until an old one hits something or goes offscreen.

How can they miss this ?

[billcopc]

OK, I'll be the first to concede that I am more sensitive (or attentive) to lag issues, being an audio/video hack myself, but how can 4+ frames of lag be ignored or even tolerated in any action game ?

I already consider the 3-frame LCD lag inacceptable and utterly shameful.. I mean the data is there, put it up already! If the de-crapifying filters need that much lookahead to function, they need to be refactored to use look-behind, and if the copycat engineers can't fix it, at least give an option to disable it per-port so we can play our games.

Now on the development side, as a so-so game dev myself, I can't think of any valid excuse for Killzone's 12 frames of lag. What the hell are they doing in the loop ? Here's what a game loop is supposed to look like :

for (;;)
{
    if(button_pushed(1) && ga_hasammo(ga_PEW_PEW))
    {
        ga_plWeapon::spawn_bullet(); /* MOTHERFUCKING PEW PEW!!!1!! */
    }

    render_scene();
}

Notice the lack of "sleep(9000)" statements ? So that's what, 20 usec worth of code ? Take input, spawn bullet, play sound and draw the goddamned frame already! If that takes you 200 msec to process, then your game is really running at 5 fps with a shit ton of interpolated frames in-between, and you should probably go back to writing Joomla plugins.

Ten years ago, this shit would not have flown. We used to tweak the everloving crap out of our loops, and VSYNC was the norm, which made late frames painfully obvious. To deal with it, we used hard-timed loops and every single piece of code had to obey the almighty strobe. You had 16 or 33ms to render your frame, and if that wasn't enough well, you had to tweak your code. Today, now that even game consoles have gone multicore, there is no excuse. You could even have one thread acting as a clock watcher, monitoring the other tasks and telling them to hustle (e.g. degrade) if they're falling behind.

To prioritize anything else is to betray the game's purpose: to entertain via interactivity. If a game is going to sacrifice interactivity, I might as well go watch Mythbusters instead :P