Slashdot Mirror
Measuring Input Latency In Console Games
The Digital Foundry blog has an article about measuring an important but often nebulous aspect of console gameplay: input lag. Using a video camera and a custom input monitor made by console modder Ben Heck, and after calibrating for display lag, they tested a variety of games to an accuracy of one video frame in order to determine the latency between pressing a button and seeing its effect on the screen. Quoting:
"If a proven methodology can be put into place, games reviewers can better inform their readers, but more importantly developers can benefit in helping to eliminate unwanted lag from their code. ... It's fair to say that players today have become conditioned to what the truly hardcore PC gamers would consider to be almost unacceptably high levels of latency to the point where cloud gaming services such as OnLive and Gaikai rely heavily upon it. The average videogame runs at 30fps, and appears to have an average lag in the region of 133ms. On top of that is additional delay from the display itself, bringing the overall latency to around 166ms. Assuming that the most ultra-PC gaming set-up has a latency less than one third of that, this is good news for cloud gaming in that there's a good 80ms or so window for game video to be transmitted from client to server."
Only in the ports that the PC gets from the consoles (or even ones that happen to be released on both systems) do I notice the horrible latency. It's awful in Oblivion, Fallout 3, Bioshock, and plenty of others. Part of it has to do with V-Sync, but turning that off doesn't eliminate all of it. I can't believe that 133ms is the norm. I've grown up a PC gamer, and that's definitely one of the top reasons I *hate* console FPS games.
...average lag in the region of 133ms. On top of that is additional delay from the display itself, bringing the overall latency to around 166ms.
Considering that until very recently all displays had an inherent lag of about 70ms -- and that new [LCD] technology has pushed that higher. But we're only considering half the equation: The average human response time for auditory or visual input is 160--220ms. This increases as we age. We are also part of this system and we're a helluva lot more lagged than our technology is.
I want an upgrade.
#fuckbeta #iamslashdot #dicemustdie
Anyone can make a comment how the lags affect gameplay on DDR? I still hesitate to buy an LCD TV and stay with my CRT, because I am not sure about it. When playing DDR, I usually listen to the music and the rhythm, so I really don't know exactly what would happen with a LCD TV.
I've seen people playing DDR with Samsung LCD TVs on Youtube. It seems it's working well.
that console players are vegetables. Be a man, buy a pc.
Just as a point of comparison, the typical latency you want in pro audio applications between when a guitarist plucks a string, and when they hear the note, is less than 15ms. This makes me think that the 80ms might be *acceptable*, but it's by no means ideal.
Causation can cause correlation
It may be that console gamers have learned to expect around 100-150ms of input latency, perhaps thanks to visual cues that help to justify the latency on some level. (If I decide to jump, it takes a certain amount of time to react to my thought and make that happen; if I tell Mario to jump, maybe he takes about the same amount of time to react to the stimulus. It makes a certain kind of sense.)
But I assure you that musicians find that level of latency unacceptable. When you're playing a software synth live, performing with other musicians, even 75ms of latency is very noticeable and makes you feel like you're playing through molasses. Same thing with recording -- if it takes longer than 25-30ms to hear my own sound coming back at me, I definitely notice it. Virtuosic music regularly exceeds an input density of 50ms per event!
One thing the article fails to mention is if the weapons he's firing are supposed to be instant. There could be a "startup" time programmed into the weapons. Think of it like a fighting game. The slower heavier attacks have a longer windup than the weaker attacks. If the weapon firing is supposed to be instant than this was a very good test.
It could just be the TV. Playing "Wii Sports/Tennis" at my friends house is awful for me because his TV introduces a 100-200ms video enhancement processing delay. It also lags the sound to compensate. He's used to it so he plays OK, but it takes me 4-5 rounds to re-train myself.
This is once reason recent TVs have a "gaming mode" that removes this delay by eliminating any post-processing.
Something I never understood in arcade games was the "shooting pause", where a maximum amount of button presses (usually associated with the fire button) yielded a maximum number of projectiles before failing to register additional button presses. A noticable break in visual projectiles was observed before the additional button presses are again registered. I can't think of a single game I've ever played where the shooting pause increased dramatic tension, added to the atmosphere of enjoyment or balanced the playability. I've always suspected the shooting pause must be a deliberate software coded effect rather than a hardware limitation.
On the old Atari 2600, the game has to be written around rendering fields (half frames) of video. On NTSC, that is 59.94 fields per second, or a little under 16.7ms. Input is usually read during vertical blanking between fields. That makes for not much more than 33.3ms latency in the worst case of input change just after vertical blanking.
Maybe new isn't really better.
"Luncheon meats make the sawdust in your stomach explode."
Kernel developers have complained that UI latency doesn't have very good measures under Linux. Now here's a methodology for measuring it. This could lead to kernels better optimized for the user experience that were provably so.
I don't think though, for the Linux kernel or for a video game, that pure latency is exactly the right measure. I think the standard deviation of latency is an important measure too. A user should be able to reliably predict the latency. They may not consciously do so, but their cerebellum will.
Need a Python, C++, Unix, Linux develop
I don't know what caused it, but my normal Call of Duty: World at War server that i used to ping 50 to just jumped to about 120 on average. It really messed with my accuracy for a few rounds until i adjusted. I didn't think it would make a difference but it definitely did.
OK, I'll be the first to concede that I am more sensitive (or attentive) to lag issues, being an audio/video hack myself, but how can 4+ frames of lag be ignored or even tolerated in any action game ?
I already consider the 3-frame LCD lag inacceptable and utterly shameful.. I mean the data is there, put it up already! If the de-crapifying filters need that much lookahead to function, they need to be refactored to use look-behind, and if the copycat engineers can't fix it, at least give an option to disable it per-port so we can play our games.
Now on the development side, as a so-so game dev myself, I can't think of any valid excuse for Killzone's 12 frames of lag. What the hell are they doing in the loop ? Here's what a game loop is supposed to look like :
for (;;)
{
if(button_pushed(1) && ga_hasammo(ga_PEW_PEW))
{
ga_plWeapon::spawn_bullet();
}
render_scene();
}
Notice the lack of "sleep(9000)" statements ? So that's what, 20 usec worth of code ? Take input, spawn bullet, play sound and draw the goddamned frame already! If that takes you 200 msec to process, then your game is really running at 5 fps with a shit ton of interpolated frames in-between, and you should probably go back to writing Joomla plugins.
Ten years ago, this shit would not have flown. We used to tweak the everloving crap out of our loops, and VSYNC was the norm, which made late frames painfully obvious. To deal with it, we used hard-timed loops and every single piece of code had to obey the almighty strobe. You had 16 or 33ms to render your frame, and if that wasn't enough well, you had to tweak your code. Today, now that even game consoles have gone multicore, there is no excuse. You could even have one thread acting as a clock watcher, monitoring the other tasks and telling them to hustle (e.g. degrade) if they're falling behind.
To prioritize anything else is to betray the game's purpose: to entertain via interactivity. If a game is going to sacrifice interactivity, I might as well go watch Mythbusters instead :P
-Billco, Fnarg.com
After my first couple experiences gaming on the XBox, I'm still convinced it has a latency of about 2 seconds. Which is sure to have been improved down from the beta of 3 seconds by the excellent testing/QA department over at Microsoft, which I am sure definitely exists.
From what I can tell, the Xbox is generally worse for lag. I own both systems, and COD4 definitely has more lag on the Xbox version.
What TV people has is far more a factor than the game. Most LCD panels are horrible, and don't come close to a even a budget plasma, and plasmas don't come close to CRTs in terms of lag.
At the Hot Chips symposium last month, Rich Hilleman, Creative Director for Electronic Arts, commented on the 100ms delay inherent in the Wii remote (Wiimote). I assumed there was an issue in the delay involved in sensing the accelerometers, but this article shows 100ms is not any different from other consoles.
I wonder what Rich Hilleman was really getting at? Maybe people are more sensitive to delays when they are a result of a full-body-type movements rather than a button-press.
This is interesting stuff, and it would be a good thing if some graduate student did a thesis on it. (Free Ph.D. here--no thinking required!)
-Todd
Omne ignotum pro magnifico.
Ignoring the flamebait (only one good LCD? Really? I'm pretty happy with my Philips), I must say that as someone who has been a PC gamer most of his conscious life, I'm pretty impressed with what consoles had (and still have) to offer. When I was introduced to the Zeldas for the N64 there were so many things going through my head at once that I couldn't tell what the first impression was, but I'm pretty sure it wasn't "unplayable". Neither did I notice severe amounts of lag, but even if it's true that console games have more lag than PC games (and I think the article is making an unfair comparison by setting consoles, which are based off budget PC's nowadays, next to performance you can only get theoretically with a top of the line PC no one can afford) that doesn't necessarily mean that console games suck. There are more factors in the equation, not the least of which is new, fun, often somewhat original games that we haven't seen before yet. Katamari Damacy, Ridge Racer (actually arcade at first), Okami, Little Big Planet, or the legendary console RPGs from the SNES era, like Tales Of and Seiken Densetsu (among many many others), DDR, just to name a few. Rant on about latency all day long, but that doesn't change the fact that they're rock solid games and that PC game designers could learn something from them.
Often the real problem players have isn't the latency itself, because our brains will accommodate almost any lag as long as it's uniform(witness the lack of "frames" for most movies, despite being (usually) at a mere 24fps). What causes the problem is actually when you have more than one set of stimuli that are going at different rates. This is most noticeable with audio and video not being in sync.
With an LCD display, this is magnified greatly unless you are going directly from the computer or machine to speakers with their own amplifier built in(or headphones).
Case in point - I like to play Rockband with my son. On a CRT, it was fine. On a LCD I had to set the audio lag to 0ms. THEN set the video to sync with that. Adding delay to the audio as well as the video made it impossible to get a decent result - one has to be set to zero.
eg: audio lag is tested at 20ms. Video lag is 35ms. The correct settings are 0 and 15, because the audio will always have that delay in it no matter what you do. Putting both at the recommended 20 and 35 yields a combined 55ms between your finger and the result. Though they are in sync, they feel "laggy" as our brains are used to video running at about 60fps continuous/30fps interlaced if we grew up in the era of CRT displays. So 55ms feels like we just dropped 3 frames versus one in this situation. And that's just about at the threshold of delay between what we hear and what we see where it starts to become annoying.
Note - it's also why you need a $20 dedicated sound card. Often games hammer the CPU and on-board audio chip set when a big group of sounds come in and that also causes lag in Direct-X games for a moment(which most all console ports are, exacerbating the issue).
One video frame? With a normal camera? That's 1000/30 = 33.333... ms. From making music, I know when you start to notice lag, and some people can notice this at around 10 ms, and I get into trouble above 30 ms. So you would have to have at least the double temporal resolution, to get useful results.
Any sufficiently advanced intelligence is indistinguishable from stupidity.
Reading TFA would have told you the reason why the PS3 wasn't tested at this time.
Idiot.
Seeing lots of comments about LCD screens - i'm thinking of upgrading from my old 17" to something around 21-24" LCD widescreen.
Am a gamer (not completely hardcore though) - so response time would be good. Am aware of the refresh issues with LCD's. Also do some photography stuff so good colour reproduction would be handy (after calibrated etc), but viewing angle not so important.
Any ideas? Looked around for reviews and found a few conflicting reports - suggestions much appreciated! Budget is low to mid (in dollars guessing $180-$250).
Two so far on short list are either:
Viewsonic VX2260WM 22"
Asus 24" VW246H
The amount of latency is not really an issue as much as the consistency of latency. There's nothing more frustrating than getting fragged because YOUR input was processed late because of too much going on, or for any other reason. I recall missing tons of jumps in Megaman 2 because of this, so it's hardly a new problem.
stuff |
Human lag time mostly doesn't matter. The brain compensates for it. For example, when you catch a ball, your brain knows how it needs to correct for the fact that what it sees is what was some time ago, and what it tells the muscles to do won't happen for some time, and it's done this your entire life so it has a pretty good idea exactly how long those delays will be.
Toss in an additional 133 ms, and you've totally fucked it all up. The brain tries to calcuate responses just as it always has, yet the calculations no longer give the correct answers, and so your performance sucks.
With some time, the brain will learn, and your performance will improve, but it will never be as good as it could be, since the brain can only compensate for lag when it is dealing with predictable events. When someone throws a ball to you from 20 feet away, your brain gets a lot of data to work with to predict where your hand needs to be to catch it, but when they throw a ball from only three feet away, the ball hits you before the brain even knows it has been thrown. Increasing the lag time allows it to be thrown from further away and still hit you before you know anything has happened.
That's the problem with lag time in games. If the lag is zero, then the game approximates reality, and the player doesn't perceive any problems. If the lag is too great, then the player must learn to compensate for that lag to accomplish even the simplest time-coordinated task, and even then won't do as well because of lack of experience with that latency. Also, the player can never compensate for the increased reaction time.
I discovered all of this years ago while playing Descent. After installing a patch which allowed different screen resolutions, I discovered that the lower the resolution, the better I could play the game. Playing in 160x100 positively looked like shit, but it ran at 70 frames per second, and so the latency was much better than at 640x480 where it ran at 15 frames per second. It's much easier to hit targets when you get to use the latency information your brain has relied on your entire life rather than the information it has only known for the relatively short time you have played a particular game.
Lowering latency is hard.
It wasn't till I started programming simple music software that I realised how hard.
In some cases, you need a different algorithmic approach as latencies get lower! (serial convolution is not obvious!)
Computational overhead goes up exponentially too when you get to the sub 2ms or so level.
(Though a lot of this is due to personal computers busses not being designed for it.)
Strangely enough though it was possible to get *single sample* latency with some soundcards on the old ISA bus!
(Though the computer won't be doing much else. other than serving interrupts:)
As a musician myself, I find most games to feel soupy and disconnected from the input.
I don't remember doom under dos feeling like that though. It would be interesting to see what that latency of dos doom actually was!
Most modern console games process graphics stuff in parallel with engine updates. So on a given frame, it moves the entities around in the engine (simulating physics, applying animation, etc). On the next frame, the graphics code renders the entity in that new position. Then there are another 1-3 frames of buffering due to CPU-GPU communication, triple buffering, and hardware output lag (the number of frames depends on how the developers configure things).
For a game running at 60 fps, 4-5 frames of delay is a still-tolerable 80-95 ms. Other than racing games and fighting games (and a few shooters like CoD 4), most console games run at 30 fps, where 4 frames of lag is already around 125 ms.
The fighting game community has been doing this for a while now, and no high-tech Ben Heck controller was needed (an LED gives a pretty good visual indication).
See NKI's videos documented here:
http://forums.shoryuken.com/showthread.php?t=144797
http://nki.combovideos.com/
For arcade-style fighting and shoot 'em up games, lag is a killer. Flat screen TVs and upscaling lag have made things worse in recent years, which is why many of us still play on CRTs. New 3D game engine developers seem to care even less, as is evident by discussions like these appearing all over gaming forums worldwide.
Just some thoughts from research I've done that used or at least looked at reactions and reaction times. If game makers are already thinking about these things, good for them. If not, got an opening for a cognitive psychologist in game design?
As noted, reaction times are greater than your response lags. A good human reaction time is around a third of a second. If your lag times are cut according to the refresh rate, a person's reactions could get placed with an earlier or a later frame. But because perception is faster than reaction, the difference could be noticeable making the play seem uneven.
Human response times vary from 200 to 600 msec. The average isn't that different. What varies far more is the variance. Some people are very consistent, and these are usually the ones with a faster average. Some people vary across a wide range, and while they may sometimes be as fast as the fastest reacters, they sometimes have very slow reactions. This can vary from one response to the next. The delay is most often due to something distracting even if it's irrelevant to the reaction. Placing distracters in a game on purpose could serve to make the game better, Having them by accident wouldn't.
Besides reaction time, there are errors of omission and commission. These occur most frequently after a distraction. They also occur with a slower reaction time because the person has to shift some from automatic processing to a go/no-go decision. Again, placement of distracters should be taken into consideration -- do you want them there? Fine. If not?
Errors also occur due to persistence. Several of the same reaction in a row will tend to cause that same reaction. You can use this in design, You can also realize where it is by accident and how that might adversely affect play and so the player's opinions. "Tricking" them into an error can be a challenge or a pisser, depending on context.
I'm not kidding about the job; there's a whole lot more than reactions and reaction times that we do in the lab and you could use or eliminate if need be. I'm not kidding about being able to put together an EEG based play system that lets a person react so fast that they see the result before they realized they decided what to do, either. I played a Brickout game using one, and it's spooky, but cool.
"I may be synthetic, but I'm not stupid." -- Bishop 341-B
An LCD (or preferably OLED) monitor + source running at 100fps, or even better, 200fps would mean no more flicker, super smooth video, and almost no input lag, and in the case of OLED, longer display lifetime (because less voltage is needed since the pixels' duty cycle can be higher). It's a win all round.
Let's all switch already. Okay, recorded video data will be 2-4 times bigger, but it'll be so worth it.
Why OpalCalc is the best Windows calc
So assuming this is even correct, which is odd considering Carmack being concerned over a single frame of lag for some thing in RAGE (60fps) then it still means most modern game would never be playable over cloud computing, since they're already too laggy. In fact this seems to suggest just the opposite, that the entire industry would have to shift towards significantly lowering input lag just to get the already bad response times we have now. In fact, assuming a good response time for input going across, rendering/game logic, and then output coming back of say 60ms then games that react on precise reaction times would NEVER be playable over cloud computing, at least not until we get to very very low input lag (30ms or less) AND very low connection latency (30ms or less).
All considered I don't doubt that mainstream cloud computed gaming will be a decade or more off.
You're wrong.
20 milliseconds is 22 feet - that's quite a distance, and yes, it would be difficult to play with someone that far away.
Advanced users are users too!
Does anyone know what the difference in people's reaction times are between something they haven't practiced, and something they've practiced to death? I see numbers quoted like 200ms, which seems about right if you sit someone down for 5 minutes and tell them to press the button as soon as they hear the beep. But I'd expect a fair increase in their reaction time if you had them practicing that test for 3 hours a day, every day for a month as their response turns into reflex. And more serious gamers that play in tournaments or even professional gaming should easily reach that point. So I'm not sure how these numbers compare once you take into account someone who has had a large amount of practice at whatever particular actions.
I've only really got that sort of experience with one old game which I've been playing for about 8 years now - it's an extremely input heavy game played with just the keyboard, about 10 key presses a second (half of them the spacebar) would probably be about average. When I play that game I don't even think about what buttons I press and if I tried I'd fail miserably. Everything gets turned into goals and 'moves' and my hands do the translating. But from my experience, as you start to get experienced enough that you're stringing together long sequences of actions and timing things down to the frame, input lag makes a difference. What I'm saying is that where input lag might be reasonable compared to reaction times for some casual gamer, when it comes to the high end competitive gaming and people have practised so much that everything has a learned response without even thinking about it, input lag needs to be a lot lower.
The other thing I've noticed is that using sound cues seems to be far more effective than visual cues - maybe it's something to do with the brain being able to use audio input for timing based actions better, or maybe the higher sample rate for sound gives more to go off than 30 or 60 fps. Anyway, it'd be interesting to see how people's reactions compare when they're based off visual and sound cues - it could be valuable information for game designers.
Street Fighter II for the SNES, I videotaped a play session because I was suspicious. I then stepped through it frame-by-frame.
Usually, when both me and the computer player were defending, and I started an attack... the opponents counter-attack animation would begin before my attack animation started...
That is, the game artifically lagged in order to increase the difficulty beyond what a human opponent could provide. (aka, it cheated)
normally, you hold the instrument you are playing in your own hands.
the latency is pretty low.
The extreme case for musician's latency is for church organ, as the pipes are sometimes far from the remote console.
sometimes as much as half a second away!
It takes a great deal of skill to be able to play like that, but it's not something casual gamers should have to deal with!
> well you obviously don't, because I don't see how you can play with other musicians that are 10 feet away. that would equal 20 ms of latency.
Think about it a lot harder before saying that people are full of shit.
There's a huge difference between hearing yourself in order to play and hearing others in order to play along with them.
People can easily play along with other musicians even if the sound from them took 20 decades to arrive (e.g. 20 decades of latency). They put on a CD, then play along with it.
But it's harder if your instrument takes 20 decades to make a sound after you do something to it. And in fact it's still easier than if your instrument takes just a second longer to make a sound - since it's a greater distraction than not hearing any sound and assuming you are playing "deaf" or without sound.
If you still can't understand the difference, too bad for you.
Anyone who played TF1 all played at 600ms+ on their 14.4k modem. We grapple hook just fine. Console noobs have it easy. Wow I sound like him
What would be acceptable methodology there? Do cats play console games? is the button actually pressed? Does it press itself? Have you checked the power cable?
The new right fascists are bilingual. They speak English and Bullshit.
I would LOVE to see this test being done on the NES running SMB. One of the reasons SMB is an all-time success is the responsiveness of the controls. After all it is really a game, and test, of timing.
- Witticism is an epitaph on the death of a feeling
There are so many types of game where this kind of lag doesn't matter, or can be compensated for.
Starting with the obvious ones: anything turn based is unaffected. Input could get very sluggish indeed before it broke a game like Civ or XBLA Carcassone. Battles in RPGs like Final Fantasy are the same.
Even a lot of action games don't depend on instant responses. Yes, something like Quake III is all about twitching. But something like Bioshock has a much more measured style, which is *not* ruined by a three frame lag.
Music software really is ruined by input latency. If the software doesn't know what you're going to play, the only solution is to remove the lag. However, as Guitar Hero's lag compensation demonstrates - software can compensate for the lag. I think it works like this:
- Transmit the video early
- Play the audio that *would* play if the player hits the note
- If the player hits the note, carry on, else switch to the failure sound.
Other games can and do compensate for lag in a similar way. For example, a racing game can steer the way it thinks you will steer, then adjust to where you actually steered when it finds out (late). People like OnLive put money into researching this kind of thing - what hints you can feed people in the picture, so that they're fooled into not seeing the lag.
Actually, I said that 25ms (not 25 seconds, as you incorrectly wrote) of latency was basically acceptable, but that 75ms of latency wasn't. As someone else pointed out, 20ms of latency is equivalent to having the musician and the sound-source be 22ft apart. If you're a guitarist, that means being about 22' away from your amp. That's a fair distance, but it's doable.
75ms of latency is the equivalent of being about 84' away from your amp. Most guitarists would find that disconcerting, and it makes it hard to keep a steady tempo even in simple music, since you would need to play well ahead of the beat. At 120bpm, i.e. a medium tempo, 75ms of latency means that you need to play 15% ahead of the beat to stay in time. At a faster tempo of 180bpm, that approaches 25%, or the equivalent of a full sixteenth note in 4/4 time.
This is also why musicians really need stage monitors pointed at them, especially people who don't necessarily use their own amps (singers, keyboard players, etc.). Even if the PA mix is great, there's still enough physical distance and propagation delay that it'll throw off someone who's solely relying on the house system.