The Insidious Creep of Latency Hell

Twinbee writes "Gamers often find 'input lag' annoying, but over the years, delay has crept into many other gadgets with equally painful results. Something as simple as mobile communication or changing TV channels can suffer. Software too is far from innocent (Java or Visual Studio 2010 anyone?), and even the desktop itself is riddled with 'invisible' latencies which can frustrate users (take the new Launcher bar in Ubuntu 11 for example). More worryingly, Bufferbloat is a problem that plagues the internet, but has only recently hit the news. Half of the problem is that it's often difficult to pin down unless you look out for it. As Mick West pointed out: 'Players, and sometimes even designers, cannot always put into words what they feel is wrong with a particular game's controls ... Or they might not be able to tell you anything, and simply say the game sucked, without really understanding why it sucked.'"

Comments

I noticed this

[GeorgeMonroy]

I thought things were supposed to get faster with newer technology but it does indeed look like the newer devices run slower becuase of bloated apps and firmware. Maybe this has to do with programming being "bestshored" =D nowadays.

Re:I noticed this

[man_of_mr_e]

It's fine to write lean and high performant code in a small application with a single developer. Once you have teams of people working on something, then you have to worry about something called "Maintainability". According to various studies, 80% of a piece of softwares total lifetime cost is maintenance and support costs. Nobody wants that to be come 90 or 95 or even 99% because people wrote code that others coudn't understand very easily.

Standardized frameworks, patterns, and components improve codes maintainability, often times at the cost of decreased performance. The idea is that hardware gets faster over time anyways, so you can write more features in less time with lower support costs.

Re:I noticed this

[Nerdfest]

For business development (not tools like FireFox, etc) it's not about being 'as fast as possible', but rather 'as maintainable as possible' while trying to be fast enough. When you write code to wring every clock cycle out of a CPU, the code tends to be difficult to maintain. Sometimes you need to do this, but in general you don't. People still write absolute crap in both situations of course.

Re:I noticed this

[cpu6502]

I'm not buying those excuses.

Why is it Microsoft Word 97 fits into my 8 megabyte 386 laptop, and has 99% of the same functions as modern Word, plus is quick and responsive. Why can't they bring that level of efficiency for today's Word 2010?

Because they aren't trying.
Because they don't care.
Because it's easier for management to tell users, "Go buy a new computer with 8x more RAM," than to pay programmers to make the code more efficient/responsive.

Re:I noticed this

[Dishevel]

I'm not buying those excuses.

Why is it Microsoft Word 97 fits into my 8 megabyte 386 laptop, and has 99% of the same functions as modern Word, plus is quick and responsive

99% of what you use.
95% of what most people use.

Lots of code for shit no one ever uses.

They call them new features.

If they did not put in "New Features" why would you pay to upgrade?

Re:I noticed this

[pankajmay]

This issue is exactly why it is worth paying attention in those academic classes that are frequently labelled as out-of-touch with real world. (refer -- countless /. news articles on this).
Introduction to Algorithms, Data structures and advanced algorithms; and the math behind them. Unfortunately since the barrier to entry is so low in today's environment (Hey look I can code --- that is enough) and that coupled with the quest to recruit brute force programmers and not thinkers have led to this mess today where even though we have faster machines, the end result is actually a degraded experience.
Couple this with (to some extent artificially created by programming environments...) trade-off between maintainability and performance, and the situation gets even worse.
And of course combine this with legacy code -- even when written with best intentions, might have faster solutions today OR the legacy code worked well for a few hundred datasets, but you have millions of them today -- scalability issues and the reluctance to leave behind what works.

Finally -- in today's world -- we have an explosion of data -- it's data, data everywhere, but a lot of our tools and programming constructs still have an antiquated view of data -- processing it serially chunk by chunk.
So, it is a lot of factors that have come together to create this issue. The resolution: Computer Scientists need to go back to basics and build concepts of data storage, access, and processing from scratch. And the realization that there is a difference between people who can code, and people who can look at a solution holistically.
For example -- not every representation of a solution of a problem captures the solution space well. Choosing a solution representation as opposed to a naiive approach can make orders of magnitudes difference between hitting the target within well defined bounds, versus having unbounded solutions that never approach a significantly better phase.

Re:I noticed this

[Gilmoure]

If Word 5.1a ran ok under OS X, yeah, I wouldn't have shelled out my $20 for Office 2011 last month.

Hell, it might be faster to launch a Mac OS vm with Word 5.1a on it than Word 14/2011 to launch.

Re:I noticed this

[man_of_mr_e]

Wow, such a well thought out and reasoned argument. I especially like your empirical facts. Yes, that's sarcasm.

I recall a presentation made by Microsoft that suggested that almost twice as many new features were added to Word and Excel between Word 97 and Word 2007. The large number of new features was one of the reasons that Microsoft chose to abandon standard menus, as standard menus just didn't scale.

This isn't quite the presentation I remember, but it has some good statistics on slides 27 and 28.

http://www.90percentofeverything.com/2008/03/20/the-story-of-the-microsoft-office-ribbon-by-jensen-harris/

Word 97 had 200 Menu items, and office 2003 had almost 300 (not sure how many items 2007 and 2010 have, but likely a lot more). While also adding almost 20 Task Panes and nearly almost 20 toolbars. That's toolbars, not buttons on the toolbars.

So, sorry.. No, Word 2010 does not have 99% of the features that Word 97 had. Not even close.

Re:I noticed this

[icebraining]

Firefox's model enables this, and I find it a fine reason to keep it that way.

Re:I noticed this

[idle12]

99% of the feature the "common man" uses.

Heck, I could get away with using WordPad almost excursively.
Throw in a spell checker and format converter and you meet a lot of people's needs and runs much faster.
But you can't bundle and sell that for $200 though :/

Re:I noticed this

[Bourdain]

While I agree with your criticism of the parent comment regarding his lack of empiricism, if he were to change his assumption from

Why is it Microsoft Word 97 fits into my 8 megabyte 386 laptop, and has
99%
of the same functions as modern Word, plus is quick and responsive. Why can't they bring that level of efficiency for today's Word 2010?

to

Why is it Microsoft Word 97 fits into my 8 megabyte 386 laptop, and has
75%
of the same functions as modern Word, plus is quick and responsive. Why can't they bring that level of efficiency for today's Word 2010?

It all of a sudden holds more merit

While I'm not claiming to hunt down statistics to prove my point, I think it's safe to say there's absolutely some truth to lazy programming becoming more the norm than the exception in certain areas.

Do you really think some ratio of perceived/available functionality over ram usage has remained constant or significantly decreased?

Don't you hesitate when you're about to download an application for a simple task but the first one you find is, inexplicably, 100 megabtyes, compressed, whereas the 2nd one you find (and download) is 10 or 15? :)

I know I do

Re:I noticed this

[man_of_mr_e]

Actually, it's closer to 50%. Which means that Office 2010 contains around twice as many features, and those features are almost certainly more complex than the first 50%. Consider a feature like Live Preview. This single feature touches most of the other features, and likely requires significantly more overhead than version of Office that did not have live preview.

As computers get more capable, the software that uses them is written to do more.

Re:I noticed this

[man_of_mr_e]

And what happens when someone sends you an Excel spreadsheet with embedded macros that you're required to fill out for things like Expense Reports, or requesting Sick or Vacation time?

Just because you don't use most features in your daily routine doesn't mean you will never need them, because you can't control what other people send you.

Re:I noticed this

[RobbieThe1st]

See, there's this thing called a 'plugin' or 'extension'. It ought to be possible to simply load a light framework, then load extension-code dynamically as advanced features are needed. Especially with how powerful today's systems are.

Re:I noticed this

[wisnoskij]

Prime example:
new HD widescreen TV, it takes like 5 seconds to turn on and about a whole second to change channels.
It is so annoying to use.

Re:I noticed this

[Surt]

Go try to reformat a thousand page novel on word 97.
And how many grammar mistakes can it spot in that novel?

Re:I noticed this

[Stuarticus]

Computers isn't good at grammar anyways, they always tell me is wrong.

Re:I noticed this

[putaro]

No kidding. We just got a combo DVR/Blu-Ray player. Takes forever to boot and takes 30 seconds to a minute after inserting a Blu-Ray disc before anything starts to play.

Re:I noticed this

[Kamiza Ikioi]

Writing code is easy. Writing good code is hard. Optimizing good code is hard and expensive.

And before someone tells me that GPL is optimized and free, its a trade off of costs, but the price is still high. Optimizing free code costs passion.

Re:I noticed this

[Bourdain]

Okay, let's say it's 50%

The only personal experience I have with this is that, among many other things I do professionally, usually at any organization I'm at (as a consultant or otherwise), I'm embarrassingly the "Excel guru".

Using Excel a moderate amount, I do try to use VBA pretty sparingly given its obvious slowness compared to other methods of calculation, but on occasion, it's markedly more efficient than other designs (e.g. testing for a series of involved conditions which would otherwise process extremely slowly using a formulas, etc.)

I've found that even on modern hardware, Excel 2007 VBA execution, of identical code, is much slower than Excel 2003.

For that reason, along with the sheer inefficiency of the ribbon design in terms of responsiveness and usage of screen real estate, I keep all Office 2007 usage relegated to a VM which I rarely even need.

To further clarify the issue, I have developed a personal library of macros I use in simply navigating spreadsheets efficiently. Even on modern hardware, Excel 2007 cannot keep up with my usage of these macros and throws errors repeatedly whereas I never see such errors in Excel 2003. Keep in mind, this is on modern hardware.

My interpretation of this is that speed and efficiency were low priorities for the Office development team. Given that their interface redesign was, I believe admittedly, largely geared towards novice users, these alleged low priorities make more sense.

To appeal to your sense of empiricism, which I appreciate, please see (perhaps not of the greatest quality...)
http://www.wilmott.com/messageview.cfm?catid=10&threadid=81967
http://www.excelbanter.com/showthread.php?t=137875
http://www.ozgrid.com/forum/showthread.php?t=78673&page=1

for what it's worth, I always disable screen refresh and calculation during a macro (except in rare circumstances when that behavior is necessary)

Re:I noticed this

[bakawolf]

And waiting on things to load as you need them might be a cause of...

Re:I noticed this

[wildstoo]

For that reason, along with the sheer inefficiency of the ribbon design in terms of responsiveness and usage of screen real estate, I keep all Office 2007 usage relegated to a VM which I rarely even need.

I can't speak to the VBA argument, but as for the ribbon taking lots of screen space, have you tried minimizing the ribbon? Right click on it and choose "Minimize the Ribbon". You'll suddenly have more screen real-estate than any previous version of Office.

Need it back? Just click on a header.

With a combination of this, keyboard shortcuts (you know those, right?) and the Quick Access Toolbar, you can assign and access all the functions you need easily and without taking up much screen at all.

Some people love the ribbon, some hate it, but most haven't even given it a chance. I got used to it very quickly, and now I think I may even prefer it.

Re:I noticed this

[Agent0013]

And what happens when someone sends you an Excel spreadsheet with embedded macros

I use Open Office, you insensitive clod!

Re:I noticed this

[Bourdain]

I can't speak to the VBA argument, but as for the ribbon taking lots of screen space, have you tried minimizing the ribbon? Right click on it and choose "Minimize the Ribbon". You'll suddenly have more screen real-estate than any previous version of Office.

Yes I have, but I'd rather always have something visible which takes up less space than the huge ribbon (I believe every other piece of software has it including 2003, called a toolbar)

In 2003, I have more screen real estate available all the time, a fully customizable toolbar and one click access to lesser used functions on the toolbar. In 2007 to have admittedly slightly more screen real estate, I'd have to maximize the ribbon and potentially navigate to a subribbon of sorts (i.e. at least one extra click if not more)

With a combination of this, keyboard shortcuts (you know those, right?) and the Quick Access Toolbar, you can assign and access all the functions you need easily and without taking up much screen at all.

I know all manner of keyboard shortcuts -- they are not as responsive in 2007 as in 2003 so I have to enter them twice sometimes which sometimes defeats the purpose.

The quick access toolbar is also not very customizable.

Long story short:
(1) I've never met an office user at my level who prefers 2007 or 2010 to 2003
(2) I have no problem if MS or any developer wants to add features, just make it backwards compatible and offer the old interface
(3) Anyone who tries to use Excel's "large data" functionality is clueless and shouldn't be trusted with that much data
(4) Access is admittedly an intentionally crippled version of SQL Server and anyone with such data needs should, in most cases, be using PostgreSQL (that said, on occasion I'll use Access as a dumping ground for big files so clueless users can more easily navigate database extracts...)
(5) What do I call the "Orb" to novice users when I want to describe it to them? "the ball in the upper left"? :)

Re:I noticed this

[tonique]

It must be said that Office 2007 is extremely irritating for a seasoned Office 2003 user (like me). Using styles, especially numbered headers, in Word is beyond braindead. (Brainstemdead?) AND MORE CLICKING!

Re:I noticed this

[damnfuct]

Yes! Plus, they made the ribbon to make "hidden" features more visible, but in Office 2003 a seasoned user can add all the "hidden" features into a custom button on a custom toolbar; all that without having to take up 1/5 of the screen real estate.

Re:I noticed this

[damnfuct]

Computer's isn't good at grammer anyways, they always tell me is wrong.

Porblem sloved.

Re:I noticed this

[damnfuct]

To be fair, I noticed that using a CRT with a digital source (e.g. satellite), it seemed like there was some interval after changing a channel where the receiver needed some amount of data (or some time interval) before it could start displaying an image. This timing seemed to be aggravated by water between the satellite and dish (rainclouds), or a poor calibration resulting in a low signal strength.

Also, managing digital pixel data is a bit more complex than syncing an analog signal to a scanning cathode. Don't get me wrong, I'm not defending the process (I agree with you that the delays are annoying).

Re:I noticed this

[man_of_mr_e]

If you look at your first article, and read all the way to the bottom, you'll find that Office 2007 was slow to execute macros at first, but later hotfixes solved that problem.

Office 2010 is also significantly faster.

Re:I noticed this

[man_of_mr_e]

The orb is gone in Office 2010. Microsoft listened to that feedback and changed it.

Access is not just a database, but a forms engine. You can't replace Access with Postgre, it's only part of the solution. What people like about access is that it's a single file that you distribute, double click on it and your app runs, including the database. Postgre simply can't do that, even if you use some other forms engine.

Lots of people were constricted by Excel 2007's 64k row limitaiton. Excel is a useful tool for a lot of people who aren't database experts. You can call them clueless, but they are getting their jobs done just fine with Excel.

Re:I noticed this

[Bourdain]

The orb is gone in Office 2010. Microsoft listened to that feedback and changed it.

Good for microsoft, it took them a few years to fix that one

Access is not just a database, but a forms engine. You can't replace Access with Postgre, it's only part of the solution. What people like about access is that it's a single file that you distribute, double click on it and your app runs, including the database. Postgre simply can't do that, even if you use some other forms engine.

Number one, it's called Postgres or PostgreSQL, not Postgre which suggests to me you're either not detail oriented or are not familiar with the actual product. That said, you're right about the forms engine positioning of Access but in reality, virtually all of the "applications" I've seen built in Access are crap and slow. I've intentionally avoided learning Access's VBA because the application is so terrible and having "skills" in it of any substance are nearly worthless. Also, in the consulting world of sorts, I see people all the time tinkering with Access typically producing wrong results or getting odd error messages because of Access's jet engine [intentionally built-in] failings. That said, in very isolated situations where the amount is data is pretty small (typically fewer than 1 million records w/o many sophisticated fields) and it has to be sent to someone who wants to run "their queries" it's acceptable.

Lots of people were constricted by Excel 2007's 64k row limitation. Excel is a useful tool for a lot of people who aren't database experts. You can call them clueless, but they are getting their jobs done just fine with Excel.

I frequently work with these people who allegedly "get their jobs done fine". These are the sort of people who sometimes ask me for help and then I do 1 to 2 weeks worth of their work in 10 minutes of my time while they watch dumbfounded since they are so ridiculously clueless. I've seen people run counts in SQL on massive servers across a wide variety of tables, one at a time. Keep in mind these tables were never indexed because these "individuals" don't understand what indexing is (to Access's credit, it does have some rudimentary indexing which you typically have to manually enable depending on the situation...). As such, these "individuals" spend days just counting records in tables whereas I'll write a script which will index all of the relevant fields and record all of the counts, typically taking a few minutes instead of days...

If you look at your first article, and read all the way to the bottom, you'll find that Office 2007 was slow to execute macros at first, but later hotfixes solved that problem. Office 2010 is also significantly faster.

I saw that too, but wouldn't you say that if their first "upgrade" is actually a downgrade in terms of performance, then the developers didn't have their priorities straight? After all, why should it takes a few hotfixes and a version that comes out ~6 or 7 years later only to get back to where you were?

Re:I noticed this

[RobbieThe1st]

Yea, but probably not. You have small enough blocks, and it ought to be able to load a single one in under 150ms. That's quick enough for it to feel "fast enough". Plus, you'd have the advantage of the program being usable almost immediately, instead of waiting for all these features(which probably won't be needed) to load.
Heck, add a 'last used' list, and have it automatically load the modules used last time(or last three times) - Probably a good indication of what's going to be used this time.

Changing TV channels

[Anonymous Coward]

And here I thoguht I was the only one complaining that changing channels gets slower and slower with every new receiver box.
On analog it was basically instant, less than 100ms.
First digital box took half a second. Full HD box sometimes takes a whole second or more (and it's not even deterministic anymore)

That SUCKS big time!

Re:Changing TV channels

[Dr_Barnowl]

That's down to compressed stream buffering. An analog box could be instant because every frame was transmitted uncompressed. With digital TV, you have to wait for a keyframe at least.

Re:Changing TV channels

[JordanL]

This does not explain the > 500ms delay when trying to use the guide.

Re:Changing TV channels

[BenoitRen]

In my experience with Belgacom's digital TV, input latency is the norm. Most of the time when you input a command, it takes at least a second for it to execute. It's quite frustrating.

Re:Changing TV channels

[Luyseyal]

It might if it's getting the channel's feed in the background. It is annoying though. They really ought to consider a "just the listings, I don't want to see the channel at the same time" option. That would be a TON faster.

-l

Re:Changing TV channels

[SheeEttin]

This is just anecdotal, and probably a one-off kind of thing, but the other day my Comcast cable box/DVR was being very slow to do anything (i.e. press a button on the remote, wait several seconds for a response). I fixed it by unplugging it and plugging it back in again.

So, I guess what I'm saying is: have you tried rebooting it? :P

Re:Changing TV channels

[rsborg]

And here I thoguht I was the only one complaining that changing channels gets slower and slower with every new receiver box.
On analog it was basically instant, less than 100ms.
First digital box took half a second. Full HD box sometimes takes a whole second or more (and it's not even deterministic anymore)

That SUCKS big time!

Or a more positive spin,maybe this will result in less TV watched. I've noticed that since we have gone Dish (wife must have TV5Monde), we hardly watch TV at all (the little tot watches the most, probably 30m-1hr a day avg, but this is all netflix streaming/DVR/appletv content).

If some startup or established company is keen on solving this, they could be highly disruptive in the TV space.

Re:Changing TV channels

[El_Isma]

That's due to how digital cable works. I'll speak of DVB since that's what I know (I don't know what USA is using, but surely it will be similar). DVB sends a big stream composed of several smaller streams, some of those are video/audio streams, some are channel information (the guide, the streams IDs (audio/video/cc) of the channel, etc), others are info on the stream itself (carrier frequencies) or general information (time).

For the video stream, as the parent poster said, you'll have to wait to get a keyframe to start to view it (sending more keyframes means less efficient coding, means more bandwith per channel).

For some of the channel information, like the guide, you have a tradeoff between bandwidth and latency. Sure, you can stream the guide with almost 0 latency, but that means using a lot of BW to be able to send all the channel guides all the time. You have less available BW for channels, which means having to use more carrier freqs which means more money on hardware to send those signals (and possibly repeaters...). So, cable operators send the guide with a "reasonable" BW. The problem gets worse the more channels you have, since the channel guide has to be sent on all carriers.

Re:Changing TV channels

[El_Isma]

Digital cable normally works unidirectionally, think multicast. The header sends information for all the receivers, is not point to point like the internet. So, it doesn't matter that you don't want to see the channel, they still have to send it because somebody else *might* want to see it. And, yes, the channel guide travels in the "background".

Re:Changing TV channels

[Omnifarious]

Since many of these technologies transmit the data for all channels simultaneously, why not just scan for key frames and store the last key frame received for each channel? It might be that even just scanning for key frames might be too CPU intensive to do for all channels. But most people, when channel flipping, do so in a fairly predictable order. You could start doing this for the most likely targets for channel flipping when channel flipping behavior is detected.

Re:Changing TV channels

[Anonymous Coward]

You're not alone, brother. I first noticed the problem a few years ago on a hotel TV; I assumed they'd bought cheap TVs. Now, I act like a good consumer and get the latest piece of crap brand-name full-HD LCD TV, and I'm subjected to the same _degradation_ in performance of a basic function. Not one word in the reviews about that, either.

Oh, and PS, the Toshiba 32E200U has the single stupidest feature I've ever encountered: a two-level mute button (press once for 1/2 mute, twice for full mute). WTF were they on when they came up with that idea?

God, I hate TV. At least Bin Laden paid for his crimes.

Re:Changing TV channels

[Alamais]

I don't understand why you wouldn't do it like my Dish receiver seems to: send the guide constantly, slowly, over and over, and store it. That way it may take a while (hours?) to first get the guide when you start the box up after installation, but once it has its first copy, it just has to delete old data and look for updates in the stream. You can have longer (more far-future-seeing) guides this way, and access is instant when you push the button. Even though it does the PiP thing in the corner while you browse, my receiver's guide has never bothered me latency-wise.

Re:Changing TV channels

[cpu6502]

TV Guide is instant on over-the-air television. Mainly because the data is immediately available from the box's memory. Anything past 3 hours takes a little longer to fill-in, since those are only sent every 5 seconds. Past 1 day, the data is only sent every 30 seconds.

As for channel changing, I've noticed some boxes are slow as snails, while others are instant and display the picture almost as fast as analog. It all depends on the manufacturer.

Re:Changing TV channels

[PhrostyMcByte]

A cable box needs to have a separate tuner for each stream. Live TV takes one, each recording show takes one, etc. and such keyframe prefetching would also take at least one. Most cable boxes only have 2-3 tuners, and inevitably they're all going to be taken. So while this could definitely hide the latency in some situations, it's not a perfect solution.

Cable companies are also beginning to use SDV, which broadcasts channels on demand instead of all at once. Some latency will be involved here too, because your cable box needs to send a request to the SDV hub if it's not streaming the channel already.

Re:Changing TV channels

[assassinator42]

I'm guessing Microsoft does something like for their IPTV platform (used in U-Verse in the US). You receive a unicast stream until the multicast stream is joined (and presumably another keyframe reached). Although this isn't always seamless and you can't rewind to what you received in the unicast stream.

Re:Changing TV channels

[TapeCutter]

When I was a kid I had to wait for the B&W TV to warm up, now I'm an old fart I have to wait just as long for the digital set top box to boot up. Somewhere in between I had a TV that came on instantly.

Re:Changing TV channels

[Omnifarious]

I the case of a request things get a little easier. The entity serving up the response can do a little extra work to send an immediate key frame and tailor the rest of the stream up until the next key frame for the requestor before shifting them to just getting a duplicate of the data everybody else is getting. You still have the request latency, but I bet you could still manage to keep the latency under 50ms if you tried hard.

Re:Changing TV channels

[MobileTatsu-NJG]

This does not explain the > 500ms delay when trying to use the guide.

That's probably due to being 'keyframe' compression. The way that works is you start with frame 0, and that is the entire picture. The computer looks at frame 1, compares it to frame 0, and determines the difference. Frame 1 is now a partial frame that is overlaid on top of frame 0. Then frame 2 repeats the process, you end up with a partial frame which looks a lot like an old cel of animation missing its background, yadda yadda yadda. Then, when you reach a certain threshold (like 96 frames, maybe?) a whole image is put into the stream. Then, 97 is looking at 96 instead of frame 0.

So why does this cause a huge latency? Well, if you go by my example and want to get frame 95, you have to start at frame 0 and build up the image for EACH FRAME before you get to 95.

With broadcast TV this gets a little more complicated. You can't rewind the signal, it is just constantly blasting into the receiver. What is probably happening is the receiver is waiting for the next key-frame to come along. Maybe in your case the stream you're getting is compressed using a keyframe every 15 frames.

Re:Changing TV channels

[scrib]

Saving the last key frame is also a waste, unless you save all the deltas up to the current time, too. That's the same as trying to play all the channels at once.

You wait for the key frame, display it, and start viewing the stream from there. Using an old key frame and viewing the deltas somewhere midstream would result in weird looking corruption until the next key frame was displayed. Best just to wait.

Re:Changing TV channels

[Omnifarious]

In my opinion, showing just a still keyframe, or a keyframe with weird corruption is still preferable to channel switching taking a full 500-2000ms. Though the weird corruption would probably be a customer education issue.

Re:Changing TV channels

[JonySuede]

it is the encryption technique utilized that introduce this lag. There is a key change at about 1s so it must wait until the next key to decode the stream.

Re:Changing TV channels

[JonySuede]

i should have posted in the parent post

Re:Changing TV channels

[clang_jangle]

Ha, I remember that. On a lot of sets a tiny dot of light would appear in the middle of the screen, then after about five seconds or so the dot would inflate to fill the screen. There was hardly ever anything on back then, too...

Re:Changing TV channels

[Ghostworks]

No, in my case I can trace the worst of it directly the the Time Warner Cable channel guide. Slowly rolled out in Austin over a few days, I immediately noticed a huge delay when changing channels, and was able to confirm with friends and neighbors across the city (some who had HD boxes, some not, some with the new guide, some without), that those who had the new channel guide were noticeably slowed (~1s), and those with the old guide didn't see anything unusual... until they too received the "upgrade".

The trouble in this case is that once you've invested a ton of money into an upgrade that clearly doesn't work as planned, it also doesn't make sense to spend more money to fix it or to go back to the old method. Unless the system is completely non-functional or people are bailing in droves, you just have them grin and bear it until the next scheduled upgrade. Say, five years or so.

Re:Changing TV channels

[jo7hs2]

Well, at least for those of us stuck with Comcast, I suppose the advertising crammed into the guide might play a role.

Re:Changing TV channels

[El_Isma]

Cheap people don't put enough RAM on their decoders :)

Re:Changing TV channels

[gmhowell]

I suspect the half mute is in case the commercials are too loud, but you still kinda want to hear them.

Re:Changing TV channels

[Ferzerp]

No, it can't. That would require every stream to be encoded separately on the fly to every cable box based on when they requested the channel to be tuned.

Re:Changing TV channels

[cgenman]

True. But the remote server could broadcast the first frame with a little extra effort. Additionally, there is no reason why some receivers have to block for input until the channel is decoded.

Re:Changing TV channels

[mattack2]

You could design a system to do that, but at least with SDV in the U.S., the tuning is still done via QAM channels like without SDV. (e.g. SDV adapter for Tivos -- all it does AFAIK is transmit the channel requested upstream and receive the info about what QAM channel is being used, which the cablecard equipped Tivo then tunes to.)

Re:Changing TV channels

[zippthorne]

WTF were they on when they came up with that idea?

God, I hate TV. At least Bin Laden paid for his crimes.

Probably thinking about all those myriad channels that compress the audio so the shows have low fidelity and commercials are ear-piercingly loud.

A better solution, though, would be to use the volume as a commercial-skip signal...

Re:Changing TV channels

[Lehk228]

What's worse is the crap firmware on those scientific atlanta boxes which cannot flip to the next channel until this one has tunes in.

Re:Changing TV channels

[radish]

It does suck. That said, I don't watch anything live anymore so it really doesn't matter. DVR/Streaming FTW.

Re:Changing TV channels

[tibit]

A cable box needs one wideband tuner and an SDR that can demodulate all the channels at once. There's obviously no need to do video decoding (decompression), just buffering of some subset of channels (say most often recently used). One QAM channel brings about 34.5 Mbit/s. A modern ASIC should have no problem easily dealing with demodulating and buffering twenty such channels (700 Mbit/s) when presented with a wideband IF output. The buffering only needs 90 Mbytes/s for 20 channels. That's not much if you ask me. Again: there's no need for 20 physical tuners. There's just one tuner that doesn't do much besides amplifying and prefiltering the signal, and then feeding it into an very fast A/D. The output of that A/D goes into the ASIC and there the individual QAM channels are demodulated and dumped to external RAM. When you switch channels, it's trivial to scan the buffered stream for a given channel to find the most recent keyframe and immediately feed it to the MPEG decoder, so that you get video output with minimal latency.

For implementation reasons, and due to economics, the cable spectrum may need to be split up in a couple subbands (say 4), and those would be individually digitized, so you end up with "4 tuners", but those tuners are somewhat simpler than a regular tuner: their heterodynes all run at fixed frequencies. The IF can be likely directly digitized for all of the subbands.

Re:Changing TV channels

[tibit]

I think that the technology to solve is there, and has been there for ages. The problem is that it's not very cheap unless you'd manufacture in the millions. A prototype will run you a couple thousand dollars just for the A/D converters...

Re:Changing TV channels

[mikael_j]

Still, this doesn't excuse the fact that when someone presses the "guide" button it takes two seconds until their TV reacts at all. It could pop up a small "loading guide" message at the top left/right corner of the screen. It could also be a bit more clever about caching information needed to show the guide (some TVs obviously are and not necessarily the more expensive ones).

A lot of times I think lag issues like these pop up because the people building the software aren't using "real" hardware and thus aren't limited in the same way that a regular user is (like say, their lab setup pushes three channels worth of data directly into their software "TV simulator", IRL you're receiving the non-video transmissions at a lower data rate and it's being sent to you rather than just read from disk).

Re:Changing TV channels

[FrootLoops]

It takes me around 5 minutes to surf through all my TV channels. Deciding if I want to watch something takes pretty much a single frame, but waiting for channels to switch takes ~1.5-3 seconds. Waves of partly synched commercials keep me from knowing what's on many of those channels. TV Guide channel is basically just as awful, since they make it scroll excruciatingly slowly nowadays. I can go online and get listings, but that's slow and roundabout. I used to have a cable box with the usual channel guide, but the box was so slow it took a second or two to scroll--which was actually much better than my current channel surfing situation.

I'd like to force the TV Guide people to use their own service day-to-day. I bet the scroll speed would triple.

Re:Changing TV channels

[tlhIngan]

This does not explain the > 500ms delay when trying to use the guide.

Easy, crap software.

Cablebox DVRs suck. Horribly. Besides the UI stinking, and random latencies and random ability to record TV.

It's a wonder, since TiVo's been out over a decade (and it's UI is a bit dated, but works well and is far less complicated to use), and cablebox software is still crap. Especially since cableboxes have more powerful CPUs than TiVos.

I've never understood it. I guess that's why TiVos cost so much more - when you pay $3 a month for a DVR, they can throw whatever crap on it and the user doesn't know better, because they'd rather pay that than $13/month.

And I can bet Moxi, Myth and Windows MCE all have better UIs.

Re:Changing TV channels

[antdude]

Yep. Another reason why older/analog stuff still rules like my current VCR (from dotcom days) and 20" CRT TV from 1996. When FCC fully moved to digital, I tried a couple converter box. Oh my Lord, so slow. Not just slow, BUGGY and no way to upgrade its firmware like its DVR!

Re:Changing TV channels

[A Friendly Troll]

My parents have an IPTV box that crashes a couple of times per day. When it does, it needs a reset; and when you reset it, you have to wait a solid 5-8 minutes until it boots back up. Black screen, blue screen, fancy graphics screen, "Loading...".

Re:Changing TV channels

[advocate_one]

you should patent that... oh wait... you've just blabbed it to all and sundry... let's just hope your post can then act as a marker that the idea is in the public domain and prevent anyone else from patenting it too

Re:Changing TV channels

[Lobachevsky]

Most people hit the "up" or "down" buttons for surfing. I don't know why devices don't just prefetch the adjacent channels' streams.

Re:Changing TV channels

[weicco]

You are lucky if it's only switching the channel that is slow. In my box, which I've already given away, everything took from hundreds of milliseconds to seconds. Powering on the device which took well over 15 seconds! Scrolling EPG channel list was awful. It was faster just to switch the channel and check if there's anything good going on.

And this box ran Linux so I don't think Visual Studio is the one to blame as the fine summary suggests :)

Re:Changing TV channels

[vegiVamp]

I was around when it was first being developed (albeit not actively involved), and I find that to be wholly unsurprising. Back then, it had to do both with the topographical complexity of the environment it runs in, and the quality of the majority of people involved. Things might have changed in the mean time (I left years ago), but your remark makes me doubtful.

Re:Changing TV channels

[16384]

When I was a kid I had to wait five minutes for a game to load on the ZX Spectrum. Now I have to wait a quite a few minutes to start the computer, steam, then the game and connect to a server. Somewhere in between I had DOS games that started mostly instantly.

Re:Changing TV channels

[xaxa]

For DVB there are several "multiplexes" (different radio frequencies) each containing several streams (TV channels, the electronic programme guide (EPG), radio channels, interactive/text channels).

It's a while since I worked with this, and I was only the placement student, but IIRC (and it makes sense) each multiplex only broadcasts the EPG for that multiplex. Here's what's broadcast in the UK. If you're watching BBC1, any TV will instantly switch to BBC2, BBC3 etc, or the listed BBC radio stations, or the text service. It should have cached what it needs to (the most recent keyframe, etc). It will be caching the EPG from that multiplex, so pressing "i" to see what's showing now/next/etc on the current channel is instant. But, unless you have multiple tuners, it can't retrieve EPG information from other multiplexes. That's why older TVs stop the TV stream when you hit "guide", and take a little time to load the rest of the guide. I think newer TVs have two tuners (you can watch any two channels at once, but when you're not doing that the TV can retrieve the full EPG).

I don't know if the American standard works in the same way. DVB is older (first properly broadcast in 1998) so ATSC might have been designed with more RAM in mind.

Re:Changing TV channels

[FrootLoops]

That might be less helpful than you'd think. Suppose they were able to prefetch and reduce adjacent channel switch times to 50ms. I'd want to switch to the next adjacent channel around ~300ms after the first switch, but the prefetching mechanism couldn't have completed its job since this silly TV takes ~1-2 seconds to switch (I timed it more accurately). You'd have to prefetch several adjacent channels to completely eliminate the delay. I don't know how involved it would be hardware-wise to do that. I'd happily accept being forced to channel surf, say, "up", to reduce that complexity.

Re:Changing TV channels

[general_re]

Digital cable normally works unidirectionally, think multicast. The header sends information for all the receivers, is not point to point like the internet. So, it doesn't matter that you don't want to see the channel, they still have to send it because somebody else *might* want to see it. And, yes, the channel guide travels in the "background".

That's mostly correct but some providers are trialling switched digital video (google it), which will only add to the latency issue.

Re:Changing TV channels

[tlhIngan]

No, in my case I can trace the worst of it directly the the Time Warner Cable channel guide. Slowly rolled out in Austin over a few days, I immediately noticed a huge delay when changing channels, and was able to confirm with friends and neighbors across the city (some who had HD boxes, some not, some with the new guide, some without), that those who had the new channel guide were noticeably slowed (~1s), and those with the old guide didn't see anything unusual... until they too received the "upgrade".

The trouble in this case is that once you've invested a ton of money into an upgrade that clearly doesn't work as planned, it also doesn't make sense to spend more money to fix it or to go back to the old method. Unless the system is completely non-functional or people are bailing in droves, you just have them grin and bear it until the next scheduled upgrade. Say, five years or so.

Like I said before, it's crap software coupled with viewers who don't give a damn other than "it's cheaper than TiVo". Basically slap dash any old crap together.

I'm disgusted with our cableboxes, but TiVo can't be used because in Canada, there's no mandate for stuff like CableCARD, so all cable providers sell their boxes which can't work with any other providers, and you can't use any box other than one the provider originally sold. It's a load of garbage software and terrible UI. (When the CableCARD mandate came out, PACE bought up all the old cablebox designs from Motorola just so they could manufacture the boxes strictly for the Canadian market).

Hell, that crap cablebox may be one reason people are cutting the cable too. When even cheap chinese-made network media players have better UIs...

Re:Changing TV channels

[juancn]

That's to show the video. But why I have to wait for the keyframe to change the channel?

If I want to jump 3 channels up I have to wait for 3 keyframes in 3 streams. I just don't care about the intermediate ones. It's just shoddy programming.

Re:Changing TV channels

[operagost]

... which was fortunate, because YOU were the remote control for mom and dad! click click click click... Hey, point the rabbit ears a little that way for channel 3!

Re:Changing TV channels

[TheGreatMcCluck]

Somewhere in between I had a TV that came on instantly.

And there it is; Here is someone who has recognized that technology's glory age has come and gone. Sorry for those of you who blinked, but you missed it.

Re:Changing TV channels

[damnfuct]

Or they use the cheapest microprocessor they can get away with

Re:Changing TV channels

[rayd75]

For some of the channel information, like the guide, you have a tradeoff between bandwidth and latency. Sure, you can stream the guide with almost 0 latency, but that means using a lot of BW to be able to send all the channel guides all the time. You have less available BW for channels, which means having to use more carrier freqs which means more money on hardware to send those signals (and possibly repeaters...). So, cable operators send the guide with a "reasonable" BW. The problem gets worse the more channels you have, since the channel guide has to be sent on all carriers.

Interesting... If only we could develop some kind of cheap technology capable of storing this 50 - 100K of infrequently-changing guide information so we wouldn't have to wait every single time we wanted to look at it. Even if this theoretical technology lost the information every time power was removed, it would still be a massive improvement over what we have now.

Re:Changing TV channels

[El_Isma]

Indeed. It seems the people that write the software haven't caught up with this "caching" technology you speak of.

Re:Changing TV channels

[BenoitRen]

There was a major UI update a couple weeks ago. The latency seems to have lessened with it (perhaps because now it's an overlay instead of a separate interface with downscaled TV in the corner), but it's still there.

Re:Changing TV channels

[mcmonkey]

You are lucky if it's only switching the channel that is slow. In my box, which I've already given away, everything took from hundreds of milliseconds to seconds. Powering on the device which took well over 15 seconds! Scrolling EPG channel list was awful. It was faster just to switch the channel and check if there's anything good going on.

And this box ran Linux so I don't think Visual Studio is the one to blame as the fine summary suggests :)

You're lucky! My box is so slow, it takes an hour and a half to watch 60 minutes!

N900

[Dr_Barnowl]

The N900 suffers from this, alas.

I can't comprehend why the phone app isn't in memory on boot. It's a PHONE. Instead, when the phone rings you have to wait several seconds for the phone application to load.

In contrast, my wife's new HTC Z snaps and zings along with Android, even though it's "bloaty" Java / Davlik.

Re:N900

[blair1q]

Is it a phone?

I use my "phone" for phoning and receiving phone calls and talking on the phone maybe...I dunno, 0.6% of the time it's lit up?

It's a computer. If I happened to put a phone in there, then that saves me having also to have a phone.

Though I will get a burner if I think Jack Bauer is on to me. Maybe a six-pack. I ain't stupid.

Re:N900

[Americium]

I have the same issue. I think one of the most appealing things about the iPhone is simply how smooth and fast is reacts. It's like GNOME or KDE vs Windows 7, the slight lag just makes Linux not fun to use.

Re:N900

[Microlith]

I can't comprehend why the phone app isn't in memory on boot. It's a PHONE.

As Nokia put it, it's a device designed around "mobile computing." The phone app is entirely secondary (unlike on most Symbian devices, which have physical buttons) and is mostly there as a convenience thing. The primary reason the circuitry for it is even there is for 3G data.

Instead, when the phone rings you have to wait several seconds for the phone application to load.

Which lies totally outside my experience, where it comes up instantly.

90% of the lag on the N900 is if you've just done something memory intensive (notably the browser) that caused stuff to be paged out to the eMMC, which isn't exactly fast. But the crippling problems that used to cause were resolved back in September or so with the PR1.3 update.

In contrast, my wife's new HTC Z snaps and zings along with Android, even though it's "bloaty" Java / Davlik.

That's because it's a phone, and not a more general purpose sort of device. In fact, they don't even want you messing with it which is why you have to root the damned thing.

Re:N900

[cforciea]

I use my iPhone really a lot (it is work issued and called all day), and I find that usually once every couple of days I fail to actually answer an incoming call because I can't actually get it to react correctly. Some of the time, I can't get the answer slider to move across, some of the time I have some frozen screen up that doesn't let me interact with the phone (I personally love it when the GUI for a phone call is up, only it is not live so I can't touch anything on it. I assume this has something to do with the background image for the call coming up without the actual interactive widgets being there). I also randomly have apps (including preinstalled apps like google maps) close without explanation and random freezes besides. I used to think it was because I was using an old beat up 3g with old-ish firmware, but my company recently replaced it with an iPhone 4 with updated firmware and very few installed apps and it has more or less the same problems (without dying pixels on the screen and a mostly dead battery that is not really end user replaceable, which is why the phone got swapped). I don't have a ton of experience using an Android to make a comparison, but I can't imagine that what I have should be the target of anyone's aspirations. I just can't figure out how they can put so much hardware into a phone and have it still be a substantially worse phone than the tiny Samsung dumb flip phone that I got for free with contract years ago.

Re:N900

[somersault]

What do you mean about the messaging? My Dell Streak has been used a lot more for texting, web browsing , ebook reading and youtube than it has as a phone. Likewise my Xoom doesn't even have anything but wifi.. Android is no more a phone OS than iOS is.

Re:N900

[Microlith]

What do you mean about the messaging?

messing, not messaging. Regardless, they don't want you stepping outside the carefully set bounds of the Android OS and they try to enforce it by making you root the system.

The N900, by contrast, is a much more typical Linux system that you can freely enable root access via official methods. Tuning has definitely been needed, but my point against the GP was that I had not seen the latency he talks about in response to phone calls, and why the phone application wasn't loaded 100% of the time.

Re:N900

[lewiscr]

The HTC's aren't phones either. Nothing running Linux qualifies as "just a phone". This includes most "dumb" flip phones (which are running embedded linux, QNX, or some other embedded OS). The real time OSes usually handle it better, but I've had those sort of problems with them too. I don't remember the last time I had a cell phone that didn't have any applications (even if it was a manufacturer installed MP3 player app).

Re:N900

[maxume]

That's a hell of an argument, it's not a phone, it is just a shitty computer with some cellular and audio hardware built in.

But at least they made a computer with a shitty phone application, and not a just a shitty phone.

Re:N900

[Stuarticus]

But at least they made a shitty computer with a shitty phone application, and not a just a shitty phone.

FTFY

iReport

[RichardJenkins]

Whenever I use iReport there is about a sixth of a second delay between interacting with it and it doing what I asked. It's barely noticeable but still drives me insane.

RTFA

[Dog-Cow]

I was going to read the article, but it took too long to load.

Re:RTFA

[mikael_j]

Oh, Windows has antialiasing, it's just that it tries too hard to force fonts into the pixel grid so while it does make them readable it also makes them look sort of ugly and broken (unless specifically designed for ClearType).

Then of course as you pointed out there is the weird window tearing thing going on, which for some reason will happen even on a top-of-the-line Windows machine with no software running except the one explorer.exe window you're moving around.

And finally there's the general UI design where your choices boil down to "whatever crazy translucent shiny MS is making the current version default to" or "Windows Classic" which is grey and boring to say the least (and not in a minimalist way, in the "this must have been designed by an accountant" way).

Re:RTFA

[gauauu]

Oh, Windows has antialiasing, it's just that it tries too hard to force fonts into the pixel grid so while it does make them readable it also makes them look sort of ugly and broken (unless specifically designed for ClearType).

Oh man, the forcing of fonts into the pixel grid is the main reason I stay with windows. Seriously. On Linux, the fonts have to be a lot bigger to be clearly readable than they do on windows, because it doesn't force them into the grid as well. On windows, I can have a lot more stuff on the screen (and still have it clearly readable) than I can on linux, because they do such a great job of forcing their weird fonts into the pixel grid. Every time I plan on switching to Linux (which is about once per year) the thing that brings me back is the fonts. Linux makes me choose between huge fonts or unreadable fonts.

Re:RTFA

[wbo]

Then of course as you pointed out there is the weird window tearing thing going on, which for some reason will happen even on a top-of-the-line Windows machine with no software running except the one explorer.exe window you're moving around.

The window tearing that you are describing was fixed by the hardware accelerated window manager that was introduced in Windows Vista.

If you are experiencing window tearing on a modern Windows machine, either your graphics card or drivers does not support the features required for hardware composting or you have disabled Aero. (Disabling Aero forces a fallback to a software desktop compositor similar to the one used in Windows XP and Server 2003.)

Please note that you can disable the transparent glass effects while still taking advantage of the new window manager by switching the theme from Aero Glass to Aero Basic.

Re:RTFA

[mikael_j]

I'll admit right now that I'm using "Windows classic" as my theme on my work computer. The main reason for this is that the various "Aero" themes just look garish and clunky to me, I just can't stand them. Is there perhaps some way of theming Vista so that it looks clean and minimalistic without replacing system dlls? (the latter being what pretty much everyone seems to recommend doing when I complain about how it looks but I'm not in the mood for screwing up my work laptop since I need it for, well, work)

Also, there's a part of me that wonders why they didn't just implement an Aero version of the Windows classic theme, when I look at the desktops of most IT professionals/developers I know who use Windows they're using Windows classic, same for "artsy types" who aren't using macs. An acquaintance of mine even hypothesized that the only people who used Aero were "gamers and grandmas".

Worst Offender: Satellite Receivers

[omnichad]

I don't personally have Satellite, but it seems that almost every satellite receiver is underpowered to run the GUI.

Re:Worst Offender: Satellite Receivers

[gknoy]

I've had a slightly different experience. I use DirecTV; while it's not the snappiest at changing channels, the channel listing menu is fairly responsive. In comparison, using my in-laws' (or my dad's) digital cable is like trying to use the application through a VPN into a slow windows ME box in Elbonia. Not only was it slow to change channels, the channel list itself had noticeable rendering latency and artifacts.

Re:Worst Offender: Satellite Receivers

[GuldKalle]

At least then you get the product slightly cheaper. I find products with plenty of cpu far more annoying. Take the Xbox 360 UI. A lot of the menu navigation has a delay of about 1 sec, just for doing some animation. And input is disabled in that time.

Re:Worst Offender: Satellite Receivers

[Alamais]

I've got Dish's fancy (and under-supported) new VIP 922 receiver, and channel-change lag and UI response are complete non-issues. The usefulness of the built-in Sling support is another thing...

slashdot

[Anonymous Coward]

Slashdot's Javascript.

Re:slashdot

[snowraver1]

^^ this

Any popular story, such as the recent Bin Laden killing story, opens SSLLOOWWLLYY... It takes maybe 2-3 minutes to open, and during that time, the browser is unresponsive. (I have a single core HT laptop with 3 gigs of ram). Once the page loads, actually scrolling is a joke. The lag there is in the 5-10 second range. C'mon /.

Re:slashdot

[Sardak]

Has it really gotten that bad? Luckily for me, NoScript blocks all of the javascript on here, so any page, regardless of its size, loads just as quick as anywhere else. They also have an option under the preferences to switch to the classic discussion system that might help as well.

Re:slashdot

[Intrepid imaginaut]

Use firefox instead of chrome. ;-) Works like a charm for me and I don't have your specs.

Re:Slashdot

[tepples]

The 10-second preview wait is to port scan your IPv4 address for open proxies. I compose long replies in Gedit or Notepad while I wait for Slashdot to catch up.

Re:slashdot

[milas]

I'm using Chrome/Ubuntu 10.10 on an ASUS netvertible which uses one of the older low-powered Atom chips clocked at 1.33GHz, and while Slashdot definitely isn't the snappiest website, it loads decently quickly and scrolls smoothly for me. I don't understand where this unattainably high standard for the Slashdot JS came from. Much worse are Disqus powered websites! Of course nothing comes close to the mess of JS that is Facebook...

Re:slashdot

[snowraver1]

I use IE7... It's a work laptop. My computer at home runs much better (FF on dual core AMD X2 with 4 gigs), but still I would expect better performance on IE7. I'm sure that I am not the only one using it on this site.

Re:slashdot

[Hatta]

Then block it. I use NoScript. Set your discussion preferences to "Classic Discussion System (D1)" for best effect.

Re:slashdot

[Penguinoflight]

IE7 was slow when it came out, compared to firefox 4 it is truly terrible.

Re:slashdot

[rrohbeck]

You're doing it wrong. 1.6GHz Atom Netbook here with no problems. Running Debian Squeeze with Firefox 4.0.

Re:slashdot

[a_hanso]

Any popular story, such as the recent Bin Laden killing story, opens SSLLOOWWLLYY... It takes maybe 2-3 minutes to open

Happens to me too. Using chrome on windows. Story section loads but the comment section seems to get stuck, which often requires a refresh of the page. Clicking a partially hidden story sometimes results in the page scrolling up to its grandparent, which makes me lose my place on the comment page. Very annoying.

the olden days.

[radioteeth]

I miss the days where manually compensating for lag in games like Quake was unquestionable and accepted as a part of the multiplayer gaming experience. It's hard to imagine trying to cope with such a thing these days.

Re:the olden days.

[Anonymous Coward]

I miss the days where manually compensating for lag in games like Quake was unquestionable and accepted as a part of the multiplayer gaming experience. It's hard to imagine trying to cope with such a thing these days.

I assume you're referring to NetQuake vs. QuakeWorld. The real argument was client-side prediction vs. no client-side prediction.

I questioned what the hell they were thinking (being that I was using the U of MN's slip/ppp dialup), and I hated ice-skating around in NetQuake. I then thanked the $DEITY the day that the QW beta started.

Client-side prediction taught me that for most FPS games (perhaps some other online game genres) you can play just fine with sub 150ms latency and a well implemented client-side prediction scheme.

CAPTCHA: playable

same as it ever was

[decora]

Linus Torvalds has rejected patch after patch after patch that would make Linux into BeOS style latency. He has rejected them all. Why?

because he wants to goose the server performance numbers. thats basically the story of the last 20 years of linux.

Re:same as it ever was

[hhedeshian]

Why shouldn't he?

Sure, a lot of people use Linux on the desktop but (not counting phones) its server market share is (much) better. Frankly, if these patches are super-duper-mega-ultra-awesome, like you hint at, distro maintainers would be including them. I don't count phones because they're so far away from what you and I call "Linux" that it doesn't really count. I'm sure there are layers of proprietary garbage that would make latency worse with or without these patches.

Most Linux users use a kernel supplied by their Linux distribution. Some distributions ship the "vanilla" and/or "stable" kernels. However, several Linux distribution vendors (such as Red Hat and Debian) maintain another set of Linux kernel branches which are integrated into their products."

Linux Kernel

Re:same as it ever was

[Twinbee]

Hi, I wrote the article in the story.

Can you give me some verification and links to your comment, and I might see if I can fit it into the article. As you say, there should always be a sensible balance between latency and throughput.

The new slashdot interface

[nu1x]

It actually drives me insane, it is markedly worse, I read less stories because of it (because I do not like the feel of the site so much).

I would bet most of the actual slashdot users feel (think ?) the same way. Why is there no mass appeal to change it back / forward in a more reasonable (i.e., simpler) direction ?

Re:The new slashdot interface

[hhedeshian]

+1 agree

Maybe slashdot should have a poll about it instead of how much you hate Sony; we already know that information.

Re:The new slashdot interface

[Anonymous Coward]

The rest of us disabled javascript.

Re:The new slashdot interface

[Culture20]

I turned it off and went with the old style. I'll take a 5 minute wait between posts any day before trying to
r
e
a
d
t
e
x
t
l
i
k
e
t
h
i
s
And yes, the posts lower in the chain would often look like that with the most recent javascript fiasco. I have no idea whether it's still that ugly.

Re:The new slashdot interface

[Teckla]

It actually drives me insane, it is markedly worse, I read less stories because of it (because I do not like the feel of the site so much).

The "Many more" button (to get more stories) has never worked for me. This is across multiple computers and operating systems. I read less Slashdot because I simply can't easily get to the older stories.

I'm sure it has something to do with my account setup, but -- bah. The new Slashdot is a train wreck.

Re:The new slashdot interface

[FlyingBishop]

Whatever they did recently made it bearable. (Though yes, I had a good two years where Slashdot was unbearable to use.)

Re:The new slashdot interface

[hankwang]

Try AvantSlash. It is a CGI script to be installed on your own web server that will reduce an article (100-200 kB of bloated HTML) to 20 kB of lightweight HTML that even my 2008'ish smartphone can handle. It supports dynamic comment expansion and new features are on the way in the beta version that I'm working on.

I also use it on my netbook these days, although not for posting.

Re:The new slashdot interface

[MMC Monster]

I think the problem is worse that you state.

A lot of people (myself included) used to read *and comment* much more often than we do now. We've given up. And since most people read /. for the good comments, the number of casual readers goes down because the number of comments are going down.

Re:The new slashdot interface

[Evtim]

Strange! /. always worked for me smooth as silk except very long threads can take some time to open. I am a lame Windows user and it worked just fine with XP+Firefox and now 7+Firefox.

BUT, on the phone (HTC desire) - tragedy!! Stock browser does not scale the text properly. Opera mini scales OK, but refuses to display more than the 250 comments. Nothing helps - open in new tab, open in new window, the same window - it just does not work. View more stories sometimes works sometimes not but never works from the first try. The Dolphin mini browser - almost impossible to log in and I have the feeling it is often because of the ads disable function - somehow the Dolphin does not like it so after log in I am back at the main page but not logged in. Even more strange - the issue is gone if I am on WiFi but on 3G it is back. It really sucks because I get quite some mod points and do take the responsibility seriously - I read ALL comments and try to pick the good posts that went under the radar. Since I spend 2 hours a day commuting with train, that was a golden opportunity to exercise the mod points. Alas, I tried once giving mod points via the phone and that also went bad.

I am not very good with computers and programming - am I missing something?

Re:The new slashdot interface

[BenoitRen]

Why is there no mass appeal to change it back / forward in a more reasonable (i.e., simpler) direction ?

Because no matter how loudly we complain, site management doesn't give a shit.

Re:The new slashdot interface

[Teckla]

Strange! /. always worked for me smooth as silk except very long threads can take some time to open. I am a lame Windows user and it worked just fine with XP+Firefox and now 7+Firefox.

My computer is a single core 1.66 GHz Atom, so I'm somewhat used to things being kind of slow, but Slashdot takes the cake. (Running Windows 7)

I am not very good with computers and programming - am I missing something?

I don't think you're missing anything. The new Slashdot is just poorly written.

Re:The new slashdot interface

[djdanlib]

It isn't.

Re:The new slashdot interface

[Culture20]

You missed a great opportunity to screw with my head:
I
t
i
s
n
'
t
:)

Re:The new slashdot interface

[djdanlib]

sor
ry;
)

VS2010 and smartphones

[MetalliQaZ]

I have definitely noticed addition latency in the UI with VS2010, and it has always bugged me. I can't be certain if it is VS itself or if the underlying WPF is to blame. My current belief, and I have no facts to back this up, is that the VS UI is simply much more complicated than the "typical" use case that was targeted for WPF, and as a result, low-to-mid range computers fail to meet a human's expectations of UI latency.

On a separate beef, why is it that so many people put up with the AWFUL latency on smartphones? Especially when browsing the web, it can sometimes become unusable. The phones get more and more powerful, but instead of making the last UI work, the designers just add more and more flashy crap instead. UI is just as slow but it looks cool in the ads.
When I don't have a real computer nearby, I can use the android browser when I need to, because I recognize the flaws and can patiently work around them. It kills me to watch my girlfriend attempt to navigate the interface of her nook color. Usually goes something like this...
1. press UI element
2. no response, press harder (because that's what humans do when a button doesn't work)
3. harder has no effect, start tapping repeatedly
4. UI finally changes, menu pops up and immediately goes away, having tapped on an unknown selection.
5. human gets mad, claims unit is broken, goes to check email on 5 year old PC.

As an added bonus, the PC doesn't require you to put your fingers in the way of what you're trying to look at.

-d

Re:VS2010 and smartphones

[Kral_Blbec]

How much of that is connection lag and how much is UI though? The connection is understandable, depending on signal strength and other variables. UI lag is not. If its lag in the browser its probably connection related.

Re:VS2010 and smartphones

[umghhh]

I hardly ever watch the TV (most of it is crap anyway and if I watch then I chose in advance and do not flip between channels). The phone I use has indeed lags sometimes but this has less to do with the languages or frameworks used but a lot with processors and batteries not being able to cope - not sure if they all still do it but they actually inserted slower processors into smart phones on purpose so that the battery lasts longer. Still I use my phone as a calendar, alarm clock and pin generator for my VPN connection. Occasionally I also take up calls from my boss but not all too often. Lag? what lag?

I also find the claims that java is slow preposterous - the problem lies in developers not having a clue or not having a choice and using legacy software while building fancy things on top of heavy and slow base applications. From this perspective java is to blame mainly because allows unskilled staff to fake good programming skills i.e. not code writing but thinking about how code is structured in advance. But of course I am a troll so mod me to hell for my half informed views.

Re:VS2010 and smartphones

[tepples]

How much of that is connection lag and how much is UI though?

If touching doesn't immediately produce a visual or audio response that the device is trying to load something over the connection, then it's perceived as UI lag. That's part of why Slashdot has that little "Working" box with a throbber that pops up at the bottom of the screen when I preview a comment.

Re:VS2010 and smartphones

[cnettel]

I don't know why, but my impression has been that the UI is actually more responsive in VS 2010 compared to 2008. My personal pet theory has been that GDI+ back-buffering can get slow on very high resolutions (2560 * 1600), while WPF would stack up better. That wouldn't conflict with your theory, though, just that there is some point where VS2010 will overtake 2008 when you have enough graphics power and enough pixels to blit.

Re:VS2010 and smartphones

[tepples]

The phone I use has indeed lags sometimes but this has less to do with the languages or frameworks used but a lot with processors and batteries not being able to cope

My Super NES with a 3.6 MHz 65C816 CPU doesn't lag when I play Super NES games. So why should a phone with a CPU 1000 times faster lag?

Re:VS2010 and smartphones

[schlechtums]

In my experience it is the underlying WPF that is to blame, and not just because the 2010 UI is so much more demanding than WPF was designed for. My work PC is a core 2 quad, not top of the line, but no slouch either, and I notice this input lag in other very small and simple WPF apps. In fact, in these smaller WPF apps the input lag is CONSTANT, while with VS the lag may only come once in a blue moon. If you minimize the window and then restore it, you will find the display updated. (Do it fast enough and you can convince yourself this minimize then restore made it update faster than if you had just waited for it to happen). I've not been able to figure out what the issue is, and it doesn't affect my VS often enough to warrant reinstalling everything as a troubleshooting attempt. (Googling has offered no help on this problem). It doesn't affect my home PC. I wonder if it has to do with the directx capabilities/implementation in the video card/video drivers? I'm really not sure, but the problem definitely lies with WPF.

Re:VS2010 and smartphones

[djdanlib]

The WPF interface is definitely not as responsive as the old Win32 GUI. The old GUI's code problems probably reached critical mass and they decided to rewrite, and chose WPF for whatever reason.

However, I find the text editor is nice and snappy for me. We beta testers definitely submitted a lot of feedback and MS actually had to eat their own dog food, which meant they pushed some updates to the framework itself to improve the situation. This was especially true with text rendering, which now looks more or less the same as non-WPF text rendering.

Could it be better? Yes. Is it at least serviceable right now? Yes.

Then again, WPF is GPU-accelerated, and if you are using software rendering or onboard graphics, you may be less inclined to get any kind of respectable performance out of it. It's the "2D Acceleration" feature from graphics cards in the mid-1990s all over again...

Slashdot

[Bobfrankly1]

I had an incredibly insightful comment, but I forgot it while waiting 2ms for the comment interface to load. I remembered and forgot it again during the 10 seconds it took the preview to render.

What the h&*)) is going on here?

[rickb928]

I read about this right here in January. And February.

Seriously, how many times can you recycle the same story with slightly (and I mean slightly) different nuances, but the same frakking story?

Next thing you know, I'll be reading another story on this with the angle 'women and children affected the most'.

Slashdot is becoming the USA Today of the Internet. Isn't it time for another site upgrade?

Re:What the h&*)) is going on here?

[antdude]

Nah, /. was bought out by USA Today. [grin]

strace / truss

[Rogerborg]

If you don't know what those are, or how to use them, or they're just not available on your platform of choice, then you're part of the problem.

Re:strace / truss

[Shin-LaC]

Modern platforms have dtrace.

Re:strace / truss

[drinkypoo]

Modern platforms have dtrace.

Which ones? I can only find it on stuff based on 4.3BSD and SVR4.

Skytopia article

[MobyDisk]

The first article in the summary has some silly ideas on how to fix this:

What can we do reverse the onslaught of latency in all its forms? ... First off, we can make LCD (or future OLED/QLED) monitors run at 120 or even 240 frames per second.

What?!?! 1) That is not possible and 2) That would not help. There are so many reasons for this I can't even list them all.

First, there isn't an LCD panel in existence that actually responds that fast. Those phony 240 hz screens don't actually change the pixels 240 times per second. We don't need faster displays to solve a software problem.

If the software provided a 1/60 second delay, that would be just fine. The issue is that we are not taking advantage of the refresh rates we have no - so increasing those rates doesn't help. Heck -- that's probably faster than the debouncing circuits in the controller!

Compensating for a longer pipeline by increasing clock speed doesn't help anyway, because to get the higher rate you need to increase the pipeline more...

Re:Skytopia article

[Mysteray]

Those phony 240 hz screens don't actually change the pixels 240 times per second.

Well your retina doesn't react instantaneously either, but your brain knows how to compensate for it. My personal theory is that the slow response of the LCD is also an analog or analog-appearing process and your brain will find it less offensive than other types of timing distortions.

But even if the LCD itself doesn't respond instantaneously, increasing the frame rate can potentially decrease every frame-count-denominated source of latency preceding them in the pipeline. By going from 33 ms/frame (30 Hz) to 4 ms/frame (240 Hz) you eliminate 14 to 29 ms worth of latency in even the shortest pipeline. Of course, if you have that much extra CPU it seems like you prevent the multiplier effect by runing the game logic at the higher rate and just throw away the extra frames without rendering them to the GPU.

But I'm not a real game developer or expert on these matters.

Re:Skytopia article

[Zuriel]

Compensating for a longer pipeline by increasing clock speed doesn't help anyway, because to get the higher rate you need to increase the pipeline more...

Where were you when Intel was designing Prescott?

Re:Skytopia article

[AdamHaun]

I think he's talking about the image processing. LCD controllers need to see multiple frames in advance so they can set the control signals right to minimize ghosting. This "response time" is a marketing number, so they try to make it as small as possible. Unfortunately, that means delaying one or more input frames plus some extra processing time. At 60Hz, one frame is 16ms, which is a large unit for input lag purposes. Changing to 120Hz means the next frame comes faster, which in theory means less lag.

I'm not sure whether that actually works, though -- it depends on whether looking 8ms ahead is just as good as looking 16ms ahead.

Re:Skytopia article

[epyT-R]

going from 60hz to 120hz on my crt helped latency considerably. however, I"m one of those old school quake heads who sees stuff like that. today's world of laggy GUIs and clunky virtualized binaries (java/.net et al) drives me insane...especially since the cpus we have today are roughly 10x faster than they were in 1998.

Re:Skytopia article

[X0563511]

That had everything to do with your brain being able to perceive the scanning (as a flicker) and absolutely nothing to do with latency.

Re:Skytopia article

[Mprx]

It has everything to do with latency. Most software takes more than one frame to react to input, so halving frame time noticeably reduces latency. I'm also an "old school quake head", and I'm just as sensitive to latency. I stuck with a CRT until true 120Hz LCDs became available, because they are the first LCDs that aren't an obvious step down from CRTs. Even so, in a side by side comparison the CRT has slightly better motion quality, so we really do need 240Hz LCDs.

Note also that CRT flicker is a good thing if you're displaying fixed framerate 60Hz content (and there's a lot of that out there), because it's the only way to avoid sample-and-hold blur without adding more latency and motion artifacts with interpolation.

Re:Skytopia article

[Anonymous Coward]

Some of the new 3D-capable LCDs support a real 120 Hz input video signal and can actually display it as well. They are out there, although they call a premium over the ordinary 60 Hz displays.

Re:Skytopia article

[Twinbee]

You missed the part in the article which explains how games (or indeed any software) which runs at 30fps tends to have twice the lag as games which run at 60fps. This is due to the rendering pipeline often used in games. The logic goes that if 60fps has x lag, then 120fps may have x/2 lag.

Re:Skytopia article

[Evtim]

If I may ask - does your "quake eye" see problems with flat TV's?

I have allegedly 400 Hz plasma (Panasonic) and I see blurring and lag all the time. But only when watching Blue-Ray, not DVD's or cable TV (digital). Could be my Blue ray player is too slow, but damn it, it was a new and rather expensive model at the time (got it middle of 2009). Also Panasonic. I have heard from colleagues that there seems to be quite some subjectivity in this - some people see the lag, some not. My wife never sees it for instance.

Also, since my PC CRT is about to die, which model 120Hz LCD you recommend?

thanks,
evtim

Re:Skytopia article

[MobyDisk]

I didn't miss that part. It is just an oversimplification and I didn't have time to explain it all. So let me clarify...

Suppose we have the following game pipeline:

1. Gather input from devices and update the game state.
2. Render based on that input and send it to the display.

We have 2 CPU cores each dedicated to one step in the pipeline. In the above example, the lag is 2 frames. Suppose the game runs at 30fps, so that is 2/30th of a second or 0.067 seconds. So the reader says "aha! We should run at 60fps and cut the lag in half!" The problem with that is the console can't keep up with 60 fps. If the console hardware was fast enough to run at 60fps, then the console would have done this:

1. Gather input from devices and update the state variables based on time and user input and render based on that input and send it to the display.

Which results in 1/60 of a second lag. We cut the lag in half, without increasing the frame rate of the display. We can't cut it any further, but 1 frame of lag is usually not a problem. Gamers start complaining when you get 2, 4, and 8 frames of lag. That's ridiculous.

Since the console was not powerful enough to do it all in one step, it took advantage of multiple CPU cores and traded latency for frame rate. This is an inevitable consequence of us not being able to manufacture faster CPUs, and instead manufacturing multiple parallel CPUs instead.

So since the above scenario isn't possible, what does the console do? Well, it could run the game at 30fps anyway, gaining nothing. Or it could add another 2 cores, and run a modified pipeline like this:

1. Get input
2. Update the state variables based on time and user input.
3. Render based on that input.
4. Send that rendering to the display (buffer swap).

Aha! Now they are using parallel processing and can render at 60fps! But the lag is now 4 frames, or 4/60 of a second which is no better.

So ultimately, frame rate is limited by CPU speed first and video frame rate second. Mandating that displays have higher frame rates won't do anything until the CPUs are fast enough to keep up. That won't happen until game console manufacturers and game developers see the need to do so. In the mean time, there are other sources of latency that aren't discussed in this post that are more important targets. Demand LCD panels that don't introduce extra latency. And don't use video upscalers that insert more lag. Don't use digital receivers that add another 30ms of lag or video-based motion capture that runs at even lower rates. Once you eliminate those kinds of things, then you can focus on video refresh rate.

Re:Skytopia article

[AdamHaun]

I think they're talking about the signal processing most LCDs do to reduce ghosting. Usually it involves looking at least one frame ahead plus some extra processing time, which gives 20+ms latency. Some monitors are really bad and get >50ms latency. This is very noticeable. In theory, a higher refresh rate would let you get more frames for processing in less time. I'm not sure if it's really equivalent, though -- it depends on whether looking 8ms (1/120sec) ahead is just as good as looking 16ms (1/60sec) ahead.

Re:Skytopia article

[Mprx]

No >60Hz TV support true high frame rate input, they only interpolate slower inputs. It's possible that your BluRay player is outputting 24fps but your TV only supports interpolating 60fps. It's also possible your BluRay player is converting the 24fps to 30fps before it reaches the TV, which will add its own motion artifacts.

I'm using a ViewSonic VX2268wm, but if I was buying a 120Hz LCD for the first time now I'd probably get a LG W2363D.

Re:Java or Visual Studio 2010 anyone?

[nomel]

I believe they were talking about the Visual Studio 2010 interface. If you've used something from some years back, you'll understand how insanely slow doing they've managed to draw text. I think it's all incredibly hilarious watching everyone squirm. It makes me feel better when I think I must be getting old because people seem to program without any thought involved...just library calls.

I coveed it in detail in this months column

[seifried]

I cover how to measure buffer bloat, recreate the problem (trivially easy, in my case a single high speed upload saturates my 3 megabit uplink and ping times go from 50-60ms to 1000+ms. http://www.linux-magazine.com/Issues/2011/127/Security-Lessons-Bufferbloat/%28kategorie%29/0

Re:I coveed it in detail in this months column

[geekoid]

Unless some one can give me a 'do that and you will see it' test that works I remain skeptical that buffer bloat is the cause.

Its sounds more like confirmation bias tied with a different issue.

If someone running a twitch game(Usually TF2), download bit torrent(WoW patch or ISO), on a computer (AMD Quade, 4 Gigs)that's may also be serving up media to my TV(10/100/1000) can't experience this, does it really exist?

I have 25/20 Mbps
.

Channels on each frequency

[tepples]

Since many of these technologies transmit the data for all channels simultaneously, why not just scan for key frames and store the last key frame received for each channel?

Only a few HD channels are transmitted on each 6 MHz carrier. Scanning for keyframes on other carriers would need multiple tuners, which increases the cost of the cable box.

Re:Channels on each frequency

[vlueboy]

Scanning for keyframes on other carriers would need multiple tuners, which increases the cost of the cable box.

It's not like defects cause them to toss your current faulty box; those get repaired and put back in another client's hands at the same $n/month rate of a new one until it breaks again. In the PC world, that is unacceptable.

Need more cash? sure, but don't raise my prices. I pay each up several times and if you think the free replacements they offer to turn in your old cablebox to get new models (incorporating the newest invention of the decade, such as cablecard, digital signal, HDTV, HDCP, pay per view, tvguide channel, scrambling) it is only to benefit the cableco: less data transmitted, tighter DRM, 10 minute cold boots. Cable service in New York is not pleasing to use with either one of the two choices. They both license boxes from Scientific Atlanta, so moving to a different area won't solve your problems.

Ugh. yea, I'd like the more expensive box for better quality of life --I'm already paying too much and I might as well enjoy yesteryear's speeds while I'm at it.

not so, sadly

[nu1x]

Uh, no ?

Taken from /. faq (if you have disabled js, trying to access some functions):

"Why does "This Function Require JavaScript?"

Welcome to the now, man!

Some elements of Slashdot's UI require the use of Javascript. In most cases we've provided backwards compatibility for the more paranoid folks in our crowd that are fearful of executing unknown code within their browsers, but sometimes it's just not practical to maintain a second UI for this. Right now this includes the customizable section menus, tagging, and a great number of user interface customization options. We know it's a potential security hazard, but sometimes you just have to jump out of the airplane to get that adrenaline rush. Try it sometime. Pack your own chute tho. "

Re:Java or Visual Studio 2010 anyone?

[sycomonkey]

Also Java and Visual Studio tend to be used by less skilled developers and students (disclosure: i'm a student). Poor responsiveness of programs written in Java or using VS is more a factor of who is writing it than anything to do with the language / VM / IDE.

Lightning

[formfeed]

Sometimes in a thunderstorm you see lightning, but it can take several seconds till you hear thunder.

Pretty bad design, imho.

Re:Lightning

[linear a]

Good design. They are pretty much simultaneous to the intended "user" of the lightning.

Re:Lightning

[Mprx]

Human brains turn 24fps into barely tolerable motion, or in the case of fast motion, completely unacceptable motion. Go watch some Showscan. Arguments about "film look" are just sour grapes used to justify inferior technology.

Re:Um, stuff's slow. Make it faster.

[PhrostyMcByte]

This isn't so much about computers doing things too slowly, or trying to compute too much. It's about computers sitting idle at the wrong times, or computing things too early or too late. It's about buffering -- which trades latency for throughput -- making it more and more difficult to coordinate those timings correctly.

In gaming you have screen lag, video card lag, render lag, input lag, and network lag. They all add up, and synchronizing their timings to make things appear fluid and lag-free is so difficult that most games end up leaving some not-so-corner cases uncovered.

Quality of gaming has been measured in FPS for so long that video drivers have been "optimized" to transparently buffer 3 or more frames if you turn vsync on. I've seen this alone destroy many unsuspecting games.

Re:Java or Visual Studio 2010 anyone?

[lgw]

Wow, I've seen no input lag on VS2010 at all - and I run it in a VM which is hosted on my slow-ass laptop, so it's pretty starved for resoruces. I wonder what I'm doing right here?

Really

[geekoid]

"(Java or Visual Studio 2010 anyone?"
Really? do you even know what the hell you are talking about? OR did these two think pop into your skull and you use your meaty finger to pound out some sort of text in a vain effort to stay relevant?

Replace:
Java or Visual Studio 2010 anyone?
With:
Crappy programmers.

And has anyone documented a repeatable real world test for 'bufferbloat' or is this still an academic issue?

Re:Really

[Wolfling1]

You're only partly right.

Poor programming practices are a part of the problem, but we're also suffering from the architectural and design impositions enforced by M$ for years.

Regardless of whether you're programming in a Windows environment or not, Microsoft's insidious 'Abstraction to the point of distraction' policy has influenced development languages and environments.

Whilst abstraction creates more elegant and maintainable code, it is entirely overhead. It is there to help our feeble minds comprehend what's going on.

The steady push towards Java, C++ - even C# are all crafting a generation of programmers who can't cut multi-tier code. All they know to do is to push the buttons on the black-box - even if that black-box bottlenecks their entire application. Its easy to point the finger at them and say 'they're crappy programmers'. They are, but we made them that way.

Personally, I think that Java and VS2010 are really good examples of black-box programming being stuffed down the throat of the world's code-cutters. And it has created a culture where IT managers expect same day results. Not many programmers out there are prepared to give up their job to stand up to an IT Manager who wouldn't know a latency issue brewing until it bites them on the ass.

Re:Really

[FrootLoops]

I think the VS2010 reference is in part to the IDE, which has tons of small delays. I just compiled a tiny .NET 4 WPF app (~100 lines and almost no interface), restarted my computer, and launched it. It took several seconds for anything to happen. Presumably various libraries were being loaded as starting a far more complex WPF app was virtually instant afterward. In any case, some delays have nothing to do with the app programmer.

Re:Really

[Chirs]

And has anyone documented a repeatable real world test for 'bufferbloat' or is this still an academic issue?

Yeah, bufferbloat is real and easily reproducible. Run the Netalyzr test: http://netalyzr.icsi.berkeley.edu/

Re:Really

[schlechtums]

What this guy said, except on my work machine the input lag is constant in WPF apps, not just the first time they are run. (If you minimize then restore the window, however, it gets updated immediately). Luckily VS2010 seems to have the least input lag, but it does happen from time to time, and it is the exact same behavior as the input delay in every other WPF app.

This doesn't happen on my home machine, so I can only speculate it has something to do with the video card/driver implementation/whatever of directx.

Re:Really

[jabelli]

Maybe I'm missing something, but who cares if the latency is due to sitting in the buffer for 1500 ms or retrying a dropped packet for 1500 ms?

Re:Not just Slashdot's JavaScript.

[The End Of Days]

Awww, was the troll afraid to put his name to the lies?

Impatience...

[MaWeiTao]

This is one of the reasons why on all the devices and computers I use I disable animations when I'm permitted.

I think the real problem, however, is that we've grown increasingly impatient. I notice it in myself. Some application takes over 3 seconds to open and I start getting agitated, wondering what the hell is taking so long.

Think back to the earlier days of computing when it might take 30 seconds or more to get an application started. I recall on my PCjr booting up games and sitting there for a good minute while the drive ground away. That said, some games nowadays have some rather appalling load times. But look at what has to be loaded compared to those games from the 80s.

I also recall trying to play games that were more demanding than my computer could handle. I'd hit a key and get an action a fraction of a second later. I got used to it, but it's something that would be intolerable today.

Even using early versions of Windows and Mac OS was an exercise in patience, especially if something was going on in the background. And what about waiting for a simple website to load, especially over 56kbps? Hell, what about loading up text-only menus on your favorite BBS via 2400 baud?

The simple fact is that we've been spoiled and as such have grown increasingly impatient over shrinking timescales. That said, given all the computing power at the tips of my fingers I do expect everything to be instant.

Re:Impatience...

[shish]

Think back to the earlier days of computing when it might take 30 seconds or more to get an application started

Back when I wrote one of my first gui programs, I added sleep() calls into the load process, because otherwise the splash screen disappeared before you saw the progress bar move and that wasn't so "cool"...

Re:Impatience...

[epyT-R]

we aren't spoiled. we're more dependent. in the 80s you used your computer to play simplistic video games and maybe type a one page school assignment.. if you were learning programming, you were coding simple routines in assembly, maybe C if you were lucky enough to have a 68000 based machine with sufficient ram to run a full suite of tools..even then you were still coding a lot of assembly.

Today, programmers are pumping out bloated binaries using point-and-stick development environments. this problem is additive, starting from kernelland up to the internet. if your system libs are bloated and laggy, your apps will be too. the reason we're 'impatient' is that most people expect computers today to be fast enough to not lag on basic things like launching a simple application..of course a simple application today is at least 50MB on the disk, with a 100MB commit. that same program in 1998 would be 5MB. both have 99% the same functionality. this code bloat has to do with the way programmers are being trained to think about th ings. I'm sorry but there's no reason a web browser should take 100s of megs of ram to render a few web pages. there is no excuse. the whole stack is a bad design and that's just dhtml. java and .net are another whole example..the latency in those applications is insane.

today it is still possible to find well coded applications that have tiny footprints but offer great functionality. it's just getting harder to do because of said point-and-stick environments that colleges pass off as programming nowadays.

Re:Impatience...

[Dhalka226]

As you eluded to at the end of your post, it depends on circumstances.

My program takes over three seconds to load. Well, okay. Did it used to take that long? If not, why is it taking that long now? Is it the Windows slow-down effect? Did some program update bloat the software up, or introduce worse code? Is my computer busy doing something else, and if so, is it because I asked it to or because one of those other things is true of some other program or yet another piece of software saw fit to pump four badly-coded "helper" apps into my startup that I neither want nor need? Some of these things should--rightly--piss me off; others not so much.

My game isn't responsive? Should it be? How old is my hardware? Am I above the requirements? Input lag, video lag, Internet lag?

Waiting for pages to load on 56k was annoying, but it was expected. If I'm loading at 56k speeds with a 8Mb down connection, shouldn't I have the right to be impatient (assuming there aren't other major transfers going on sucking the bandwidth of course)?

In other words, I'm okay with things being laggy if there's a good reason for them to be laggy. If they're laggy because people write shoddy code or bloat their applications or webpages up with loads of unnecessary crap, usually installed for THEIR benefit and not mine, then yeah, I get impatient. I think that's justified.

Re:Impatience...

[BetterSense]

Our computers are orders of magnitude faster than they were decades ago. Why should our programs be orders of magnitude more bloated? In fact, it seems quite a coincidence that applications STILL take several seconds before launching....why not 30 seconds or 1/30th of a second? The answer is that if it launched in 30s, the programmer would realize that's unacceptable, and if it launched in 1/30th of a second, the programmer would bloat the application until it no longer did.

At work, I use a variety of online applications, GUI Java applications, and of course *soft productivity software, But the fastest and most downright blazing piece of software that I use on a daily basis is the '80s vintage IBM 3270-based framework that runs our factory.

It's amazing how much computing has advanced without improving.

Re:Impatience...

[swilver]

I blame users with ridiculous demands, like demanding that every application be "themable" etc...

Re:Impatience...

[WuphonsReach]

Think back to the earlier days of computing when it might take 30 seconds or more to get an application started. I recall on my PCjr booting up games and sitting there for a good minute while the drive ground away. That said, some games nowadays have some rather appalling load times. But look at what has to be loaded compared to those games from the 80s.

Back with the PCjr, you could *hear* the drive grinding away. I think that had a large part of why it seemed acceptable at the time, that you could tell something was happening. Modern systems are a lot quieter, so if there's no visual indicator on the screen, it's harder to tell whether something happened or not.

(All of which is why I've switched to either 10k RPM SATA or SSDs for my primary drive. The SSD puts the bottleneck of the system back on the CPU and really helps with program loading / responsiveness.)

Re:Impatience...

[Rich0]

Yup, at work everything is moving to sharepoint and we're still stuck on IE6. I click on tabs and literally wait 10-15 seconds for all the scripts to finish running. This is on a dual-core laptop - a few years old, but really. Granted, the full-disk encryption and McBloatee running all the time can't be helping either.

Linux isn't always better. The other day I noticed that my linux box was using 750MB of slab memory. Yikes! I'm not even running a tainted kernel any longer. I used to run my server with only 1GB of RAM...

Re:Java or Visual Studio 2010 anyone?

[geekoid]

"program without any thought involved...just library calls."

That's actually a good thing. Software will only progress when basic routine things are just called..

And there is a difference between using libraries and not thinking. You use know measurements and techniques to build a bridge, you don't recreate everything from scratch all the time.

Re:Finally!

[Culture20]

And then things took a turn downhill, I could speculate why but I doubt we'll ever really know.

It happened right about when you purchased a plasma TV, right? There you go.

Re:Um, stuff's slow. Make it faster.

[drinkypoo]

Read what he has to say about Bradley Manning and you'll wet your kecks.

Remember when...

[devphaeton]

.....I want to say it was around 2003 or so around here, when there were all sorts of discussions about "the cost of hardware and bandwidth is so cheap now, we don't need to optimize for the machine, we should instead optimize for the programmer". This came up again and again, and all the n00b programmers danced around their shiny Powermacs and sang and held hands and ratified it.

Notice how everything has exploded exponentially in size, and each revision of any bit of software has been noticeably more sluggish than the last?

This is why. Don't say I didn't warn you.

Dell Axim X5

[jo7hs2]

One key example I can come up with was the external folding keyboard for the Dell Axim X5 I bought as an "upgrade" from a Handspring Visor I was using for note taking in undergrad. The input delay was so bad that you could type several paragraphs before they would appear in the device. Totally destroyed the device for me, and I ended up going back to paper, as the Handspring was no longer in my possession. Not to focus on Dell, but my Inspiron 8000 had a similar problem, too. I got it as a desktop replacement when I started college, it had one of those early GeForce 2 GO chipsets, so it was capable of playing a game or two. Sadly, there was an odd latency to the internal keyboard that evaporated with an external, just bad enough to make FPS games essentially unplayable.

Re:Um, stuff's slow. Make it faster.

[Omnifarious]

That's why I made him a foe in the first place. Not Bradley Manning per-se (he's a traitor, one I happen to not be all that upset with, but still), but with Wikileaks in general.

Re:Java or Visual Studio 2010 anyone?

[Lunix Nutcase]

What about the interface? I use it for hours on end daily on a 3 year old laptop and I notice absolute zero lag in drawing or anything else. Maybe you need to ditch the 15 year old dumpster-dived computer?

Re:Java or Visual Studio 2010 anyone?

[Lunix Nutcase]

Visual Studio tend to be used by less skilled developer

Since when?

Re:ffs

[Americium]

Well I only have a ipod touch 4, so it's doesn't do calls, but everything else is quick. My N900 doesn't do any UI smoothly, it looks like 24fps video.

Re:Java or Visual Studio 2010 anyone?

[clockwise_music]

Also Java and Visual Studio tend to be used by less skilled developers and students (disclosure: i'm a student). Poor responsiveness of programs written in Java or using VS is more a factor of who is writing it than anything to do with the language / VM / IDE.

Java and Visual Studio less skilled developers? Nope. No it doesn't.

But thank you for your disclosure, that explains a lot.

Re:Java or Visual Studio 2010 anyone?

[clockwise_music]

Visual Basic 4.0 starts up in three seconds on a ten year old computer. I have much more advanced IDEs on my machine nowadays, but if I just want to code up something for the fun and expect quick results, I'll still fire up VB4 from 1995.

Wow, really? Maybe you should try using notepad and vbc.exe to compile. Super quick!

iPhone

[sqrt(2)]

Input lag is something I seem to be more sensitive to than most people. It is is my main, daily, frustration with my iPhone. I have a 3G, one of the last ones sold new, and although it was never fast even the first day I got it, it gets slower and slower with each iOS update. I can type out whole sentences before the screen reacts and shows the words, which usually results in mistakes. If I go slower to allow it time to display each keystroke I get a miserably slow, and I've measured this, 1 character (not word, CHARACTER) per second. Imagine the ticking of a clock, that's how fast I can input letters, any faster and it's unable to keep up. If I hold the phone in portrait view and then rotate it to landscape it can take as long as 12 seconds to respond. Restoring the phone doesn't help, this is normal behavior from the moment I reflash the firmware.

Starting an app can take 10-15 seconds, longer for more intensive ones. Google Maps (an Apple provided app) is one of the worst, it can take 25-35 seconds before I am able to begin inputting a destination. Moving around the map view is jerky and slow, and requires waiting after moving every little bit for it to redraw. When I use the iPod app, it takes 7-8 seconds before the controls become responsive after the now playing screen appears. I can sit there pressing pause or play and it just waits...and waits...and waits...and then decides to accept my press of the button.

Re:iPhone

[rayd75]

Boy, have I been there... That phone was almost entirely unusable on iOS 4.0. I actually missed calls because it wouldn't respond to my touches fast enough to answer. It was incredibly frustrating to deal with crap like that and the keyboard literally hanging for 15 or 20 seconds with one key highlighted. It improved with 4.1 and 4.2 but it's still ridiculous. Thing is, it's not just CPU horsepower or memory at work. Those make it more noticeable on the 3G, but since upgrading to an iPhone 4, I still notice tons of latency. Sure, apps load like lightning compared to the 3G, but stuff that should always be instantly responsive like navigation bar buttons, scrollviews, and the keyboard simply aren't. Latency is a far bigger issue on an iPhone 4 running iOS 4.3.x than it was on an iPhone 3G running OS 2.x. (its release OS version)

Re:iPhone

[sqrt(2)]

I think it's because of Apple's development cycle. I feel like the only version of iOS actually made with any intention of running well is whatever version ships when a new iOS device revision is launched. Older models get updates to the newest version, but once launch is over Apple is already starting on the next model and all future iOS updates will have that models hardware specs as the baseline for testing performance, then if it works good enough on the other hardware versions great, release it for them too until it gets too bad that it simply won't install (3G won't get anymore updates, for example).

So the iPhone 4 is the newest iPhone, but it's not the newest iOS device (the iPad 2 is, and is faster than the iPhone 4), so all future updates to iOS will assume iPad 2 level hardware or greater. As you've said, you're already seeing latency even on an iPhone 4, and it'll only get worse when the iPhone 5 comes out and the newest version of iOS assumes your phone is at least that good.

That's just my interpretation of the Apple development cycle I have no way to confirm this except speculating based on what I see in the field.

The Insidious Creep of Latency Hell

[Jeremi]

I believe that is the alternate job title for the Chief Technical Officer at AOL.

Re:Not just Slashdot's JavaScript.

[Surt]

If I hadn't posted already I could mod the gp up. Javascript is a significant misfortune for the world of computing.

Re:Finally!

[smellotron]

It happened right about when you purchased a plasma TV, right? There you go.

Really? In all of the research I did, I found that the worst latency came from LCD TVs, not plasmas. It tended to be due to image "enhancement" algorithms that are more popular with high-end LCDs than high-end plasmas. Check out this "official" plasma input lag thread. Those guys do some pretty serious tests.

Re:Finally!

[Culture20]

I should have just said flat panel. The big problem for most gamers is the loss of the CRT.

It's all in engineering

[tibit]

The latency problem in non-networked applications is ultimately caused by poor software engineering, starting with system-provided APIs. Most of "bog standard" system libraries were designed for something entirely different than what they end up used for. The "normal" C I/O paradigm is used everywhere, yet it was really designed for batch applications, not for interactive use. The only way to do almost any filesystem and network interaction should be to submit a request, then react when the results come back, all the while being receptive to other "results" (events) coming in. Unfortunately, designing things this way requires a certain discipline and a mindset, and default APIs and "industry practices" simply don't encourage it at all.

A correctly engineered system API should not have any blocking calls apart from the "wait for events" call, it's as simple as that. It's very rare that an application is only waiting for one thing to happen. Even something as simple as a UNIX cat has two file descriptors open, and simultaneously waits for stuff to come and and for stuff to finish going out, with a buffer in-between (I'm ignoring no-copy APIs for a moment). Coding it up as a read followed by a write is, at best, wishful thinking. Of course event-based programming is something that seems like a lot of extra work, but it's just a matter of getting used to doing things the right way.

In fact, if you decide to code up your whole system in an entirely reactive way, you gain other benefits. By reactive I mean you could reduce every application thread's interface to a single processEvent(event), run-to-completion function that you implement. As it turns out, it becomes almost natural to get the guts of the processEvent() function implemented as a state machine. The state machine formalism often helps in producing better quality code, and it certainly makes it very easy to trace interaction with the outside world. Miro Samek shows a striking example: the supposedly "so simple it couldn't possibly go wrong" calculator example from old Visual Basic has several bugs that stem from its bug-prone yet commonplace design. The calculator's state is spread out in an ad-hoc manner in various variables, and the tests done on those variables in response to external UI events pretty much amount to a buggy reconstruction of a single state variable to drive a state machine.

The state machine paradigm is in somewhat stark contrast to the way a typical GUI application is designed, where you have on_fooBar methods that get invoked when fooBar event happens. In the fooBar method it's up to you to verify that the application is in the correct state to do whatever fooBar calls for. This requires forethought, and status quo indicates that it's easy to get it wrong. Perhaps that's the reason: the de-facto mode of implementing reactions to external events is so broken that it's not used for much besides the GUI. Perhaps this is why "quick" system calls are usually done in line and end up blocking the whole application, or at least one thread, and those are not free either so why waste them with blocking APIs?! Apart from perhaps querying the current time or current username, there are really no "quick" system calls. Simple things like listing a directory or getting a key's value from the registry can potentially take seconds if your drive is thrashing around due to high I/O demand, or if the network happens to be slow.

Of course the line has to be drawn somewhere, so let's assume that paging of code, libraries and heap is something we should not worry about because it cannot be helped much. At that point one realizes that indiscriminate memmapping of data files can be problematic in itself: a memory-mapped file is, after all, supposed to hide the fact that you are doing a request-response that can be either very fast or very, very slow. The latter is something you should explicitly handle, and with memmap it's at best cumbersome: you have to use some API to check if given page is av

Re:It's all in engineering

[tibit]

As for the example of cat, which IS implemented as a read followed by a write (at least on FreeBSD and Gnu), in practice, this works better than you might think, because the Linux and FreeBSD kernels may prefetch reads in anticipation that you will call read again, and isn't required to have written to disk (though it may be for other output channels) when it returns from the write syscall, it is only required that subsequent reads (that are proven to occur afterward) return the written data, in essence, the caching of the data has been moved out of userspace and into the kernel, for cases where caching is appropriate. It should be obvious why someone would not want an async_io cat if you think about the cases where you would use cat between two sockets (not files), you can't write if you haven't read anything, and you shouldn't read if there's nowhere to write to, because then you'd have to allocate memory, and stack up a read buffer (of what size?), and any fault that occured at the output would not be propagated synchronously to the input socket. If you want a cat that violates the semantics of POSIX cat, please don't call it cat.

Prefetching of course only applies to block devices. What I'd expect of cat or dd is to have enough of a buffer to prevent stalls -- to cushion latency of source and destination devices. This should work automagically, within some limits. Of course you shouldn't indefinitely read if there's nowhere to write to, but perhaps the destination is fast on average, but has latency peaks? This can be quite cheaply characterized at runtime, each and every time. Think of a tape drive. A tape drive that normally streams megabytes per second will easily stall a TCP/IP socket on a T1 link when it has to reverse the tape due to loss of streaming. So you end up with an underutilized T1 link as the network stack will not buffer much more than a second or two of data (not for me, at least). All this even though on average T1 can only push about 140kbytes/s, while the tape drive should "cope" with more than an order of magnitude more.

What you of course correctly highlight is the very sad shape of system APIs, and that includes POSIX. There's very little consistency. This is to a point where the internal kernel synchronization APIs are usually cleaner than what the userspace has to live with. On Linux and Windows, AFAIK if you have kernel code that waits for something to come over the network, or from a USB device, or from the drive, the API used to wait on "somethings" is exactly the same. In the userspace, as you point out, there's no consistency, and there's generally no joy to be had from dealing with non-blocking APIs. I claim that the bad shape of those APIs is partly responsible for the refusal of toolkit and application developers to do the right thing. In this area winapi is somewhat better than POSIX, in fact, and that's just sad.

Re:Finally!

[X0563511]

You -did- try the calibration in the options menu, right? The one that lets you tune the latency compensation?

Real-Time Languages for User Interfaces

[inglorion_on_the_net]

I've long been thinking about using or creating and using a real-time language for user interfaces. What I have in mind is a language where you can specify time constraints as part of the contract for functions, and the compiler will then enforce those by refusing to emit code if that code would take longer to run than permitted by the time constraints.

Obviously, there are some hurdles here: not every computer is equally fast, many operating systems introduce unpredictable timing, and so on, but I think it would be interesting to at least see how close we could get. I really think the user experience would be much better if at least the user interface always responded quickly to user input.

Re:Real-Time Languages for User Interfaces

[swilver]

That's not needed at all. This article makes it sound like it is impossible to create a fast responding UI. Good practices will result in a responsive UI.

Re:Java or Visual Studio 2010 anyone?

[Twinbee]

I was mainly talking about the GUI from both VS and the apps that Java tends to create.

Re:Java or Visual Studio 2010 anyone?

[AlphaGremlin]

I'd venture a guess that you're not using:

• TFS - lag on every operation that gets worse the more files you have checked out, and the larger your source tree is. I've had it take several minutes to rename a single file. As soon as you do a check-in, suddenly it's back to instantaneous.
• Database projects - lag that gets worse the more complex the project is. Save a file, wait 20 seconds. Save a lot of files, you might as well go to lunch. Don't even think about referencing other projects. And if Visual Studio crashes, you lose an hour whilst it regenerates the database model.
• Silverlight projects - Start getting to a decent size, and you'll endure multi-second lags every time you switch tabs. Make that double if the tab is a XAML designer. And the build times are glacial.

I should note these experiences were all on a quad-core, 8gb of RAM 64-bit version of Vista, but we had developers on XP and Windows 7 finding exactly the same.

Thank goodness I don't have to deal with Visual Studio any more. It's fine for small projects, but as soon as you start having large, enterprise-scale applications, the constant freezes and lag become unbearable.

Mouse controls

[Sky Cry]

I have similar problems with mouse controls in some games. For example, in Fallout: New Vegas the mouse moves by several pixels no matter how little and how slowly you move it physically. Overall the mouse controls make it an extremely poor shooter when compared to many FPS games. Luckily the game has a lot of other things going for it, but if it was a pure shooter, I wouldn't play it for even 5 minutes.

Channel crawling

[Drakkenmensch]

Imagine hitting "channel up" on your remote control and waiting... waiting... wondering if the infrared signal made it to the receiver or was blocked somehow, only to finally see the channel change a full minute later. It's my daily experience with my digital cable company.

Re:Finally!

[Evtim]

Amen! I dread the day when my 22 " Sony CRT will die (it's coming, I can see the sings). Should have bought 2-3 monitors at the end to see me through the next 2 decades. After that I would be too old for gaming anyway.....I did such a preemptive purchase with the Seinheiser PX 100 head phones. Those are brilliant for their price range. No, they are better than many models that are twice as expensive. Fortunately Seihn kept them going, but there was a period when all stock was depleted....

Mobile Phone Voice Latency and HDTV Audio Latency

[AJ Mexico]

Latency is particularly annoying when you are trying to have a two-way conversation. Talking over a mobile phone adds 100s of milliseconds of latency. If both ends are mobile, the latency is double. This escapes most people's direct notice, but it causes those awkward moments when both parties start talking over each other, pause, and then do it again. This is a horrible amount of latency, and they virtually all do it. This is one of the reasons to still prefer a land line.

Another unbelievable latency issue is HDTV. The audio often fails to be synchronized with the video. It is particularly annoying when the video is behind the audio. This happens frequently on all types of telecasts. I have spoken to a TV station engineer about this, and incredibly, there is no standard way to ensure that the audio and video of HDTV are in sync. Someone at the station can make an adjustment for a particular program, but the offset can be completely different in the next program.

I see this often on direct reception via antenna. No telling what weird latency problems are added when the signal is transmitted through a cable or satellite system.

HDTV in general has at least two or three seconds of latency compared to analog. When you're watching a "live" event, you're not.

Individual TV receivers introduce varying latencies. If you have multiple TVs set to the same channel, you can see and hear this. That ought to be a published spec on TVs, as well as the other gadgets mentioned in the article.

Re:Java or Visual Studio 2010 anyone?

[lgw]

Ahh, good point. I avoid all of thiose things. Especially: everything that involves XML usage by VS has been impossibly slow since they switched over to XML formats (and I can't understand how they could have done XML parsing so slowly - boggles the mind).

VS is a great editor and debugger (even in a large shop), but the related source control/build environment is useless at scale. I don't understand why people use it that way - never seen it done right, maybe?

Re:Finally!

[Twinbee]

I was very wary about buying an LCD monitor a couple of years back due to latency and off-angle viewing. But the one I ended up with (Hazro 24" S-IPS) works a treat. Unless you notice lag even more than I do, something like that should be fine. Make sure you get an IPS panel though as the cheap TN ones are crap. Alternatively, you could wait for Blue Phase mode LCD or the upcoming OLED TVs which should correct all problems (including blurring).

Hardforum.com or avsforum.com is probably a good place to start your research as they know a bit more about lag compared to most.