Maximum Latency for ISPs?

fluor2 asks: "My ISP is providing me 8mbit ADSL, and my speed is in fact 8mbit (downstream). However, we all know that there is no relation between transfer rate and latency, eg, a high transfer rate and high latency will kill your FPS games. A packet that travels through the sky and up to a satellite is bound to give high latency. Using pathping, I discovered that my ISP provides me with a latency of 22ms before my sent packets are sent out of my ISP's backbone (6 hops). I have a friend that also tried the same, and he got only 10ms before he was out of his ISP's network. I know 22ms is decent, but I still think that it's far too high if one uses IP-phones and similar. What kind of latency can we accept for a normal 8mbit ADSL connection, and isn't it about time that we get more focus on this subject?"

Comments

CAP or DMT?

[austad]

What transmission scheme are they using? With CAP, you can expect lower latencies, around 10ms I think. However, most telcos are switching to DMT because I think it's more scalable. Unfortunately, DMT gives crappy latency, I've seen 60ms in some cases.

22ms latency to leave your ISP's backbone is actually quite good for DSL.

Featurewise, most cable modems are crappy, but their latency is better than DSL in most cases.

As far as VoIP goes, 300ms will still give good results. Some codecs don't play nicely with high latencies, but I've used VoIP with a 600ms latency satellite link, and it worked just fine. The latency on your TDMA or GSM phone is several hundred ms, just call another cell phone from yours and put one up to each ear and talk, there's almost a second delay.

Re:CAP or DMT?

[ManDude]

Telcos normally aim for 150ms or lower. It isn't a problem for local, but cross-country + trans-ocean it can be a bit of a problem. When I call the UK you can catch bad switching. One second delay is brutal since it ends up being 2 seconds for return. I have ended a conversation saying "bye" more then thrice :) Anything like 22ms + a few 10s would be ideal.

Re:CAP or DMT?

[doogles]

Some codecs don't play nicely with high latencies,

It's my belief this is a myth.

VoIP itself cares little about one-way delay, but cares a whole lot about jitter. If I can provide you with a one-way link that has extremely high one-way delay, but I have routers on each end of the link to ensure voice gets queued and transmitted before any other data in the queue, the service will be acceptable. The only piece that may be unreasonable to users is the delay itself; the ITU standard is 150ms of one-way delay (300ms, as you mention, would be a correct "round trip" time assuming delay in each direction was the same).

High delay can lead to users talking over one another; we're not use to this when we call granny down the street. But for remote locations or international calls, they are use to extremely high delays and so taking their call across an extremely high-delay path (such as satellite, as you mention) results in no net difference for the end-user. Yes, VoIP endpoints will add some of their own delays, but these will be fairly insignificant if you're talking about a 500ms one-way delay budget.

I have a collegue with a customer who has 100+ sites about Alaska, in VERY rural areas. All their voice calls go across an satellite-transported IP network. Sure, _you_ might have a little trouble getting use to it at first, but the regular users are use to high delays on their calls (much like your cell phone, as you pointed out). In the grand scheme of things, I would argue that as long as the packets arrive jitter-free (meaning there are not huge inter-frame gaps, which would mean there is nothing for the far end codec to decode and play out) the quality of the call itself will be acceptable.

In the end, though, I think we both agree -- I just don't know that I've ever run in to an environment where delay was the cause of "poor voice quality". Loss and jitter will typically be the root cause for why codec isn't producing the quality you'd expect. Delay is just, well, delay.

Cable pings better than DSL

[xWeston]

Cable modems generally ping better than DSL for whatever reason, and I'm sure even fatter dedicated lines are better as well.

On my cable modem (adelphia) I get 10-12ms for the first 8 or so hops as they are all on the adelphia servers, after that I can get as low as 20ms or even 18ms for more local stuff (I get about 23 to www.yahoo.com). I live in San Diego and this type of service is only about $35/month. On my DSL (Pacbell) I used to get 15-20ms to the first hop even, whereas i get 9-11ms now.

Re:Cable pings better than DSL

[afidel]

WHAT??? your clock better be more accurate than ms's or your PC isn't going to be doing JACK. Linux IP stack is accurate to hundredths of a millisecond, although at that point you are measuring delays in packet processing in the NIC and kernal =) Trust me on our Gig-E network all the windows servers show 1ms to anywhere in the LAN, but Linux can tell me the real times and I can often tell how many switches I am going through =)

ISP's

[BrookHarty]

Verizon has 2 networks in our area, one is a T1 (fijitsu)based, the other is T3 (westell) based dsl modems.

I was on the fitjistsu on the 768/128, about a 33ms ping to the seattle bbnplanet backbone, I moved down the street, and they put in the new higherspeed network. 1500/384 and 10ms to the bbnplanet backbone.

USwest back in Spokane was about 15ms on a 768/768 cisco modem.

While I find Verizon and other telcos to be better bandwidth and ping, smaller mom and pop ISP's tend to oversell. Speakeasy was would be choice if the telco is oversold, and earthlink if ISDN is your only choice. Thou small ISP's do re-sell ISDN cheaper, and ping is good enough for multiplayer games, 20ms+. (Remember its different for each user and location!)

I'd check out dslreports and ask other people in your area. And networks change from city to city, cable/dsl/isdn/frame all depend on the routers and hop count. Plus if your ISP is a peering partner with local ISP's, they connect all major ISPs locally, thats a plus. Sometimes you notice crazy routing, like Seattle to California and back to go across town to an ISP without a local peering agreement.

Also, you call your ISP, and ask them to do a traceroute from their network to a gameserver and email it to you. I've asked this from hosting services, and who they having peering agreements with. Some will even give you a network diagram or have them posted on the site, like Verio. (Who while expensive, does seem to have good peering agreements.)

Re:ISP's

[cdrudge]

Verizon has 2 networks in our area, one is a T1 (fijitsu)based, the other is T3 (westell) based dsl modems.

The actual difference between the two is the Fijitsu is frame-relay based and the Westell is ATM.

count yourself lucky

[elmegil]

I had to fight and fight to get my ISP to take seriously my demand that the first hop be less than 50 ms or I was going to find someone else. See, I went with a provider that I thought was a local ISP who turned out to just be reselling service from...halfway across the country. So, I get the ATM link from here (Oak Park) to my CO, but I'm positive that that gateway router is in Virginia. If I wanted to give the business to my ILEC, I could probably do better, but as long as it's 50ms or less I can live with it. If I changed to Ameritech I'd probably have to give up my static IP and unblocked ability to have a small web server too.

Latency and Throughput

[blate]

Your post states that latency and throughput are unrelated. For TCP connections (FTP, HTTP, IMAP, POP, and many games), this is absolutely not true.

The maximum possible throughput of a TCP connection is one "window" of data per round-trip time. The "window" size is essentially the amount of unacknowledged (ACK'ec) data that can be outstanding. This is often called the bandwidth-delay product, I think.

What you need to take away from this is that even if you had infinite bandwidth between you and your peer, the throughput of a single TCP connection is upper-bounded by the delay product. For example, if your window size is 32KBytes (I'm going to use 32,000 to make the numbers prettier) and the round-trip time is 100ms, then you can transmit (or receive) at most 32KB * 10 = 320KB per second. To go faster, you have to either increase the window size (which consumes more memory) or decrease the round-trip time (which is sometimes impossible, since the speed of light is a constant, or so my physicist friends claim).

A couple other points.

You're probably not capable of noticing the difference between 10ms and 20ms in terms of response time for interactive applications, including online gaming. if it were 10ms vs 100ms or 200ms, then yes, but 10ms is less than one refresh interval on your monitor, so you really can't "see" the difference.

As far as VoIP (IP telephony) and other multimedia network applications are concerned, again, you must consider the end-to-end latency (one-way delay) and/or the round-trip time, not the latency between you and some arbitrary router at your ISP.

The phone companies spec their systems (or so I've heard) such that the *round trip* latency for a domestic call is always less than or equal to 100ms. We're talking POTS here, not cell service, which experiences higher latencies.

I work on VoIP software; in an IP call (both ends are IP clients), it's very hard to keep the *one way* latency below about 100ms, if you're lucky, even if both clients are on a LAN. This is because you have to have various buffer and jitter compensation delays so that the sound quality is acceptible under somewhat adverse network conditions. In a typical call across the internet, 200ms one-way latency, IMHO, would be considered quite good.

So your 20ms intra-ISP latency (vs. the 10ms that your friend reports) is in the noise.

Oh, I should also mention, for completeness, that packet loss (or even reordering, which is more common that you may realize) can *really* hurt both TCP and VoIP (which usually uses UDP) performance/quality. This gets into some messier technical issues... basically, though, if your DSL isn't lossy, and you're getting 20ms intra-ISP latencies, you're doing as well or better than most of us.

Your friends who are running on 56k modems, who eat 200ms just to get their packets to the ISP's router on the other side of the PSTN are really going to be hurting :)

ADSL vs. SDSL

[EinarH]

Have you considered changing your ADSl into either a SDSL or VDSL service?
Some of those ISP's that offer ADSL have started to offer SDSL or VDSL. VDSL is currently very expensive in my area and only people within a short distcance from a telephone central can get it. SDSL is more flexible when it comes to max distace. Most people on SDSL get lower ping.

When I got my new connection I could either choose between 1024/512 ADSL at $85 or 1024/1024 at $140.
A bit expensive, but I get my own permanent IP, no pay per GB thing, can have my own servers etc.
And I can't complain at the latency, since many of the other users on the ISP are offices and bussiness whom almost only use their computers at office hours I get very low latency. Approx. 15 ms. to many CS-servers and the same to a backbone.

So I'm happy, but I still gaze at the connection of a friend of mine. He just got a VDSL 12500/6250 at $227. Officially, According to their User Agreement he cant't resell but the ISP is not that strict on it so he allready has 10+ customers... ;-)

Re:Spoiled

[admbws]

It may be obvious, but have you put your name down on BT[uk incumbant telco]'s Demand Tracking Scheme? It is almost certainly worth it - because they won't upgrade your area until people do (it does work!). They have also been working on extending the reach of ADSL so you may be able to get it if and when your area is finally activated.

If that fails, get yourself an ISP with their head screwed on (i.e. not a big one) and have a chat with them about it. They can ask BT to supply it on an if-it-works-it-works basis (I know a few people who have "passed" this way).

But for now, find yourself an ISP and register your interest!

Hope this helps.

Free equivalent

[0x0d0a]

You're looking for the excellent mtr.

Believe me, there isn't anything you can do on a network in Windows that you can't do better in Linux.

8mb dsl latency

[BTF]

22ms is definitely good. Especially leaving your isp.

Dsl speeds won't affect latency too much. I know it's not supposed to, but it does.
A 1184/160 dsl will ping around 15ms to its gateway
A 1728/384 dsl will ping around 11ms to its gateway
A 3488/800 dsl will ping around 7ms to its gateway

I have a feeling its more related to the upstream speeds than the downstream. An 8MB dsl has an 800 upstream maximum so the pings will most likely be the same as a 3MB dsl. Isp's can have different upstream speeds for all downstream speeds.

All these speeds are assuming your isp doesn't put you on interleaved channel which gives you 55ms+ ping to gateway. And if you have a decent isp further hops will only increase by 1-3ms a hop.
As for my experience with several cable providers ping to gateway is 6ms. But all cable providers i've had go to hell during peak periods. I know this is not the case for everyone. When i originally had cable 5 years ago i had no problems, then again no one else had cable... And i enjoyed 20ms in game.

10ms might not sound like much but to a gamer it does :)

router equipment

[blosphere]

Your 22ms is not that bad... but you can get it down to 10ms with fast-path. I was able to squeeze out to 7-9ms (on first hop), but then I have heavily tweaked my own dslam profile (used to work for an dsl provider).

The other thing is, that you shold really only be interested in end-to-end RTT, not the individual hops. For example, if there's cisco 4xxx series switches with SUP-3's out there, your icmp/traceroute IP packets gets processed in the processor card, not on the interface, causing an 10ms more latency to pings/traceroutes. The actual IP packets of your connection get forwarded by the interface so no latency there.

Re:Pathping in 'nix

[glenstar]

pathchar! It takes a bit of time to complete, and is not terribly user friendly, but it is pretty precise.