Slashdot Mirror
Controlling Bufferbloat With Queue Delay
CowboyRobot writes "We all can see that the Internet is getting slower. According to researchers, the cause is persistently full buffers, and the problem is only made worse by the increasing availability of cheap memory, which is then immediately filled with buffered data. The metaphor is grocery store checkout lines: a cramped system where one individual task can block many other tasks waiting in line. But you can avoid the worst problems by having someone actively managing the checkout queues, and this is the solution for bufferbloat as well: AQM (Active Queue Management). However, AQM (and the metaphor) break down in the modern age when Queues are long and implementation is not quite so straightforward. Kathleen Nichols at Pollere and Van Jacobson at Parc have a new solution that they call CoDel (Controlled Delay), which has several features that distinguish it from other AQM systems. 'A modern AQM is just one piece of the solution to bufferbloat. Concatenated queues are common in packet communications with the bottleneck queue often invisible to users and many network engineers. A full solution has to include raising awareness so that the relevant vendors are both empowered and given incentive to market devices with buffer management.'"
The Internet is not getting slower. It is becoming laggier. Comeon people, learn the difference.
Wonder what the public key field is for?
Today, there is no incentive for an ISP to consider spending money on this. For their private customers, they sell QoS, which guarantees their customers a better queuing method. Extremely profitable. For consumers, it makes sense to simply continue investing in infrastructure. Adding capacity from the street to the CO not only eliminates the issue, but also allows the ISP to provide better, more profitable services. In short, we will likely see better queuing methods integrated with future routers. The may be one of them, but only time will tell, and nobody will discard all of their equipment today to get it. The issue is just too minor while capacity remains cheap and QoS profitable.
Yep, same cause. They attempted to minimize packet loss by increasing the buffers in their network. The user experience was horrible.
http://blogs.broughturner.com/2009/10/is-att-wireless-data-congestion-selfinflicted.html
Is the internet getting slower? (laggier)
because the simplest pages are HUGE BLOATED MONSTROSITIES!
Between flash and ads. And every single page loading crap from all around the world as their 'ad partners', hit counters, click counters, +1 this, like this, digg this, and all the other stupid social media crap that has invaded the web. All this shit that serves no purpose other than to some marketers. And EVERY SINGLE PAGE has to have a 'comment' section and other totally useless shit tacked on as well.
Just this little page here on slashdot. With less than a dozen replies. Tops 80k so far. And that's with everything being blocked that can be.
slower? laggier? no... the signal to noise raito is sucking major ass.
We all can see that the Internet is getting slower.
Can we? It looks like it has been getting faster to me....
According to researchers, the cause is persistently full buffers,
What researchers? What buffers? Server buffers? Router buffers? Local browser buffers? Your statements are so vague as to be useless.
and the problem is only made worse by the increasing availability of cheap memory, which is then immediately filled with buffered data.
Buffering is a way of speeding servers up immensely. Memory is orders of magnitude faster than disk, and piling RAM on and creating huge caches can only help performance. I call bullshit on your entire claim. This summary is so awful, I don't even want to read whatever article you linked to.
HA! I just wasted some of your bandwidth with a frivolous sig!
We all can see that the Internet is getting slower.
Can we? I'd suggest that most people are unaware of any such trend, perhaps because it has happened too gradually and too unevenly. Indeed:
A full solution has to include raising awareness so that the relevant vendors are both empowered and given incentive to market devices with buffer management.
Exactly. Consumers don't know or care about low latency, so the market doesn't deliver it (that plus lack of competition among ISPs in general, but that's another kettle of fish).
We need a simple, clear way for ISPs to measure latency. It needs to boil down to a single number that ISPs can report alongside bandwidth and that non-techies can easily understand. It doesn't need to be completely accurate, and can't be: ISPs will exaggerate just like they do with bandwidth, just like auto manufacturers do with fuel efficiency, etc. What matters is that ISPs can't outright make up numbers, so that a so-called "40 ms" connection will reliably have lower average latency than a "50 ms" connection. That should be enough for the market to start putting competitive pressure on ISPs.
What kind of measure could be used for this purpose? Perhaps some kind of standardized latency test suite, like what the Acid tests were to web standards compliance? Certainly there would be significant additional difficulties, but could it be done?
The boundary they are transiting is one between a fast network and a slower network, similar to what you see at a head-end at a LATA or broadband distribution point and leaf nodes like peoples houses, or one the other end, on the pipe into a NOC with multi gigabit interconnects much bigger than the pipes into or out of the NOC.
The obvious answer is the same as it was in 1997 when it was first implemented on the Whistle InterJet: lie about the available window size on the slow end of the link so as to keep the fast end of the link from becoming congested by having all its buffers filled up with competing traffic.
In this way, even if you have tasks which would otherwise eat all of your bandwidth (at the time, it was mostly FTP and SMTP traffic), you can still set aside enough buffer space on the fast side of the router on the other end of the slow link to let ssh or HTTP traffic streams make it through. Beats the heck out of things like AltQ, which do absolutely nothing to prevent a system with a fast link that has data to send you crap-flooding the upstream router so that it has no free buffers to receive any other traffic, and which it can't possibly hope to shove down the smaller pipe at the rate it's coming in the large one.
Ideally this would be cooperatively managed, as was suggested at one point by Jeff Mogul (which is likely barred due to the lack of a trust relationship between the upstream and downstream routers, if nothing else). Think of it like half your router lives at each end of the link wire, instead of both sides living on one end.
It's the job of the device on the border who happens to know there's a pipe size differential to control what it asks for from the upstream side int terms of the outstanding buffer space it's possible for inbound packets to consume (and to likewise lie about the upstream windows to the downstream higher speed LAN on the other end of the slow link).
I'm pretty sure Julian Elischer tried to push the patches for lying about window size out to FreeBSD as oart of Whistle contributing Netgraph to FreeBSD.
While people are looking at that, they might also want to reconsider the TCP Rate Halving work at CMU, and the LRP implementation from Peter Druschel's group out of Rice University.
-- Terry
My InterTubes are BETTER because I HAVE ZERO LOSS!!!
Oddly enough such a business model turned out to be unsustainable due to
(1) it's finanically expensive (between one thing an another)
(2) doing this the less expensive way (ie by slathering on bigger buffers) introduces excessive latency (for some customer designated value of "excessive")
For the life of me I don't understand how ANYBODY can be allowed to run a company without at least vaguely understanding the concept TANSTAAFL.
And, finally: No that Hot Blonde Supermodel with MASSIVE Bazingas does NOT find you attractive, not in the slightest, no matter how much she may drink/snort/inject or pop.
If you want to fix throughput issues without spending lots of money or sacrificing latency then you're going to need a beter algorithm (yes folks, hard research and careful thinking).
Visit CryptoGnome in his home.
Slashdot already has the equivalent: ACM (Anonymous Coward Management).
They also have ATF (Assisted Troll Flagging) as a kind of belt-n-suspenders thing.
Visit CryptoGnome in his home.
It's not clear from the paper whether packet dropping is per-flow, in some fair sense, or per link. There's a brief mention of fairness, but it isn't explored. It sounds like the new approach has no built-in fair queuing.
Without fair queuing, whoever sends the most gets the most data through. Windows (especially) starts up TCP connections by sending as many packets as it can at connection opening. There used to be a convention in TCP called "slow start", where new connections started up sending only two packets, increasing if the round trip time turned out to be good. That was too pessimistic. But Windows now starts out by blasting out 25 or so packets at once. This hogs the available bandwidth through everything with FIFO queues.
If the routers at choke points (where bandwidth out is much less than bandwidth in, like the entry to a DSL line) do fair queuing by flow, the problem gets dealt with there, as the excessive sending fights with itself, trailing packets on the biggest flows are sent last, and everything works out OK.
"Bufferbloat" is only a problem when a small flow gets stuck behind a big one. A flow getting stuck behind the preceding packets of the same flow is fine; you want those packets delivered. Packet reordering is better than packet dropping, although more computationally expensive. Most CIsco routers offer it on slower links. Currently, this means links below 2Mb/s, which is very slow by modern standards. That's why we still have kludgy solutions like RED. This new thing is a better kludge, though.
We all can see that the Internet is getting slower *Citation needed* Have you tried turning your modem off and on again?
http://www.awfullybigmoustache.com
My first thought after reading the story was 'Hope whoever patents those ideas doesn't charge too much for them."
I want a list of atrocities done in your name - Recoil
Correction: There is no get-rich-quick scheme with a high probability of success. There are a few (like the lottery) which may get you rich quick, but with only a small probability.
Mathematically optimal too, providing all customers/packets take equal time to process. The only problem is that in the real world it requires awkward physical queue layouts.
Yeah, it is irritating when you arrive at the airport at five o'god in the morning to catch a flight leaving at nine, and as the only customer, you have to walk a labyrinth a mile long to get to the way too cheerful check-in assistant who is patiently waiting.
Net result: Added latency.
Then you hit the security line, which is already full, presumably by people who spontaneously spawned from the walls between check-in and security. And in the security line, this one line approach does not appear to help speed at all.
It's a part of the solution but not a complete solution. Boundary problems are different from center-of-network problems. Playing with TCP window sizes only works well at the edges and only in the outgoing direction (the 'inflight' sysctls that we've had forever), but does not completely solve the problem because you always need to add one or two additional packets above and beyond the calculated sweet spot to absorb changes in latency from other links and give the algorithm time to respond whenever reality changes.
This solution in both incoming and outgoing directions suffers from a connection multiplication problem. That is, it works fine if you have only a few simultaneous TCP connections running but it breaks down when you have dozens or hundreds due to the need to have 1-2 extra packets of slop in the reported window. It degrades gracefully in the outgoing direction but blows up very quickly when you are trying to control bandwidth in the incoming direction by changing the window size you report available in the outgoing ACKs (anyone running torrents can tell you this but the problem occurs for any busy network).
However, bandwidth limiting in this fashion DOES reduce packet backlogs and queues significantly. Not enough (it's simply impossible to make the algorithm stable at the sweet spot so you always need 1-2 additional packets), but significantly. Hence it is part of the solution.
Step 2: On speed boundary changes you have to run fair queuing, period. Not only that but you have to do it on both sides of the boundary (i.e. in both directions each at the choke point). In my case I could never get truly reliable operation by only running fair queuing in the outgoing direction. I had to actually run the fair queue on *both* ends of the link, meaning I had to colocate a server to serve as the terminus for a VPN and run all the traffic over the VPN so I could control both ends. The fair queue also has to reserve bandwidth for pure TCP ACKs to prevent restricting the bandwidth in the opposite direction due to ACK starvation.
Fair queuing takes care of all remaining packet buffering issues at the edges of the network.
Center-of-network issues can't use the above solutions simply because there are too many connections flowing through the center of the network to track and not enough packet buffer space to sufficiently buffers all those connections for fairq operation (N x B is just too big). It's impossible to calculate where the choke points are based on connection tracking at the center-of-network. Easy at the edges, impossible in the center.
So the center-of-network has to provide additional congestion control through some sort of AQM, and if early-warning requests go unheeded it must start dropping packets. Personally speaking I hate the idea of having to drop packets, it leads to all sorts of problems everywhere, but the simple fact of the matter is that there is not enough per-connection buffer space at the center of the network to run fairq, nor is it an appropriate place for that. Without tracking you cannot mess with tcp window sizes in the center-of-network and even if you could track you can't bundle the connections based on where the choke point is at the moment, there is simply not enough information.
So we are talking at least three and probably closer to half a dozen different mechanisms being needed to reduce network latencies and still provide good and fair performance.
-Matt