First: In-Q-Tel is the venture capital arm of all of the U.S. intelligence services, including DHS, FBI, etc; not just CIA. DHS, for example, will be blamed for any big security disaster; you should not presume that the motives of the agencies are uniform. Nor is all of what those agencies do bad.... It's the pervasive surveillance we *must* stop, and compromising our security standards. See: https://www.iqt.org/about-iqt/ for In-Q-Tel rather than the Wikipedia entry for Dan.
Second: Dan has never taken a security clearance, over his entire career.
Third: He's actually not a In-Q-Tel employee, but a consultant (full time) for them. This is so that he does *not* have to sign a employee agreement, but can remain able to speak freely. Which he does regularly: See http://geer.tinho.net/pubs for some of his publications. One I sparked him to write recently is: http://geer.tinho.net/geer.lawfare.15iv14.txt in reaction to the information I cover in my Berkman Center talk you can find at: https://cyber.law.harvard.edu/events/luncheon/2014/06/gettys
Fourth: people who know Dan, who is really one of the founders of the computer security field, hold him in very high regard and trust, as I do.
If you look at Dan Geer's career, rather than jumping to unfounded, ill informed presumptions based on news reports that don't bother to go beyond reading the Wikipedia entry, you will find:
1) he managed the development of Kerberous at Project Athena (where I got to know him)
2) he co-authored the famous Microsoft is a dangerous monoculture paper a bit over a decade ago (which Microsoft hated so much they
got @Stake to fire him.
3) he is a holder of the USENEX Flame award https://www.usenix.org/about/flame
You can get a packet from your worst enemy, and it's ok. The path it took to get to you doesn't matter. If you need privacy, you encrypt the packets at the time of signing.
The day that CDE finally appeared (badly late) on my workstation was the day that I knew there was no hope for the UNIX desktop. Design by committee never works, and it was a camel of 5 humps.
Since all broadband connections have bufferbloat (to some degree or other), in all technologies (fiber, DSL and cable alike), it isn't a good idea to volunteer to run an NTP server on such a connection, even if it is/has been reliable. Bufferbloat will induce transient bad timing into your time service; even more fun, in often a asymmetric way, pretty much any time you do anything over that link.
- Jim
AQM's don't usually look at the contents of what they drop/mark.
We expect CoDel to be running on any bulk data queue; voip traffic, properly classified, would be in a independent queue, and not normally subject to policing by a CoDel.
While 10 years ago, a decent AQM like CoDel might have been able to get latencies down for most applications where they should be, browser's abuse of TCP, in concert with hardware features such as smart nics that send line rate bursts of packets from single TCP streams has made me believe we must also do fair queuing/classification to get the latencies (actually, jitter) where they need to be due to these "bursts" of packets arriving.
The article's subtitle is: "A modern AQM is just one piece of the solution to bufferbloat." We certainly expect to be doing fair queuing and classification in addition to AQM in the edge of the network (e.g. your laptop, home router and broadband gear). I don't expect fair queuing to be necessary in the "core" of the network.
I'll also say that an adaptive AQM is an *essential* piece of the solution to bufferbloat, and a piece we've had no good solution to (until, we think, now).
That's why this article represents "fundamental progress".
You are correct that replacing one bad constant with another is a problem, though I certainly argue many of our existing constants are egregiously bad and substituting a less bad one makes the problem less severe: that is what the cable industry is doing this year in a DOCSIS change that I hope starts to see the light of day later this year. That can take bloat in cable systems down by about an order of magnitude, from typically > 1 second to of order 100-200ms; but that's not really good enough for VOIP to work as well as it should. The enemy of the good is the perfect: I'm certainly going to encourage obvious mitigation such as the DOCSIS changes while trying encourage real long term solutions, which involve both re-engineering of systems and algorithmic fixes. There are other places where similar "no brainer" changes can help the situation.
I'm very aware of the research over a decade old, and the fact that what exists is either *not available* where it is now needed (e.g. any of our broadband gear, our OS's, etc.), and *doesn't work* in today's network environment. I was very surprised to be told that even where AQM was available, it was often/usually not enabled, for reasons that are now pretty clear: classic RED and derivatives (the most common available) require manual tuning, and if untuned, can hurt you. As you, I had *thought* this problem was a *solved* problem in the 1990's; it isn't....
RED and related algorithms are a dead end: see my blog entry on the topic: http://gettys.wordpress.com/2010/12/17/red-in-a-different-light/ and in particular the "RED in a different light" paper referenced there (which was never formally published, due to reasons I cover in the blog posting). So thinking we just apply what we have today is *not correct*; when Van Jacobson tells me RED won't hack it (which was originally designed by Sally Floyd and Van Jacobson) I tend to believe him.... We have an unsolved research problem at the core of this headache.
If you were tracking kernel changes, you'd see "interesting" recent patches to RED and other queuing mechanisms in Linux; this shows you just how much such mechanisms have been used, that bugs are being found in this day and age in such algorithms in Linux: in short, what we have had in Linux has often been broken, showing little active use.
We have several problems here:
1) basic mistakes in buffering, where semi-infinite statically sized buffers have been inserted in lots of hardware/software. BQL goes a long way toward addressing some of this in Linux (the device driver/ring buffer bufferbloat that is present in Linux and other operating systems).
2) variable bandwidth is now commonplace, in both wireless and wired technologies. Ethernet scales from 10Mbps to 10 or 40Gps.... Yet we've typically had static buffering, sized for the "worst case". So even stupid things like cutting the buffers proportionately to the bandwidth you are operating at can help a lot (similar to the DOCSIS change), though with BQL we're now in a better place than before.
3) the need for an AQM that actually *works* and never hurts you. RED's requirement for tuning is a fatal flaw; and we need an AQM that adapts dynamically over orders of magnitude of bandwidth *variation* on timescales of tens of milliseconds, a problem not present when RED was designed or most of the AQM research of the 1990's done. Wireless was a gleam in people's eyes in that era.
I'm now aware of at two different attempts at a fully adaptable AQM algorithms; I've seen simulation results of one of those which look very promising. But simulations are ultimately a guide (and sometimes a real improving insight): running code is the next steps, and comparison with existing AQM's in real systems. Neither of these AQM's have been published, though I'm hoping to see either/both published soon and their implementation happening immediately thereafter.
So no, existing AQM algorithms won't hack it; the size of t
There are significant networks that do not look like the consumer edge Internet, one of which reportedly collapsed in a nasty way (not necessarily in the same way as 1986). Don't presume that the network that may collapse is the global internet (though time based self synchronizing phenomena are a worry there). One of the functions of AQM algorithms is to ensure that TCP flows don't synchronize their behavior. And those AQM algorithms are MIA on many networks today.
Those of us who lived through the 1986 congestion collapse are somewhat worried.
That we don't know is what worries us, as we're flying in an area we don't fully understand.
To solve this, I think we need both AQM fully at the edge of the network, and some sort of "fair" queueing at the edge. The headache is that the classic AQM algorithms won't work in the edge case (and are flawed). "Fairness" is in the eyes of the beholder and a complex question. You may consider it "fair" your kids get half the bandwidth you do, for example. But having a situation where talking to a local CDN 10ms away gets tons more bandwidth than something across the globe may not be "fair" in your view, nor that your web browser can put a horrible transient into the queues as it opens 10 TCP connections simultaneously, and it's unfair what it does to other people sharing your home circuit.
1) Some sort of AQM is necessary to tell the end points to slow down in a timely fashion (by signalling the end point TCP's). Or the buffers fill, and stay full, and you are in fundamental trouble. Consider this the "high order" bit to solving the problem. 2) TCP does not guarantee any "fairness" between flows of different RTT's, Note that what you consider "fair" inside your house is your business: your ISP has some obligation around "fairness" between customers. This is the next bit of precision. Anyone who thinks TCP is "fair" by itself believes in magic that does not exist.
Note these two issues might be addressed by one algorithm, or multiple algorithms, and what is best might be different in a host than in your home router, and different yet again elsewhere in the network. Exactly what we need (and will work well) is as yet unknown, though almost anything (even going back to semi-sane buffer sizes selected with even trivial amounts of thinking) can improve things a lot. And getting there is going to require analysis and testing. I encourage people to help out: for example, we've not yet really played with SFB to see how it behaves in the face of variable bandwidth. Classic RED won't work in a sane fashion at all in that case.
But hugely oversized, dumb, unmanaged, tail-drop buffers have got to go.
Thankfully, at the edge of the network (your host and home router), we have lots more cycles to play with per packet than in a core router and can burn some of them well for this purpose. And that's where we're almost always congested (your ISP may have other congestion problems, but at the aggregation level of those devices, classic AQM algorithms can function, even if maybe not ideally).
The article that was posted yesterday is the short CACM article; Kathy and I have a much longer paper in preparation that will go into this in more detail. But CACM has a word count limit we could not meet, and the "fairness" topic is mostly on the cutting room floor, and now being picked up and finished. I wasn't expecting the Queue posting to go "live" until next month (i was being naive), so the long paper is not finished. CACM really wanted to lead off January with a bang, so redirected our efforts toward the short, and in many ways incomplete, discussion.
You can have an overall congested network. I've seen this on occasion.
But it is very easy (and even more common) for you (or people in your house) to do it to you, than to have the overall ISP network congested. This is something a simple file copy can/does do to you, in practice.
Some ISP's run AQM properly (e.g. RED) in the cores of their networks; some do not. On the ones that do not, you'll see problems at peak hours. Similarly on corporate networks.
Heh. The worst problems (as far as I've seen) we have are in the edge.
In our hosts and home routers (most of which are Linux boxes).
The problem occurs anytime you are next to a bottleneck, and the "municipal and/or corporate resistance" have now built out the core of the internet to the point that the problem is most severe at the very edge, including your laptop/handheld device.
*EVERYONE* has made the same set of mistakes. I have done so too at times....
Yes, and since the BDP is not knowable usually (neither the bandwidth or delay are predictable these days in many/most circumstances), we have to be smart (need AQM).
The network is operating in a regime it wasn't intended to... And we lack the instrumentation we need to understand how the internet is operating.
The buffers are now so large that we've defeated the basic congestion avoidance algorithms (slow start and congestion avoidance). Technically, TCP congestion avoidance is operating, but the time gets so long that TCP thinks the path changed, and probes more aggressively for a new operating point, as explained in the article.We are flying in a piece of the flight envelope that has not been tested. That was a big surprise, when we dug into the traces.
That should make us nervous.
We've also had one credible report of a significant network that collapsed and was very difficult to restart. We don't know exactly what all happened (and hopefully never will; we just are frustrated we don't have packet traces).
Re: 1. I've always thought that the congestion window to the same end-point should be shared: but that's not the way TCP implementations work, and wishing they worked that way won't make the problem go away. And, as I've shown, bufferbloat is not a TCP phenomena in any case.
Re: 2. HTTP is a lousy protocol in and of itself, and having to do it on top of TCP makes it yet harder. It is the fact that HTTP is so ugly that makes so much else difficult. And I disagree with your claim that high latency links won't use the bandwidth; in fact, lots of sessions is just making things harder. You can read our HTTP/1.1 Sigcomm performance paper.
I'll be writing more on this topic this week.
And we need to replace HTTP, and something other than TCP would be highly desirable. Personally, I'm much more fond of CCNx than any IP based transport.
Bufferbloat is the cause of much of the poor performance and human pain using today’s internet. It can be the cause of a form of congestion collapse of networks, though with slightly different symptoms than that of the 1986 NSFnet collapse. There have been arguments over the best terminology for the phenomena. Since that discussion reached no consensus on terminology, I invented a term that might best convey the sense of the problem. For the English language purists out there, formally, you are correct that “buffer bloat” or “buffer-bloat” would be more appropriate.
I’ll take a stab at a formal definition:
Bufferbloat is existence of excessively large (bloated) buffers into systems, particularly network communication systems.
Systems suffering from bufferbloat will have bad latency under load under some or all circumstances, depending on if and where the bottleneck in the communication’s path exists. Bufferbloat encourages congestion of networks; bufferbloat destroys congestion avoidance in transport protocols such as HTTP, TCP, Bittorrent, etc. Without active queue management, these bloated buffers will fill, and stay full.
More subtlety, poor latency, besides being painful to users, can cause complete failure of applications and/or networks, and extremely aggravated people suffering with them.
Bufferbloat is seldom detected during the design and implementations of systems as engineers are methodical people, seldom if ever test latency under load systematically, and today’s memory is so cheap buffers are often added without thought of the consequences, where it can be hidden in many different parts of network systems.
You see manifestations of bufferbloat today in your operating systems, your home network, your broadband connections, possibly your ISP’s and corporate networks, at busy conference wireless networks, and on 3G networks.
A very long time ago, in a galaxy far, far away (MIT, mid 1970's, when I was an undergraduate and a member of MIT"s Planetary Astronomy Laboratory of that era), I remember having conversations with Mike Gaffey about asteroid mining. I see a reference to Technology Review on asteroid mining from Mike in 1977, so I think this got all published; I don't have any TR's of that era around to refresh my memory.
I remember one interesting scheme, where you might take a m-type metallic asteroid (which is mostly iron, nickel, and other useful metals) to earth orbit, by any of a number of propulsion schemes (solar sail, ion engine, or the like). It would probably take a number of years to move it from the asteroid belt to earth orbit. Then foam the asteroid (use solar mirrors to make it molten, and inject gas), and shape it into a lifting body. Then you would fly it into the earth's atmosphere, and land it in the ocean outside any port you would care to deliver it to. The point of foaming it was to reduce its density so that it would reenter the earth's atmosphere without much heating and ablation (we don't want to dump lots of metal into the earth's upper atmosphere), and float when you landed it.
Then you take a tug boat and pull it to a dock, and you have however many kilotons of metal you like. And without the huge energy cost of mining and environmental problems on earth.
As I remember, all the physics work (without having to invent fundamental new technologies), and there are lots of metallic asteroids. Now we just have to figure out how to actually do it. And it is way, way easier to deal with getting to and from the asteroids than the moon or any planet.
- Jim
And Chuck is building hardware which is often, or even usually, running Linux right now.
Track down the research project that's doing big programmable multiprocessors at Berkeley/Stanford/MS research and others.
The x86 instruction set is too baroque to fit in a sane number of gates, so in fact most of the software on that hardware is free and open source software.
His contributions are inspiring; in fact playing with an Alto so many years ago was the first time I got to mess with a graphics display and mouse, if only on an occasional basis for a few hours.
And I had a chance to work with Chuck a bit: he's great people, and has continued to do first class stuff ever since.
I live about 22 miles outside of Cambridge, where I have often worked. So that is 44 miles/day@ $.5 per mile (U.S. government reimbursement). Your actual costs will vary; but the government rate isn't far from reality. Parking is about $20/day in Cambridge; sometimes more, sometimes less depending on the lot.
$5500 - Mileage $5000 - Parking
Round numbers for automobile commute: $10,500
Note that there are hidden costs of road maintenance, etc.
Additionally, it is my time; on the commuter rail, at least I get (at least) an hour of my time back.
And the following data: http://w3schools.com/browsers/browsers_os.asp is very different (4x higher).
And I can argue both sides of these numbers: people learning web technologies may be more or less likely to be Linux users, statistically.
So right now, we're seeing data that is based on marketing: the web site with the best marketing department has the most widespread results (and who knows who pays their marketing budget, to be paranoid?).
Only Google and similar organizations know "for sure", and even then *they* don't due to embedded uses of Linux, POS systems, and the developing world where traffic statistics will be undercounted.
At best, you can use these data sources for *trends*, and not absolute numbers.
Slashdot Mirror
First: In-Q-Tel is the venture capital arm of all of the U.S. intelligence services, including DHS, FBI, etc; not just CIA. DHS, for example, will be blamed for any big security disaster; you should not presume that the motives of the agencies are uniform. Nor is all of what those agencies do bad.... It's the pervasive surveillance we *must* stop, and compromising our security standards. See: https://www.iqt.org/about-iqt/ for In-Q-Tel rather than the Wikipedia entry for Dan.
Second: Dan has never taken a security clearance, over his entire career.
Third: He's actually not a In-Q-Tel employee, but a consultant (full time) for them. This is so that he does *not* have to sign a employee agreement, but can remain able to speak freely. Which he does regularly: See http://geer.tinho.net/pubs for some of his publications. One I sparked him to write recently is: http://geer.tinho.net/geer.lawfare.15iv14.txt in reaction to the information I cover in my Berkman Center talk you can find at: https://cyber.law.harvard.edu/events/luncheon/2014/06/gettys
Fourth: people who know Dan, who is really one of the founders of the computer security field, hold him in very high regard and trust, as I do.
If you look at Dan Geer's career, rather than jumping to unfounded, ill informed presumptions based on news reports that don't bother to go beyond reading the Wikipedia entry, you will find:
1) he managed the development of Kerberous at Project Athena (where I got to know him)
2) he co-authored the famous Microsoft is a dangerous monoculture paper a bit over a decade ago (which Microsoft hated so much they
got @Stake to fire him.
3) he is a holder of the USENEX Flame award https://www.usenix.org/about/flame
In short, guys, he's one of "us"....
Don't be ill-informed slashdotters....
*All* content is signed in CCNx by the publisher.
You can get a packet from your worst enemy, and it's ok. The path it took to get to you doesn't matter. If you need privacy, you encrypt the packets at the time of signing.
The day that CDE finally appeared (badly late) on my workstation was the day that I knew there was no hope for the UNIX desktop. Design by committee never works, and it was a camel of 5 humps.
Jim Gettys
Since all broadband connections have bufferbloat (to some degree or other), in all technologies (fiber, DSL and cable alike), it isn't a good idea to volunteer to run an NTP server on such a connection, even if it is/has been reliable. Bufferbloat will induce transient bad timing into your time service; even more fun, in often a asymmetric way, pretty much any time you do anything over that link.
- Jim
AQM's don't usually look at the contents of what they drop/mark.
We expect CoDel to be running on any bulk data queue; voip traffic, properly classified, would be in a independent queue, and not normally subject to policing by a CoDel.
While 10 years ago, a decent AQM like CoDel might have been able to get latencies down for most applications where they should be, browser's abuse of TCP, in concert with hardware features such as smart nics that send line rate bursts of packets from single TCP streams has made me believe we must also do fair queuing/classification to get the latencies (actually, jitter) where they need to be due to these "bursts" of packets arriving.
The article's subtitle is: "A modern AQM is just one piece of the solution to bufferbloat." We certainly expect to be doing fair queuing and classification in addition to AQM in the edge of the network (e.g. your laptop, home router and broadband gear). I don't expect fair queuing to be necessary in the "core" of the network.
I'll also say that an adaptive AQM is an *essential* piece of the solution to bufferbloat, and a piece we've had no good solution to (until, we think, now).
That's why this article represents "fundamental progress".
It is *any* transition from fast to slow, including your computer to your wireless link or vice versa from your home router to your computer.
Bufferbloat is an equal opportunity destroyer of time.
You are correct that replacing one bad constant with another is a problem, though I certainly argue many of our existing constants are egregiously bad and substituting a less bad one makes the problem less severe: that is what the cable industry is doing this year in a DOCSIS change that I hope starts to see the light of day later this year. That can take bloat in cable systems down by about an order of magnitude, from typically > 1 second to of order 100-200ms; but that's not really good enough for VOIP to work as well as it should. The enemy of the good is the perfect: I'm certainly going to encourage obvious mitigation such as the DOCSIS changes while trying encourage real long term solutions, which involve both re-engineering of systems and algorithmic fixes. There are other places where similar "no brainer" changes can help the situation.
I'm very aware of the research over a decade old, and the fact that what exists is either *not available* where it is now needed (e.g. any of our broadband gear, our OS's, etc.), and *doesn't work* in today's network environment. I was very surprised to be told that even where AQM was available, it was often/usually not enabled, for reasons that are now pretty clear: classic RED and derivatives (the most common available) require manual tuning, and if untuned, can hurt you. As you, I had *thought* this problem was a *solved* problem in the 1990's; it isn't....
RED and related algorithms are a dead end: see my blog entry on the topic: http://gettys.wordpress.com/2010/12/17/red-in-a-different-light/ and in particular the "RED in a different light" paper referenced there (which was never formally published, due to reasons I cover in the blog posting). So thinking we just apply what we have today is *not correct*; when Van Jacobson tells me RED won't hack it (which was originally designed by Sally Floyd and Van Jacobson) I tend to believe him.... We have an unsolved research problem at the core of this headache.
If you were tracking kernel changes, you'd see "interesting" recent patches to RED and other queuing mechanisms in Linux; this shows you just how much such mechanisms have been used, that bugs are being found in this day and age in such algorithms in Linux: in short, what we have had in Linux has often been broken, showing little active use.
We have several problems here:
1) basic mistakes in buffering, where semi-infinite statically sized buffers have been inserted in lots of hardware/software. BQL goes a long way toward addressing some of this in Linux (the device driver/ring buffer bufferbloat that is present in Linux and other operating systems).
2) variable bandwidth is now commonplace, in both wireless and wired technologies. Ethernet scales from 10Mbps to 10 or 40Gps.... Yet we've typically had static buffering, sized for the "worst case". So even stupid things like cutting the buffers proportionately to the bandwidth you are operating at can help a lot (similar to the DOCSIS change), though with BQL we're now in a better place than before.
3) the need for an AQM that actually *works* and never hurts you. RED's requirement for tuning is a fatal flaw; and we need an AQM that adapts dynamically over orders of magnitude of bandwidth *variation* on timescales of tens of milliseconds, a problem not present when RED was designed or most of the AQM research of the 1990's done. Wireless was a gleam in people's eyes in that era.
I'm now aware of at two different attempts at a fully adaptable AQM algorithms; I've seen simulation results of one of those which look very promising. But simulations are ultimately a guide (and sometimes a real improving insight): running code is the next steps, and comparison with existing AQM's in real systems. Neither of these AQM's have been published, though I'm hoping to see either/both published soon and their implementation happening immediately thereafter.
So no, existing AQM algorithms won't hack it; the size of t
No, it isn't based on who you are talking to....
There are significant networks that do not look like the consumer edge Internet, one of which reportedly collapsed in a nasty way (not necessarily in the same way as 1986). Don't presume that the network that may collapse is the global internet (though time based self synchronizing phenomena are a worry there). One of the functions of AQM algorithms is to ensure that TCP flows don't synchronize their behavior. And those AQM algorithms are MIA on many networks today.
Those of us who lived through the 1986 congestion collapse are somewhat worried.
That we don't know is what worries us, as we're flying in an area we don't fully understand.
To solve this, I think we need both AQM fully at the edge of the network, and some sort of "fair" queueing at the edge. The headache is that the classic AQM algorithms won't work in the edge case (and are flawed). "Fairness" is in the eyes of the beholder and a complex question. You may consider it "fair" your kids get half the bandwidth you do, for example. But having a situation where talking to a local CDN 10ms away gets tons more bandwidth than something across the globe may not be "fair" in your view, nor that your web browser can put a horrible transient into the queues as it opens 10 TCP connections simultaneously, and it's unfair what it does to other people sharing your home circuit.
1) Some sort of AQM is necessary to tell the end points to slow down in a timely fashion (by signalling the end point TCP's). Or the buffers fill, and stay full, and you are in fundamental trouble. Consider this the "high order" bit to solving the problem.
2) TCP does not guarantee any "fairness" between flows of different RTT's, Note that what you consider "fair" inside your house is your business: your ISP has some obligation around "fairness" between customers. This is the next bit of precision. Anyone who thinks TCP is "fair" by itself believes in magic that does not exist.
Note these two issues might be addressed by one algorithm, or multiple algorithms, and what is best might be different in a host than in your home router, and different yet again elsewhere in the network. Exactly what we need (and will work well) is as yet unknown, though almost anything (even going back to semi-sane buffer sizes selected with even trivial amounts of thinking) can improve things a lot. And getting there is going to require analysis and testing. I encourage people to help out: for example, we've not yet really played with SFB to see how it behaves in the face of variable bandwidth. Classic RED won't work in a sane fashion at all in that case.
But hugely oversized, dumb, unmanaged, tail-drop buffers have got to go.
Thankfully, at the edge of the network (your host and home router), we have lots more cycles to play with per packet than in a core router and can burn some of them well for this purpose. And that's where we're almost always congested (your ISP may have other congestion problems, but at the aggregation level of those devices, classic AQM algorithms can function, even if maybe not ideally).
The article that was posted yesterday is the short CACM article; Kathy and I have a much longer paper in preparation that will go into this in more detail. But CACM has a word count limit we could not meet, and the "fairness" topic is mostly on the cutting room floor, and now being picked up and finished.
I wasn't expecting the Queue posting to go "live" until next month (i was being naive), so the long paper is not finished. CACM really wanted to lead off January with a bang, so redirected our efforts toward the short, and in many ways incomplete, discussion.
Huh?
It talks primarily about what I observed on my *home* connection...
The Netalyzr data is all about edge connections.
There are problems in the core, but the worst problems I've seen are at the edge, and the edge is where most of the congestion is these days.
You can have an overall congested network. I've seen this on occasion.
But it is very easy (and even more common) for you (or people in your house) to do it to you, than to have the overall ISP network congested. This is something a simple file copy can/does do to you, in practice.
Some ISP's run AQM properly (e.g. RED) in the cores of their networks; some do not. On the ones that do not, you'll see problems at peak hours. Similarly on corporate networks.
Heh. The worst problems (as far as I've seen) we have are in the edge.
In our hosts and home routers (most of which are Linux boxes).
The problem occurs anytime you are next to a bottleneck, and the "municipal and/or corporate resistance" have now built out the core of the internet to the point that the problem is most severe at the very edge, including your laptop/handheld device.
*EVERYONE* has made the same set of mistakes. I have done so too at times....
Yes, and since the BDP is not knowable usually (neither the bandwidth or delay are predictable these days in many/most circumstances), we have to be smart (need AQM).
The problem is *we don't know* what will happen.
The network is operating in a regime it wasn't intended to... And we lack the instrumentation we need to understand how the internet is operating.
The buffers are now so large that we've defeated the basic congestion avoidance algorithms (slow start and congestion avoidance). Technically, TCP congestion avoidance is operating, but the time gets so long that TCP thinks the path changed, and probes more aggressively for a new operating point, as explained in the article.We are flying in a piece of the flight envelope that has not been tested. That was a big surprise, when we dug into the traces.
That should make us nervous.
We've also had one credible report of a significant network that collapsed and was very difficult to restart. We don't know exactly what all happened (and hopefully never will; we just are frustrated we don't have packet traces).
Re: 1. I've always thought that the congestion window to the same end-point should be shared: but that's not the way TCP implementations work, and wishing they worked that way won't make the problem go away. And, as I've shown, bufferbloat is not a TCP phenomena in any case.
Re: 2. HTTP is a lousy protocol in and of itself, and having to do it on top of TCP makes it yet harder. It is the fact that HTTP is so ugly that makes so much else difficult. And I disagree with your claim that high latency links won't use the bandwidth; in fact, lots of sessions is just making things harder. You can read our HTTP/1.1 Sigcomm performance paper.
I'll be writing more on this topic this week.
And we need to replace HTTP, and something other than TCP would be highly desirable. Personally, I'm much more fond of CCNx than any IP based transport.
Thank you for the kind words.
It helps for the several months of lonely work setting the stage for this for it to have succeeded this way.
Your Linux (and Mac and Windows) laptops also suffer from bufferbloat.
As does your home router and 802.11 network.
As do some ISP's.
Don't think the problem is just in your broadband. It's all over.
You asked, I just provided:
http://gettys.wordpress.com/what-is-bufferbloat-anyway/
Good question.
Bufferbloat is the cause of much of the poor performance and human pain using today’s internet. It can be the cause of a form of congestion collapse of networks, though with slightly different symptoms than that of the 1986 NSFnet collapse. There have been arguments over the best terminology for the phenomena. Since that discussion reached no consensus on terminology, I invented a term that might best convey the sense of the problem. For the English language purists out there, formally, you are correct that “buffer bloat” or “buffer-bloat” would be more appropriate.
I’ll take a stab at a formal definition:
Bufferbloat is existence of excessively large (bloated) buffers into systems, particularly network communication systems.
Systems suffering from bufferbloat will have bad latency under load under some or all circumstances, depending on if and where the bottleneck in the communication’s path exists. Bufferbloat encourages congestion of networks; bufferbloat destroys congestion avoidance in transport protocols such as HTTP, TCP, Bittorrent, etc. Without active queue management, these bloated buffers will fill, and stay full.
More subtlety, poor latency, besides being painful to users, can cause complete failure of applications and/or networks, and extremely aggravated people suffering with them.
Bufferbloat is seldom detected during the design and implementations of systems as engineers are methodical people, seldom if ever test latency under load systematically, and today’s memory is so cheap buffers are often added without thought of the consequences, where it can be hidden in many different parts of network systems.
You see manifestations of bufferbloat today in your operating systems, your home network, your broadband connections, possibly your ISP’s and corporate networks, at busy conference wireless networks, and on 3G networks.
Bufferbloat is a mistake we’ve all made together.
We’re all Bozos on This Bus.
A very long time ago, in a galaxy far, far away (MIT, mid 1970's, when I was an undergraduate and a member of MIT"s Planetary Astronomy Laboratory of that era), I remember having conversations with Mike Gaffey about asteroid mining. I see a reference to Technology Review on asteroid mining from Mike in 1977, so I think this got all published; I don't have any TR's of that era around to refresh my memory.
I remember one interesting scheme, where you might take a m-type metallic asteroid (which is mostly iron, nickel, and other useful metals) to earth orbit, by any of a number of propulsion schemes (solar sail, ion engine, or the like). It would probably take a number of years to move it from the asteroid belt to earth orbit. Then foam the asteroid (use solar mirrors to make it molten, and inject gas), and shape it into a lifting body. Then you would fly it into the earth's atmosphere, and land it in the ocean outside any port you would care to deliver it to. The point of foaming it was to reduce its density so that it would reenter the earth's atmosphere without much heating and ablation (we don't want to dump lots of metal into the earth's upper atmosphere), and float when you landed it.
Then you take a tug boat and pull it to a dock, and you have however many kilotons of metal you like. And without the huge energy cost of mining and environmental problems on earth.
As I remember, all the physics work (without having to invent fundamental new technologies), and there are lots of metallic asteroids. Now we just have to figure out how to actually do it. And it is way, way easier to deal with getting to and from the asteroids than the moon or any planet.
- Jim
And Chuck is building hardware which is often, or even usually, running Linux right now.
Track down the research project that's doing big programmable multiprocessors at Berkeley/Stanford/MS research and others.
The x86 instruction set is too baroque to fit in a sane number of gates, so in fact most of the software on that hardware is free and open source software.
I'm tickled pink.
His contributions are inspiring; in fact playing with an Alto so many years ago was the first time I got to mess with a graphics display and mouse, if only on an occasional basis for a few hours.
And I had a chance to work with Chuck a bit: he's great people, and has continued to do first class stuff ever since.
I live about 22 miles outside of Cambridge, where I have often worked. So that is 44 miles/day@ $.5 per mile (U.S. government reimbursement). Your actual costs will vary; but the government rate isn't far from reality. Parking is about $20/day in Cambridge; sometimes more, sometimes less depending on the lot.
$5500 - Mileage
$5000 - Parking
Round numbers for automobile commute: $10,500
Note that there are hidden costs of road maintenance, etc.
Additionally, it is my time; on the commuter rail, at least I get (at least) an hour of my time back.
$2400/year - Commuter rail ticket (also covers unlimited subway use)
$1500/year - Mileage to train station.
Commuter rail commute is therefore about $3900, before any tax breaks (or lower auto insurance rate, due to less mileage and lower theft rates).
Savings for me (excluding tax break and insurance break) was about $6-7K/year.
And the following data:
http://w3schools.com/browsers/browsers_os.asp
is very different (4x higher).
And I can argue both sides of these numbers: people learning web technologies may be more or less likely to be Linux users, statistically.
So right now, we're seeing data that is based on marketing: the web site with the best marketing department has the most widespread results (and who knows who pays their marketing budget, to be paranoid?).
Only Google and similar organizations know "for sure", and even then *they* don't due to embedded uses of Linux, POS systems, and the developing world where traffic statistics will be undercounted.
At best, you can use these data sources for *trends*, and not absolute numbers.