Slashdot Mirror
Tier One ISPs Dying
xbmodder writes "Two tier one ISPs are down today. At about 23:30PST both Verio and Level 3 starting having problems with routes. According to Level 3 this is a software upgrade gone awry. Is this the end for Level 3?" Many, many reports about this are coming in, and if you're wondering why the stories were rather sparse overnight, it's because it's difficult to post them without internet access. Hope everyone else is back online too.
Take a look at the scoreboard now. The mentioned problems are gone and Level 3 is no longer in the red.
One that doesn't lease their infrastructure.
Eg. you have your own large backbone, you own all your equipment.
In effect, a small and wholly owned internet that peers with other internets.
Tier 1 are the huge ISPs, which peer with eachother (and don't pay eachother transit fees) and sell transit services to smaller ISPs (which do pay fees to send traffic through the Tier 1 ISPs). So yeah, bandwidth wholesalers is pretty accurate. See this wikipedia article.
The Official Steve Ballmer Webpage
This is not _just_ their server being down, this is the entire network of two tier 1 carriers.
The (basic) implications of this is that a good chunk of the internet as a whole is inaccessible to the rest of the internet.
"When a ball dreams, it dreams it's a frisbee"
While this only lasted a few hours, it still caused a mess across the North American Internet during those hours. The point is a small amount of big networks are responsible for over 90% of the traffic on the Internet. If alter.net went down it would be total chaos. If just one of the major peering points went down, sure the traffic would be rerouted, but overloading the other points at such high latency that it would be almost unusuable. You better hope no one destroys MAE-EAST or we'll have a live example of what ife without the Internet is like.
When a Tier 1 provider goes down, their customers go down too. That picture on the Boing Boing page shows a list of the Tier 1 providers. Every ISP that is NOT a Tier 1, gets their access from a Tier 1.
People speculate that Level 3 is dying because they've been making some really bad decisions lately, resulting in a lot of outages. A couple of weeks ago, they actively filtered out traffic from their competetor, Cogent, over a dispute from how much to charge at the point their networks exchanged traffic (called a 'peering point'). Now this. The rumor is that the company is in financial trouble.
Domesday is something like a census of Britain circa 1085. It has nothing to do with internet outages, which is more akin to doomsday.
Toronto-area transit rider? Rate your ride.
This sort of event provides motivation for overlay routing schemes, which can compensate for major outages along various routes of the backbone:
e rs/subramanianOver/subramanianOver.pdfn focom.pdf
http://www.usenix.org/events/nsdi04/tech/full_pap
http://www.eecs.umich.edu/~farnam/pubs/2005-hwj-i
An unjust law is no law at all. - St. Augustine
Tier 1 ISPs own huge swaths of networks-- literally miles and miles of cable, and sometimes radio and other links. They route the traffic across these lines.
:-)
More precisely, Level3 seem to own 23,000 miles of optic fiber.
The rumor is that the company is in financial trouble.
Yeah, not so much of a rumor anymore either -- Level 3 loss widens.
Beware: In C++, your friends can see your privates!
I'm sure you know this, but for the rest: "flapping" is the common term for when a router's routing tables rapidly cycle between two invalid states. The dead bird analogy is pretty descriptive, but the term "flapping" has technical and not allegorical origins.
Dewey, what part of this looks like authorities should be involved?
The reason that Level 3 isn't happy with the peering arrangement currently is that it's not even remotely even. Level 3 sends almost nothing over Cogent's network and Cogent sends over a vast majority of their traffic through Level 3. A peering agreement is based on the premise that the companies will be sending almost equal amounts of traffic through each network. Level 3 has been analyzing that for a time now but the last straw was when Cogent had a sales blitz targeting Level 3 customers saying that they would dramatically drop their prices to almost nothing to get them to switch away from Level 3. They are now also using the downtime that was experienced due to the peering problem in their advantage even though Cogent is in the wrong. Cogent knew about the depeering and did nothing to resolve it.