Level 3 and Cogent Reach Agreement on Peering

Armour Hotdog writes "Level3 and Cogent have announced an agreement on a modified peering contract that provides for settlement-free peering subject to certain unspecified conditions. This is a welcome announcement considering the disruption caused earlier when Level3 depeered Cogent. After that earlier dispute, Level3 temporarily restored peering, but announced that they would once again depeer Cogent on November 9th, unless the parties could come to an agreement."

Comments

How was this allowed to happen?

[Jeff DeMaagd]

I think the "depeering" probably shouldn't have happened, and should not have affected people that weren't involved in the dispute, i.e. the rest of us. Had this happened with any other utility, there would be investigations.

Internet Latency

[JhohannaVH]

Good! Everytime they do this, businesses are affected, on the backend if nothing else. It screws up B2B and EDI xactions like mad. If companies can prove that it affected their bottom line, what recourse do they have???

Re:Internet Latency

[elm3r]

Not only are businesses affected, but educational institutions are as well. At my workplace, we conduct many internet-based courses that completely *halted* when Level3 depeered Cogent. The professors of those courses had to quickly make major changes to their deadlines and course structure, to work around the outtage. That is the first time such actions have ever had to be taken here (including changes made due to hurricanes).

Re:Internet Latency

[dekemoose]

Important applications should not be run on single homed networks.

Free market solution regulation

[dada21]

After hearing hundreds of posts clamoring for government regulation (ie, slow to respond expensive monopoly), the best solution came quickly.

Why did this agreement happen? It happened because the market required it. Customers were unhappy, producers lost money, no one profited on either side.

If we pushed for regulation, how many years and billions of dollars would replace what two corporations did in a week or two on the demands of their customers?

Stronger ties, but still breakable

[elm3r]

In the new agreement, there are clauses that state that Level3 can again try to charge Cogent if their traffic amount is grossly over that of Level3's. So, while this is definitely an improvement, it doesn't rid all potential future problems.

If anything, this definitely hammers home the idea of multihoming...

Re:Free market solution regulation

[Jeff DeMaagd]

When was the last time you remember that Sprint customers were cut off from being able to call MCI subscribers?

I don't want massive regulation, but something simple to prevent deliberate cut-offs would be nice, and it appears that the free market didn't solve that problem.

"Pay up or we disrupt your business..."

[DamienNightbane]

Is it just me, or does Level 3's ultimatum sound alot like an old fashioned protection racket? How is this any different from the Don sending someone to smash up someone's shop after the owner misses a payment?

Is there any way to get law enforcement involved? What about a class action lawsuit?

Needs to be regulated

[nexUK]

Tier 1 peering needs to be regulated in certain situations. The Cogent and Level 3 "who has the bigger dick" contest has caused isolated pockets where full routes/reachability to certain parts of the Internet wasn't available for some Cogent and L3 downstream customers. Get these big boys to maintain settlement free peering when a certain amount of the routing table "belongs" to them. simple.

Re:Needs to be regulated

[fuzzy12345]

Tier 1 peering needs to be regulated in certain situations.

I'm for some regulation of the Internet, but not here. These guys went back to the table because they each had guns to their heads; their customers (on both sides) didn't really care whose fault it was and would've started leaving.

Calling for regulation would likely lead to California energy crisis-type situations: PG&E and Con Ed were both required to retail (at a fixed price) stuff they had to buy wholesale (on the open market) and when the wholesale price went above retail, bankruptcy. (Don't get into market manipulation, that's a peripheral issue). The Internet has been remarkably successful precisely because any yahoo with a router and a cable crimper could build out more of it, without a license, approvals or anything else.

Re:Needs to be regulated

[Uhlek]

You're missing the point. Cogent isn't a Tier 1 ISP. They're close, but not quite. To be a Tier 1, that means you don't pay for peering -- period. Cogent does.

This was a fairly straightforward business problem. Settlement-free peering only occurs when its in the best interests of both parties to do so. There are massive costs still incurred on each end, but they simply don't exchange money. The traffic in both directions is equal enough that neither side is incurring a loss. L3 determined that they were, and announced to Cogent that their settlement-free peering agreement was going to end.

Rather than doing what they should have done, and either ponied up the cash to L3, or reached a transit agreement with another ISP (say, a tier 2) to receive L3's prefixes and get its own prefixes onto L3's network, Cogent allowed the depeering to occur and used the resulting disruption to the Internet to their own advantage by calling L3 out.

They, in effect, allowed a major outage to occur in order to avoid paying for transit to L3. L3 gave them something like 90 days notice, plenty of time for Cogent to develop a contingency plan.

Yet, they didn't. Thier customers immediately became unreachable from L3's network, and their customers were unable to reach L3. They allowed this situation to continue, leveraging it for a public relations backlash against L3, and attempted to lure L3 customers to Cogent.

I'll be the first to admit my understanding of the issue is not 100% -- so if I'm missing a critical point, please let me know. But, from my understanding, let me be the first to say this is not a major problem with the Internet, nor is it something that regulation would do anything to fix. This is a bullshit back-room business decision by an ISP trying to save a buck and make a name for itself.

Re:Needs to be regulated

[nolife]

A simple approach but obviously not the status quo method of deciding on who should pay for what..

L3 customers are requesting more traffic from Cogents customers then is going the other way. Why is any one direction of traffic considered a load and another considered a source for income and different from each other? It seems to me as these two companies are concerned, more L3 customers desire and need Cogent traffic then Cogent people that need the L3 traffic as noted by the obvious business difference that is raising this issue. Why is the traffic from Cogent considered Cogents traffic when in reality, you could consider it traffic requested by L3 paying customers? The traffic from Cogent to L3 is a mutual benefit to both Cogent supplying and L3 customers requesting equally. Cogent could claim that L3 is sucking down a disproportionate amount of its bandwidth compared to what they are own customers are getting back?

I know, not a status quo, that's the way is always has been, your an idiot, blah blah, but what makes one way right and accepted and the other not? Both ends have customers paying for the bandwidth they send and recieve.

Cogent isn't without fault here

[Dracos]

Are they going to learn their lesson and strike peering agreements with more tier ones then just Level 3?

Re:Cogent isn't without fault here

[fuzzy12345]

(with-cluehammer "You don't get it. Tier 1s have lots of peering agreements. A peering agreement with someone else DOESN'T entitle you to use their network to get to a third -- that would be transit. Basically, these guys said they wouldn't exchange traffic directly for free, and they wouldn't pay some other provider to act as a go-between, which, if you understand how these things work (reputation, game theory and all that) is perfectly logical.")

Re:Cogent isn't without fault here

[slive]

Nope. Routing over one network to get to another is called *transit*. Cogent refused to pay for transit to get to L3. So they still had plenty of peering arangements with other networks, but none of those peering arangements allowed Cogent to reach L3. (And the same could be said in reverse.)

consider an aphorism

[Quadraginta]

Give a man a fish, he eats for a day. Teach him to fish and he eats for the rest of his life.

If a government agency just enforced some prior restraint on the companies, what have they learned? Not to do what they did. What have they learned by being forced to solve their problem themselves? Not to do what they did, and also how to successfully negotiate with each other when things go awry, what the market really wants from each firm, how to rapidly re-evaluate corporate strategy in the face of adverse external events -- in short, how to be more "grown-up" in managing their own affairs.

The interesting bit

[fuzzy12345]

Oct. 7: We determined that the agreement that we had with Cogent was not equitable to Level 3. [...] Cogent was sending far more traffic to the Level 3 network than Level 3 was sending to Cogent's network.

Oct. 28: The modified peering arrangement allows for the continued exchange of traffic between the two companies' networks, and includes commitments from each party with respect to the characteristics and volume of traffic to be exchanged. Under the terms of the agreement, the companies have agreed to the settlement-free [i.e. no-charge -- ed.] exchange of traffic subject to specific payments if certain obligations are not met.

So what happened? It's unlikely Cogent could say "Oh yeah, we'll get 50% more retail customers so as to send traffic your way." Level 3's customers squawked and Cogent insisted they wouldn't pay? (That's Internet Mutually Assured Destruction)

Re:The interesting bit

[petermgreen]

the exact details of what was agreed to will probablly never be public

one possible condition could be moving some of the peering to other locations so level3 has to do less work and cognet has to do more to get the traffic between the desired endpoints. I belive depeerings have caused changes like that in the past.

another possiblity is the peering is theoretically setlement free but cognet may end up paying some of the "fines" mentioned.

yet another possibility is as you suggest level 3's customers said enough and they backed down but put in some secret conditions to try and save face and make it look like not thier fault.

Pissing contest

[sunderland56]

Level 3 and Cogent are in a pissing contest?? Oh, wait a minute. That was peeRing. My bad.

Re:Pissing contest

[groot]

They should wait 'til December when their pee rings can make a real dent in the snow.
--laz

Re:Hah

[Armour Hotdog]

Oops - I guess I could have phrased that a little more clearly. The November 9th ultimatum came when Level 3 originally restored peering back on 10/07. Today's agreement supercedes that, so the danger of another depeering (between these two ISPs) has passed unless somebody (read: Cogent) violates the terms of the new agreement.

How does the phone company handle this?

[koehn]

I assume that the phone companies and mobile companies have similar (though not identical) issues to this. Aren't they mandated to provide access to their networks to other providers (e.g., Vonage)? What restrictions/costs are typically involved?

Re:so basically ...

[dekemoose]

Do you even have any idea what it is you're discussing here? This is two companies who had a business agreement, one company abused it and got smacked by the other company.

Think this is about the US? Why don't you look into France Telecom's de-peering of Cogent awhile back.

This is not an Internet thing in that it affects the entire Internet. It is an internetworking thing in that it affects the way two ISPs exchange data.

Sit your knee-jerk, loud-mouthed, over-opinionated, under-educated ass down and shutup until you can remove your head from where you have it stuck.

Leve3 and us. Also why did it break stuff?

[starwindsurfer]

I am a sys admin for a small atlanta ga ISP, when Level3 de-peered you dialup users that were connecting to Level3 POPs couldnt connect, our call center was flooded and we were scrambling like mad to switch everone over to the Aligence Telcom POPs. L3 really needs to think about the broader ramifications of their actions.

Also, The internet is suposed to be dynamicaly routed, ya know BGP3 and so on. Why did this break so many things? If the route was down, shouldnt the routers just use the next best preffered?

Re:Leve3 and us. Also why did it break stuff?

[Uhlek]

The core of the Internet is not run like you think it would be. While BGP is dynamic, when and where various prefixes (network address blocks) are advertised is tightly controlled.

When you peer with an ISP, that means you only exchange their prefixes for yours. Any other networks that may be reachable via that ISP are not advertised back to you, just like they don't send your prefixes to the rest of the Internet.

Access to other parts of the Internet via an ISP is called transit -- this is what we're all most familiar with. You give your prefixes to them and they take care of exchanging it with the rest of the Internet, and they give you the rest of the Internet's prefixes.

Lets say there are two ISPs. Lets say its Alpha (a Tier 1 ISP) and Beta (a non-Tier 1). Alpha and Beta have a settlement-free interconnect agreement -- meaning they peer with each other. Remember, that means that they can only access each other directly. To go to the rest of the Internet, it means that they have to go to other providers. Alpha uses its other SFI agreements with all other Tier 1 providers to do so, Beta may use other SFI agreements and paid-for peering or transit agreements with other providers. For so-called "tier 2" ISPs, it's often a complicated mess.

Then, lets say Alpha decides that its SFI agreement with Beta is no longer in its best interest. So, they tell Beta they're going to depeer if they don't start paying for the peering.

Beta has two options to ensure that their customer base will stay functioning. They can either come to an agreement with Alpha, or come up with a transit agreement with another ISP, either another Tier 1 ISP or Tier 2.

It gets more complicated than that, but, that's the basic jist of it.

Re:Free market solution regulation

[BridgeBum]

DNS and routing really have very little to do with each other.

Most websites of any size whatsoever not only have multiple IP addresses assigned to the site (DNS), but also multiple links to the internet across carriers (routing). A problem in either area can cause diruption to clients, but that doesn't make them the same system.

The link you provided (minus the marketing noise) sounds like a proximity based DNS solution...also not revolutionary. Many site-to-site load balancing solutions use response time for DNS queries to determine the best IP address to return. In the event of fragmentation like happened between L3 and Cogent, if only one DNS server was avaiable on the isolated cloud (Cogent), the IP returned should be a web server on the same network. Voila, no customer disruption.

In short, yes it is possible to use DNS to work around some routing issues. It's also possible to use routing to work around some DNS issues. (example: cached DNS entries leading to a dead site, but natting the traffic to a real server and then routing the traffic accordingly). In reality though, they are separate systems. There already is a need for 'multiple routing paths', at least for any web site which wants to come close to 99.999% availability.

It wasn't de-peering.

[Anonymous Coward]

Level 3 didn't just switch its connection to Cogent off, it left it running and tarpitted any traffic going through it. Like other people are saying, there'd be a class action against them if, say, they were a power company deliberately sending surges into other companies' grids.

STOP THE CLUELESSNES!

[fuzzy12345]

Dudes, there's a lot of cluelessness here, about tarpitting, routing around failure, next best route etcetera.

Being a tier 1 means, essentially, HAVING NO DEFAULT ROUTES. You make deals with all the other tier 1 providers for direct connections at various places around the country and, if you can't colocate with a particular tier 1 in a particular geographic location, you pay another provider for transit from you to that tier 1. Being at the top of the pyramid, there's no default route you can hand packets off to when one of your connections fails - because that would mean somebody else was providing you with a free lunch.

Of course, these guys are constantly squabbling ("we're bigger than you, so you should be paying us for the privilege") but, since disconnecting affects both peers' customers, it's really cutting off your nose to spite your face.