Slashdot Mirror
The Other Side of the Sprint Vs. Cogent Depeering
Swoolley writes "A month back this community discussed the Sprint vs. Cogent depeering. Now a story I wrote for Forbes.com tells the inside story of the fight, based on the lawsuits the two companies filed against each other in Virginia state court. For once, thanks to those suits, the public gets to see the details of a confidential peering agreement between two of the Internet's largest autonomous systems, as well as the circumstances leading up to the depeering. (Which company is in the right? Read the facts and decide for yourself.) While some people have argued that the depeering is reason for more government regulation, the Forbes story makes the case that details of the recent Cogent vs. Sprint fight argue for exactly the opposite: keeping the Internet backbones free of government meddling."
This is Forbes, after all. According to Forbes, the Great Depression was proof of the need for less government regulation.
Lacking <sarcasm> tags,
So tell me, what impact could the Government of Botswana have on the Cogent-Sprint issue?
OH WAIT, I forgot, only the USA has the internet...
Is anyone else here tired of knee-jerk partisanship framing discussion in terms of false dichotomies? Government involvement can do a whole lot of good or a whole lot of bad. The devil is always in the details.
Good: regulate to prevent monopolization of last-mile utilities and reduce barriers to competition.
Bad: let lobbyists who supported your campaign write bills that hand out huge billion dollar tax breaks to carriers to build out the next generation "information superhighway" and sit idle while all of that money goes straight into the pockets of shareholders instead while countries like South Korea and Japani take the lead in broadband while America slowly turns into a broadband backwater.
Hopefully things will work out a little differently in the new administration.
TurnKey Linux: if it can be easy, it should be easy
I don't know what others have been suggesting for regulation, but I would strongly support two simple regulations on depeering. 1) Provider A must give provider B at least X days notice of intent to depeer (say 180 days) 2) If some agreement isn't reached between provider A and provider B, both providers must notify all thier customers of the planned depeering giving thier customers at least X days notice (say 90) Nothing too invasive, just some basic comsumer protections.
"reality has a well-known liberal bias" - Steven Colbert
This is not the first time Cogent act like bandits.
http://gigaom.com/2008/03/18/cogent-ceo-peering-breakdown-is-telias-fault/
When this story broke two months ago, the Sprint claim was that Cogent was having an unbalanced share of the traffic, and that was given as reason for depeering; mostly due to Cogent signing up large numbers at 10% of the price of Sprint and the other tier 1 ISPs. Makes you wonder if Forbes has an axe to grind there now.
I'm aging rapidly, I bought a new game and had no idea if my machine was good for it.
Looking at the tags for this story (and many others), it seems tags are being used more for comments on the story than as a useful means to group stories by tag. For instance here we have the tags 'corporatewhining' and 'fuckemboth', both of which are most definitely a comment on the story, not a useful tag as such, well, not very useful as comment either, truth be told.
For that matter, the more useless a tag, the more likely it is to be of a derogatory nature.
That's pretty broken really, not even slightly useful as a feature.
Perhaps there should be a list from which people select, such as there is when submitting stories
A learning experience is one of those things that say, 'You know that thing you just did? Don't do that.' - D. Adams
As a Slashdot story that basically says hey look at me I got an article in Forbes aren't I the awesomest.
Had the federal government responded initially by cutting taxes and spending, lowering trade barriers and streamlining regulation, it probably would have been just a very bad recession. You can't spend your way out of a bad economic cycle; that's like drinking more beer as a solution to a hangover. What you need to do is calm things down, encourage trade and not experiment with the economy and organizing society. I'm not going to say that the federal government caused the Great Depression, but it certainly didn't do anything positive to stop it and return the economy back to sanity.
You can't spend your way out of a bad economic cycle; that's like drinking more beer as a solution to a hangover.
That's a great analogy! You might be able to drink away a hangover, but it's just going to result in a worse hangover later.
Prov 9:8 Do not rebuke mockers or they will hate you; rebuke the wise and they will love you.
When *I* kill a peering, the traffic is rerouted through the Internet. Please don't tell me Cogent and Sprint don't use BGP! So why did traffic stop flowing?
i am actually for some regulation. not really of any content, but to force the larger companies to be more open. fact is, the Internet/phone backbone was built using taxpayer money. if that is the case, then the taxpayers have a vested interest in competition and choosing their providers. minimally, all public institutions should be on a government owned/operated network.
large companies like Sprint have paid off enough local FCC chairs that they are now deregulated, and are gladly unplugging all other ISPs that don't belong to the top 6 or 7. fees just to open the plug-in process are over $10,000 a month, and the bigger ISPs aren't even required to do anything. many local companies here have spent $10,000 for several months, having an open account with AT&T, and AT&T is allowed to sit on their hands because they can. you can pay $10,000 to AT&T and request that they hook you up at the local CO, and they will gladly take the money and say "Thanks for making a formal request", and that is it. end of story.
and here in Oklahoma, AT&T is even double billing the local schools and libraries, but the FCC won't do anything about it. AT&T has a contract in Oklahoma to provide schools/libraries with connections for a certain base price, but because the schools/libraries get and pay their own bills, AT&T sends them bills with higher rates, knowing that the local mayoral staff won't have any clue on what they are supposed to pay.
truth is, we should have an easy way to link into a system that was built with taxpayer money. and we need to actually be able to VOTE on what these big ISPs do, and not rely on the incredibly corrupt state FCC.
You started off that post great, but it all went downhill in the "Bad:" section.
Using tiny, *tiny*, whole countries as an example is flawed. Over 90% of Japanese live in less than 20% of the total area Japan occupies. To illustrate further, the US could easily bring New York City into the 'fiber to your door' reality for about the same cost of the entire country of Japan. Unfortunately for your argument, 90% of Americans don't live in New York City, or rather, in 2% of the area of the entire USA.
If that were true, it would be very cost effective and easy to roll-out new technologies. This is the same reason that the Japanese didn't get charged when ISDN was rolled out to the entire country, and the same for DSL.
Next time try comparing apples to apples.
With all those people being cut off, what about the claim the internet "heals itself" and routes around damage? It looks like these big corporations have partitioned things so that's no longer true. If there's any possible route between me and http://slashdot.org/ I want the system to find it, dammit!
Pretty much.
...the Forbes story makes the case that details of the recent Cogent vs. Sprint fight argue for exactly the opposite: keeping the Internet backbones free of government meddling.
It is, in fact, inconceivable that Forbes would make any other case. Ideology predetermines their arguments, and in this case, the ideology at work is a sort of economic anarchism that, quite frankly, has been completely discredited by the current state of affairs in the US economy. Not all regulation is "government meddling"; some of it is necessary to protect consumers -- and often even vendors -- from dishonesty and short-sighted greed that is often harmful in the long run to the miscreants themselves.
It is at least mildly ironic that the proponents of economic anarchism are often simultaneously proponents of a hardline law-and-order position in other areas of law.
Proud member of the Weirdo-American community.
Cogent argue that under the terms of the contract they passed. They kept the link open at their end because as far as they were concerned they had passed and Sprint was simply following its end of the bargain. They're arguing that they don't have to pay because if Sprint really didn't think they had passed, they could have severed the link at their end.
The confusion is because both sides measured the performance in different ways. From Sprints' complaint:
Cogent unreasonably claimed that the amount of interconnection traffic satisfied the
utilization threshold requirement in the Trial Agreement because the port utilization peak figures
for each of the ten ports (used to calculate billing) exceeded the average utilization criteria across
all ports. Cogent ignored that Paragraph 5.E. required a sustained threshold average utilization
across all ports for the entire period, and instead focused on snapshot figures based on the
commercial pricing model of peak usage. As a result, Cogent argued that it was entitled to
settlement-free peering with Sprint.
I find it hard to believe that Cogent walked away from negotiations with the wrong idea about how the test was going to be measured. In any business negotiations both sides go to great pains to make sure everyone understands what's being agreed because otherwise it winds up in court like this. If the judge takes the view that Cogent was mislead (deliberate or not) then this becomes a big PITA for Sprint.
So yea, a balls-up for both parties.
Nick
You can't spend your way out of a bad economic cycle; that's like drinking more beer as a solution to a hangover.
While we're on analogies: Government stimulus packages don't - because the money they hand out has to come from somewhere. That somewhere is either additional money they tax away (typically from the most productive - the ones they were trying to "stimulate") or by "printing" (or equivalent) new money which gets its value by pulling value out of the money already out there. And the government handling of this money has costs. The stimulus is always less than the stifling.
So government "economic stimulus" is like trying to lengthen a blanked by cutting a strip off one end and sewing it onto the other. The blanket not only ends up no longer, but even a bit shorter.
(If not for that loss it would be like daylight savings time. B-) )
For more on this see the broken window falacy.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
What I want to know is, what happened to that 3rd provider Cogent used as an in-between with Sprint originally? Did they cancel it after getting the 90-day test from Sprint? It sounds like Cogent wanted to get out of a peering deal with a smaller provider, and wanted to use the "but we don't have any other way to connect to you" defense when Sprint decided to charge.
> This is Forbes, after all. According to Forbes, the Great Depression was proof of the need for less government regulation.
Too many Libertarians here will actually agree with that. Let's put that another way that Slashdotters might understand:
This is Forbes. They had Daniel Lyons writing about how SCO would win against those Communist Linux hippies.
The real question on regulation is if it does more good than harm. The easiest way for me to think about it is as a controls system.
It can be underdamped (no regulation), meaning that the industry will go to extreme highs and lows as companies go for short term profits and take advantage of monopolistic opportunities only to be bitten in the ass by those same policies later. Any slight impact on the industry will send companies fortunes flying high or crashing low. There is also little need to innovate since once you have secured your position you can simply remain there until a competitor begins to make inroads on your market.
It can also be overdamped (too much regulation), meaning that the government is so involved that it is slowing innovation and holding companies to the same playing field even if one has a much better product than the others. Industries will be slow to recover from negative effects and slow to take advantage of new opportunities as they wait for the regulation to catch up with changing technologies.
Of course, it is technically possible for a system to be perfectly damped, where regulation would protect the industry from wild swings while still allowing innovation to flourish. Of course, this is the knife edge that is nearly impossible to walk, especially tech industries that are constantly changing.
In the US, it would seem to me that we are underdamped, telcos are taking short term profits rather than improving infrustructure. If the government could reduce the cost of entry into the market or legislate a maximum cost / bandwidth it would improve the infrustructure immensely.
What public interest? These companies spent their revenues from other areas of business to invest in the infrastructure to provide a commercial service to paying customers in order to increase their overall revenues and profitability.
Just because it's a service that Joe Plumber is oblivious of but still needs in order to download the latest in plumbing-snake-porn, does not mean it's a "public service".
If Sprint wanted to disconnect their entire network from the rest of the world, firewall off every one of their customers and run nothing but local copies of Gutenberg and Wikipedia as a content service, that's their choice.
Any talk about 'public interest' is a red herring and mindlessness.
namaste---
This depeering reveals the fragility of the Tier 1 "people pay us, but we don't pay anyone" philosophy.
Tier 1 is about long-haul vs. last-mile. The last-mile providers have the customers who pay the bills. The Tier 1 carriers have the big long-haul lines which chew up money and don't pay any bills. Money has to flow from the customers to pay for all the pieces of the path.
It's not so cut and dried, of course. Tier 1 companies typically are also last-mile providers as well - just big ones whose internal interconnects span continents and/or bridge them. And smaller last-mile providers may themselves have a lot of long-haulage, even though they're regional, and may provide bridges between, say, a Tier 1 carrier and another regional last-mile provider or two Tier 1 carriers.
But the point is that there are asymmetries in the cost and billing structure of the carriers, with the bigger ones typically providing more benefit to the smaller than the other way around (the bigger ones being transit-heavy, the smaller paying-customer-heavy). Money sometimes gets exchanged to even this out, usually flowing from the little guys to the big ones.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
You do realize, I hope, that you are citing the conclusion of your hypothesis as proof of it?
As long as we're on speculative economics in an alternate history, would you care to address the events of 1937-1938?
Lacking <sarcasm> tags,
Government involvement can do a whole lot of good or a whole lot of bad.
AND. Government involvement does a whole lot of good AND a whole lot of bad. Any time there's government involvement you can pretty much count on getting both.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
Please do not reward things like these.
Thank you, moderators.
This is Forbes, after all. According to Forbes, the Great Depression was proof of the need for less government regulation.
How would government regulation help in this case? Peering has to make economic sense for both parties or they wouldn't do it. All that happens after peering is broken is that the routers are reconfigured to send traffic over their transit links instead of the peer links. Ultimately, customers are not hurt (except for downtime because of an unplanned link outage).
The government has no business inserting itself into this agreement. The government is not in the business of understanding the economic conditions that provoke peering agreements.
I recall reading an article a few years ago about how Yahoo gets approximately half of it's total bandwidth for free. It makes economic sense for content providers to peer with content consumers. This is where the net neutrality thing breaks down. Large content providers make sure they create links that make sure their content gets to eye balls quickly. The smaller content providers don't get this privilege unless they use content caching services or they find a co-lo that has a network with plenty of peering agreements already in place. Is it unfair? Yeah, so? That's how the chips fall.
If Verizon finds out that enough of their customer traffic is destined to Cogent, it only makes sense for them to peer. Both Cogent and Verizon have a huge number of peering agreements and I wouldn't be surprised if not having this agreement in place really makes that much of a difference to either one of them.
Regulation is required to get some transparency and a better sense of confidence into markets. CDO's are the perfect example.
How big is the market for CDO's? What's the liability to investors? Were counterparties *required* to put up capital? What are the terms of the CDO agreements? What kind of leverage is there in CDO's?
None of those questions can be answered at this time and yet once-mighty investment banks literally vanished overnight with unknown leverage conditions.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
History has shown a country can and has spent their way out of recession(s). It was called the WPA. Learn a little history: http://en.wikipedia.org/wiki/Works_Progress_Administration
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
Shameless self promotion: This article at Ars Technica explains how peering and transit works. For some more info see my blog: Internet Thought.
Use Adsense for Charity
"It's in the public good" is not a valid reason to do something, partially because the definition of the public good is so subjective. In the Hannibal films, for example, Dr. Lecter killed flutist Benjamin Raspail to improve the quality of the orchestra to which he belonged. We are all likely in agreement that this was murder, but apparently Lecter would disagree.
Is anyone else here tired of knee-jerk partisanship framing discussion in terms of false dichotomies?
Hell yes!
I agree with your analysis of the causes and possible solutions during the Great Depression. Unfortunately, I think drinking well known as the best solution for a hangover.
I have a theory that the truth is never told during the nine-to-five hours. - Hunter S. Thompson
But nearly all of the government players from the dot-com demise era will be back in power as of 1/20/2009. Sometimes "change" isn't necessarily a good thing.
Do subscribers get better search functionality? I'm not really whining, but the search sucks. Hence the tagging system?
Maybe "acting" is incorrect, they are idiots.
I've seen these stupid acts like this before at the company I worked for between two vendors. I remember at that time that preschool children acted better than these two Fortune 500 companies which were fight each other in a similar manner.
A good government should step in well before this happens to prevent damage to major infrastructure. However the word "good" is the key word in this and we don't have a "good" government now.
"If conflicts between individual users of a common resource ... cannot be resolved through the "spontaneous order" of those individuals pursuing their individual interests... a governing authority is instituted to preserve those individuals' rights and to regulate access to that shared commodity."
This is precisely what the courts are for and exactly the reason we don't have a natural monopoly providing internet access as a utility. He has nullified his own argument for regulation. I guess pundits default to Freedom of Speech when their synapses cannot make a further connection.
If that's really true, that Cogent purposefully sabotaged their own customers' ability to do business, I would suspect there is grounds for a class-action lawsuit somewhere there. Time will tell, but if there's any *chance* of a payout, some lawyer must be on the scent by now.
If you just put a pile of money out in the street, sure, nothing will change, but your analogies are just that. History and reality are better barometers of effective policy.
If the government employs people to improve infrastructure, it lowers the cost of doing business and benefits the whole economy, while evening out the down cycle when other businesses are cutting back. The biggest reasons western countries do well as economies are their workers and their infrastructure and their reliance on government technology and protectionism. While America unfortunately does not see the benefit of having a well educated populace, it does see the benefit of having a reliable power grid, sewage system, telecommunications network, etc. Some societies see single payer health care as part of infrastructure, which is the main reason it's cheaper to build a car in Canada than it is in Detroit.
(Here's an article that discusses two facts unknown to most Americans: our car companies employ more people in Ontario than Michigan, and they do it because of their more efficient health care system and the canadian dollar.)
In fact, the computer you're typing on and the internet it travels over are all due to government research. Do you imagine we would be less prosperous if China had been the ones who were licensing technology for us to manufacture instead of the other way around? Government is capable of doing good things, but not in the hands of those who attend to the needs of corporations instead of people.
Ideology predetermines their arguments, and in this case, the ideology at work is a sort of economic anarchism that, quite frankly, has been completely discredited by the current state of affairs in the US economy.
It's only been discredited if you're one of the people who believes that the US has had a free market for the last several decades. The people at Forbes (as well as those who actually study economics) have been living under no such misbelief.
The real litigious bastards...
"freenix" is the same person as the one that started this thread.
The company provides a service. If you don't like their service, find a different company.
They complain that Congent doesn't pay for peering so the other company has to pay anything. But seriously they probably have to pay more to route that trafic through another company.
The costs usually are neglectable. A port at an exchange point only costs a few hundred to thousands dollars a month.
Companies should stop caring about all that business stuff when it comes to peering and just peer!
So you managed to do just that for roads and telephones but you are suggesting that it would be impossible to do for broadband?
I disagree. It would be difficult and expensive but, given the will, it could be done. The problem is that it is not your Government but businesses that would have to carry out the work, and they want to see a profit. So it is not in their interests to do it. Of course, your Government could offer a financial package to help pay for the work. Oh, wait, they already have done.....
People once said that you couldn't travel at speeds greater than 25 MPH without suffering physical injury, that you couldn't go to the moon, that an African-American could be President, and so on. All these things have come, or are coming, to pass but America is still becoming an broadband backwater. It is not because of ability but simply a lack of will.
Have a look at soylentnews.org for a different view
[T]he ideology at work is a sort of economic anarchism that, quite frankly, has been completely discredited by the current state of affairs in the US economy.
The most regulated sector of the financial industry is... the mortgage industry. The financial crisis started in... the mortgage industry. Before you start crowing about regulation, remember that.
I have another complaint along similar lines. Why should there be any secret agreements between these companies, or any two corporations for that matter? Whatever the reasons why they might want to keep the agreements secret, the secrecy would seem to make public oversight difficult or impossible.
Granted, the entities involved are free to make some informal agreements among themselves. (They aren't free to make agreements to fix prices, hidden or otherwise!) But if they want to be able to use the legally constituted court system to enforce such agreements, then the contracts should be completely open and available to public inspection. (And not just after a lawsuit is filed: from the moment of signing.)
Aside: why should there be public oversight in this and similar cases? Imagine if your power were out for two days because of some secret spat between power companies!
You do realize you're saying that about one of the most monopolistic services ever, right? But I guess when you have your "free-market" ideology shoved so far up your ass, telling someone to sell their home and move for better internet access doesn't look COMPLETELY retarded, eh?
It would be totally Communist of us to insist that these recipients of hundreds of billions of tax-dollars put something back into the community. The free-market needs subsidies, but can't be expected to offer any consideration for such, right?
Yay hypocrisy!
I'll tell you what, Sprint can just drop out of the ISP business altogether, and we can just apply the principle of eminent domain to their hardware, so our tax-dollars can continue to serve the public good. If you can do it to someone's home in order to build a strip mall, I don't see any reason at all to not do it in order to keep jobs, provide a public service, and maintain the infrastructure we've built with taxes.
Neither Sprint nor Cogent is completely in the right here. Sprint just outright cut off the connection endangering both government, corporate, and university's business. As cogent and sprint are mostly used for data centers and large organizations.
But Cogent was also at fault because it was just getting bills from sprint and just brushing them off. They were in the wrong because they just assumed that sprint wouldn't doing anything.
Its Cogent's fault for just expecting nothing to happen and it is Sprint's fault for not being fair and refusing to work out a fair deal.
All in all some sort of government regulation needs to be created. There is no other thing Sprint could have done so its not completely their fault. The internet is based on universal connectivity. Backbones need to be fully connected and backbones should have direct connections to eachother. Connections should be free if the proportion is within a certain threshold (eg. 10% either way) and should have low, set fees if the proportion is more one sided.
Its not neither of their fault as they are just businesses looking out for their own interests, which frankly is what they were supposed to do. I am not an idealist who thinks that all corporations should or will ever not be greedy or focus on themselves and try to make as much money as they can....thats what capitalism is about. But at the same time, since corporations are inherently self driven their needs to be regulation to make sure that corporations don't stifle innovation, quash competition, price fix, or destabilize vital infrastructure. Both Cogent or Sprint could have prevented it but they couldn't come to an agreement. We just need to make sure that the government keeps the world wide web truly worldwide and that backbones and ISPs don't partition their networks and wall them off like cell phone companies do. You should just as good a connection to a bellsouth customer as you would a comcast customer (not withstanding latency of course0
Well I guess we can so say goodbye to the world wide web because the ISPs would love nothing more than to not have to pay for bandwidth and just turn their networks into oversized LAN networks.
Well its ok we will still have are massive websites like google because they will just create a server in every ISPs local networks xD.
An African American president is not a technological achievement. You're mixing apples and oranges again.
I don't know who modded this flamebait and I to fix it but I expect someone is going to be meta modded into oblivion.
Sprint has its fair share of cheating customers, peers, and even competitors. It's a phone company, that's what they do. I dare say this situation is entirely their fault. They created the link with only the intention of getting money from cogent, not to create settlement free peering (purely speculation).
Why should Cogent have a secondary route, but not Sprint? After all, Cogent's customers weren't the ones hurting, it was Sprint's customers. Where was Sprint's alternate route to Cogent?
Cogent is cheap because they built their network from the ground up to serve as an Internet backbone, through strategic acquisitions and planning. Sprint's backbone is piggy-backing on their phone network, just like Verizon, just like ATT, who also have a dark history with cheating people and taking from the tax coffer.
Even if Cogent was in the wrong, and was the sole cause of the depeering, that doesn't change the fact that tens of millions of people were affected by the spat.
Someone earlier posited regulation, but I feel their idea was drafted with only the needs of the end customers in mind. What should happen during a dispute is that the connection must be kept until *both* parties find alternate routes to each other, or until a court decides one side is in the wrong, and must pay for use of the connection. This essentially means no free rides, and none of the small people (us) get hurt in the process either. In all reality, that should have already been covered in the contract.
I don't use Cogent as the primary transit for any of my networks, but I do use Sprint. I am writing this as a person who's seen the Incumbents repeatedly try to hurt the new, more efficient competitor.