ISP Fights Causing Netflix Packet Drops

An anonymous reader writes "We've been hearing more and more reports of ISPs throttling Netflix and other high-bandwidth services lately. The ISPs have denied it, and even Netflix itself seems to believe them. If that's the case, what's going on? Well, according to this article, the blame still lies with the ISPs. While they may not be explicitly throttling connection speeds, they're refusing to upgrade network connections as they demand more money from content distributors. For example, Netflix pays Cogent to distribute their internet traffic. Cogent has an agreement with Verizon to exchange traffic — which works fine until the massive amount of traffic from Netflix makes it a lopsided arrangement. Verizon wants more money from Cogent, and one of their negotiating tactics is simply to stop upgrading their infrastructure so that service degrades. 'There are about 11 Cogent/Verizon peering connections in major cities around the country. When peering partners aren't fighting, they typically upgrade the connections (or "ports") when they're about 50 percent full, Cogent says. ... With Cogent and Verizon fighting, the upgrades are happening at a glacial pace, according to Schaeffer. "Once a port hits about 85 percent throughput, you're going to begin to start to drop packets," he said. "Clearly when a port is at 120 or 130 percent [as the Cogent/Verizon ones are] the packet loss is material."'"

Comments

Network vs Content providers

[future assassin]

If companies provide network access they should not be be allowed to be a content provider. Too much conflict of interest and they can concentrate on properly managing and not OVERselling their network.

Re:Network vs Content providers

[thule]

Cogent likes to think of themselves as a pure bandwidth company. No frills bandwidth for a great price. No content, no VoIP, nothing. They have colocation data centers, but that came when they purchased a company for their network.

Re:Network vs Content providers

[Lawrence_Bird]

That isn't even the real point, though I understand where you are going with it. The real issue here is that Verizon, Comcast, Cablevision, etc all have agreed to provide their customers with X/Y data connections for a hefty monthly fee. They are refusing to do what it takes for those customers to be able to use what they have paid for. In effect, we are back to the early internet days of ISPs oversubscribing dialup lines except now it is oversubscribing routing equipment during peak hours.

Netflix should know better

[Spazmania]

Cogent has a long history of instigating peering disputes with other networks. Normally I'd complain about Verizon's behavior but this is Cogent we're talking about. They have -zero- credibility.

Netflix should get benefit from desirability

[SuperKendall]

Netflix is breaking the long standing status quo. Last I checked, they accounted for ~30% of ALL of the traffic on the internet. Obviously that is going to skew the metrics, and that is why Netflix is trying to push their own CDN. I do not know the particulars there. IMO, if Netflix expects ISPs to pay for their CDN, they are on drugs.

Why? A lot of people might only get internet, or faster internet anyway, BECAUSE of netflix.

If I were not streaming stuff on Netflix I might very well just use a cellular internet connection and not get cable internet at all. Netflix is helping the ISP's make money, and Netflix should gain some benefit from that fact as a result.

Re:Netflix should get benefit from desirability

[vux984]

Right now Netflix is getting to benefit at the expense of ISPs. [...] My issue here is that Netflix is not a good champion to use to make the case for ISPs being tightfisted when it comes to circuit maintenance.

If the ISP cannot transfer me the traffic that I am paying them to transfer me, then they need to upgrade their circuits.

If they legitimately cannot afford to do that, then they need to charge me more money.

But if they cannot afford to do that, why is their investment news a string of "profits up 40% in 2013, ARPU up 9.4%, 14.7% year over year increase in FIOS revenues, 78c EPS 2013 vs 64c EPS in 2012, ... etc...

Imagine if your neighbor turned his house / apartment into a concert venue because there were not any ordinances to prevent it. All day long, you could not find a place to park, could not have guests over because they could not park, suffered brown and black outs because the music equipment kept blowing out the local transformers, etc. How would you handle that?

My neighbor and I both subscribe to Verizon to provide "parking and electricity" in this situation. So if Verizon can't provide
"parking and electricity" to me per our agreement, because my "neighbor" is using too much, then Verizon needs to build more parking and provide more electricity. If they have to charge me and my neighbors more, so be it... but see above regarding large increases in revenues and profits... it seems Verizon is simply failing to upgrade our electricity and parking and pocketing the money we paid them to do that instead.

Re:Net Neutrality laws?

[Anrego]

Yeah, but it's not like Netflix is using a free service. They are paying for that bandwidth. I assume they are paying quite a bit. More importantly, someone is selling it to them.

Re:sell is the key word. Cogent not paying Verizon

[msauve]

Customers PAY for those connections. Verizon's customers are paying to receive traffic from the Internet. Whether that's slashdot or Netflix doesn't matter, it behooves Verizon to deliver the service their customers are PAYING for.

Re:Can confirm

[schnell]

It has been a while since I was in the peering game - that was back when UUNet meant something and please get off my lawn - but here's the general idea:

To provide access to the Internet - either as a content host or a content consumer - you need a full view of routes to get to all other ISPs where your sources or destinations are. To get the routes, you need "peering" with the other ISPs or you need to buy "transit" from some other ISP that gets their routes via peering or transit.

Some ISPs are, frankly, more important than others because they provide access to more subscribers or more content than others - they used to be called "Tier 1" ISPs. Peering is valuable because it's traffic you're exchanging for free that you could otherwise be charging a lot of money for. Tier 1 ISPs generally agree to peer with each other because they all need each other, and it makes economic sense for them to say they're all on an equal footing - they were "peers." The economic rationale was because Tier 1 ISPs had to pay for large national or global networks, while Tier 2 or 3 ISPs had small or regional networks and that the Tier 1 ISP was bearing most of the cost of delivery. Traffic ratios were preferred to be equal (content vs. users) for peering, because if you're a content host with one datacenter and some outbound circuits, your cost is far less than having a big national network to serve end users - so web hosting/colo provider ISPs had a harder time getting peering with the big consumer/business access providers. The Tier 2 or Tier 3 ISPs would peer with each other freely because they had equivalent footprints, etc., but the big guys knew that access to their network was extremely valuable and it would be foolish to give it away for free.

So if you're a smaller ISP, and *you* need the Tier 1 more than *they* need you, don't expect to get peering. The ISP will tell you to buy transit from them, or at least buy transit from someone else who does (and the fewer "hops" to get from you to them, the better for your customers). Cogent may host much of Netflix, but they are by no means a Tier 1. This may no longer be the case - like I said, I have been out of the Tier 1 ISP world for years - but at least historically Cogent was known as a bottom feeder of the industry. They charged dirt-cheap rates but ran a crappy network and skimped on their upstream connections to cut costs.

So what's happening here most likely is that Cogent has either bought transit from Verizon and doesn't want to buy more and says "peer with us, we won't buy more." Or Cogent does have peering with Verizon but VZ has said, in effect, "you are not our 'peer'" and beyond a certain amount of peering bandwidth, you should start buying." Cogent is using Netflix to try to argue that "our content is more important to Verizon customers than the other way around," and Verizon is saying, "Um, nope." I won't say who I think is right or wrong here, but this is not the first time Cogent has had peering fights with other large ISPs and I think you can see a pattern here.

Re:sell is the key word. Cogent not paying Verizon

[i.r.id10t]

so my ISP should be paying me instead of vice versa? i certainly download much more than I upload....

Re:sell is the key word. Cogent not paying Verizon

[NormAtHome]

I know this topic has been beaten to death here, but I see it the same way you do. Verizon customers pay a toll (their monthly charge for internet access) to use Verizon's connection to the internet as a whole. No Verizon customer should have their data throttled no matter what site they are accessing as long as they are in compliance with Verizon's TOS.

Re: "Lopside"?

[sjames]

Simple solution, Netflix should alter their client so it always echos the data back. That should balance the transfer and make Verizon very happy.