Slashdot Mirror
Verizon Accused of Intentionally Slowing Netflix Video Streaming
colinneagle writes "A recent GigaOm report discusses Verizon's 'peering' practices, which involves the exchange of traffic between two bandwidth providers. When peering with bandwidth provider Cogent starts to reach capacity, Verizon reportedly isn't adding any ports to meet the demand, Cogent CEO Dave Schaffer told GigaOm. 'They are allowing the peer connections to degrade,' Schaffer said. 'Today some of the ports are at 100 percent capacity.' Why would Verizon intentionally disrupt Netflix video streaming for its customers? One possible reason is that Verizon owns a 50% stake in Redbox, the video rental service that contributed to the demise of Blockbuster (and more recently, a direct competitor to Netflix in online streaming). If anything threatens the future of Redbox, whose business model requires customers to visit its vending machines to rent and return DVDs, it's Netflix's instant streaming service, which delivers the same content directly to their screens."
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
This wouldn't be the first time people have had issues with Cogent having saturated peering links. A common complaint among Cox customers is that latency is high to certain WoW servers, and saturated Cogent links has been found to be the cause - and they don't seem particularly interested in fixing it.
Careful with names containing L slashdot.org/~AiphaWolf_HK slashdot.org/~AlphaWoif_HK slashdot.org/~AiphaWoif_HK
My provider solved the fairness problem by making everything slow and spotty.
Table-ized A.I.
...or does that not apply to internet service providers?
In Canada it does, back a few years ago Rogers was involved in throttling everything, even though they said they weren't. Took the work of a few very determined people who brought it before the CRTC, and were told to stop or face fines. As a fun note, Rogers and Bell Canada were two of the greatest throttlers in the world back then.
Om, nomnomnom...
...or does that not apply to internet service providers?
Nothing prevents Cogent from purchasing access to Verizon network. What Cogent expects instead, is for Verizon to purchase more network ports so Cogent can offload their traffic for free. "Peering" is usually mutally beneficial, meaning traffic ingress and egress is balanced. If it is not, it does not make sense to provide free access and it is fair to expect on of the parties to pay.
Essentially, Netflix pays Cogent as their "ISP". Cogent probably won that deal with their ridiculously low pricing. And now Cogent expects Verizon to invest in their network so that they can act as an extension of the Cogent network, through a "peering" agreement.
Probably necessary disclaimer: I am not in any way affiliated with Cogent nor Verizon. I do, however, work for a vendor of high quality networking equipment.
I'm not a complete idiot... Some parts are missing.
um that is the entire point of the internet.
I pay an ISP, you pay an ISP, Company A, B and C all pay different ISP's.
It is the 5 different ISP's job to share the data load between them. Once you start having ISP's charge different rates to other ISP's the entire network collapses into AOLhell. Once ISP's stop working together to connect each other entire value of all ISP's fails. ISP's solely exist to connect tiny communities to larger ones.
i thought once I was found, but it was only a dream.
Verizon already got paid, by their customers, the ones who are requesting to stream from Netflix.
Verizon already got paid, by their customers, the ones who are requesting to stream from Netflix.
More importantly, Verizon's paying customers -- the ones who are requesting to stream from Netflix -- are expecting Verizon to invest in their network so that they can deliver the contracted-for services. The fact that Netflix uses Cogent versus Billy Bob's Bass Boat, Bait Barn, and Content Distribution Network does not really play a role here.
The Busy Coder's Guide to Android Development
Yeah, 'cause the rest of the country doesn't hate those with Google fiber enough yet.
Is it just my observation, or are there way too many stupid people in the world?
There are but they were pretty well gutted back in the days of the Reagan Administration. Now, the ones that are left are mainly ignored. The big exceptions, like the Microsoft case, usually come as political punishment or when the infraction is so blatant that it cannot be ignored.
If we had a Justice Department that was more than a bunch of cronies and amateurs, there wouldn't be a single telecom with any interest in content providers, and there certainly would not have been any of the mega-mergers in the airline industry and others.
We haven't had a real Justice Department since before the days of Ed Meese. Meese is really the very model of the modern attorney general, who believes his main job is to make sure no rich people get in any trouble and to find ways to subvert the Constitution.
You are welcome on my lawn.
More likely YouTube, too, is being throttled or at least left in a state of benign neglect. Verizon FiOS, supposedly to be the fastest anywhere, consistently has trouble delivering YouTube videos. I work on many different networks and peering points but the only one that has trouble with YouTube is Verizon FiOS. Even if the YouTube video is serving from a local edge server (Ashburn) it will pause within the first twenty seconds each and every time.
Oddly enough, and likely because we are only down the road from AWS-East, we never have trouble with Netflix or Amazon Instant Video on our FiOS connection.
Cox never had any sort of problem but that might be a lack of customers since FiOS came into town.
Kriston
"Hey, we're not throttling. It is just that our peering is maxed out. Use this other service, it works better for our customers."
Totally legit way of doing this. I haven't seen any Net Neutrality discussions cover this possibility.
This is an example of a "natural monopoly", where a limited community resource is owned as property by a corporation. In this case it's the easements and permission needed to run the phone lines, and the RF spectrum for cell-phone service.
If you treat the resource as property, you get the situation we have now: high fees for access and discouraged use. Phone service has high monthly fees (access) as well as data caps, fixed monthly "minutes", and roaming charges (discouraged use). Similarly for internet: high monthly fees (access), data caps, throttling, kicking off high-usage users, and so on (discouraged use).
As an alternative, take the revenues from the carriers and divide by the total minutes of service. I don't know what that figure actually is, but for purposes of discussion let's say it's 5 cents a minute. A similar calculation can be done per gigabyte of internet data.
Suppose the government mandated that carriers could only charge that amount or less, with no other restrictions. Any phone could be used with any carrier, and you choose a carrier at call time by scanning the available carriers like we scan wireless access points. (You wouldn't explicitly scan for each call. Most likely you choose one carrier as default, like we now do with wireless access points.)
Now instead of making money by getting people to sign up and not use the service, carriers make money the more people use the service. They have to encourage more people to use it, and for longer periods. They have an interest in putting unused capacity to work, and promoting innovative new uses. If a channel is overallocated, they have an interest in building out more capacity.
The reasoning can be applied to cable TV, internet, and phone service. If the cable company can only charge 15 cents per hour of viewing/downloading (whatever the fee structure works out as), then they will encourage more usage rather than throttling.
If this change is made, the existing players will make the same profit as now: initially the profits are the same, and no workers need be laid off. Their bottom line doesn't change, only their focus of service.
It's game theory: change the rules so that the outcome is more desirable.
Sure they do! A few years ago, Microsoft was found guilty of violating the laws, and received a harsh sentence. They had to give people coupons or something...
"Convictions are more dangerous enemies of truth than lies."
Nothing prevents Cogent from purchasing access to Verizon network.
Verizon is a Tier1. Tier 1 providers do not buy transit, period.
"Peering" is usually mutally beneficial, meaning traffic ingress and egress is balanced. I
No: settlement-free peering is usually mutually beneficial, meaning the benefit to both parties of the relationship is larger than the cost.
Traffic ratios are almost irrelevent. Although, they are commonly used for negotiation purposes.
Pushing more traffic into Verizon's network than you pull, means that Verizon's users are requesting data from you.
If Verizon were not a monopoly; there is no question that this would be mutually beneficial --- if there is poor connectivity to Netflix, or greater latency / worse performance, then competing providers would be favorable for subscribers.
Better connectivity to Netflix is beneficial for an ISP; moreso, than the cost of some extra ports.
Verizon already got paid, by their customers, the ones who are requesting to stream from Netflix.
Not only that... if you are ISP, and you have enough traffic to Netflix; Netflix will provide a 'local cache box' to install on your network. OpenConnect hardware appliance.
Netflix pays for the hardware and such.
Large ISPs such as Verizon, can potentially put multiple boxes on their network, so they save cost and do not transport large amounts of Netflix traffic long distances.
Verizon chooses not too. Obviously, they cannot think their customers do not value Netflix. Clearly, they don't care much about their customers -- or there's an alterior motive; or just plain ignorance, blindness, and stupidity.
And peering is like agreeing to allow guests to use your toilet as long as you are allowed to use theirs.
Okay. Fine.
But what Cogent does after making this agreement with its neighbors is open up a buffet next door with a big sign directing its customers to your bathroom. Then when their customers complain about the backups and stink, Cogent demands you build more toilets.
[BEGIN ISP REASONING MODE] Of course, it does. You see, Netflix makes lots of money. Partly, they make that money in a method involving Verizon's network. Verizon doesn't get any of that money. Therefore, it deserves lots of money from Netflix. What's that you say? Verizon gets paid by their customers and Netflix pays their ISP? *sticks fingers in ears* LA-LA-LA-LA-LA-LA-LA I CAN'T HEAR YOU!!!! LA-LA-LA-LA-LA-LA-LA GIVE ME MORE MONEY!!!! LA-LA-LA-LA-LA-LA-LA-LA [/END ISP REASONING MODE]
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
But Redbox has a streaming service now. Coincidentally, it's owned by Verizon. But I'm sure Verizon doing this in no way is a plot to make people think Netflix is horrible and Redbox Streaming is wonderful. I'm positive that they're not trying to leverage their network to benefit one of their unrelated services over a competitor. After all, big companies are owned by good, kind-hearted people who only seek to make as many people happy as possible. (Also, the sky is the most beautiful shade of orange in the world I live in.)
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
You do have mod points...it's just taking time for them to show up because you're throttled...
Sig Follows: "Suppose you were an idiot. And suppose you were a member of Congress. But I repeat myself." -- Mark Twain
There's absolutely no reason I should be footing the bill for a service I have no intention of using.
You realize that a caching appliance for a heavily-used service like Netflix could save an ISP bandwidth costs, right? Presumably more than enough to offset the cost of switch ports, rack space and electricity.
Or they'd simply rather not spend time and money to solve someone else's problem?
You're looking at this the wrong way. The problem is their customer not being able to access the services they wish to in a reasonable manner.
It's not like rack space is free, or electricity is free, or ensuring that someone else's hardware isn't going to harm your network is free. If I were an ISP, Netflix would "get" to install hardware in my network over my dead body - simply because I DO NOT TRUST HARDWARE AND SOFTWARE I HAVEN'T VERIFIED.
You do realize that the whole point of the internet is to connect to servers, clients, and peers of an unverified nature, right? And if they co-locate for any of their clients, they already deal with this issue on a daily basis? Go ahead and google Verizon colocation services, just for fun.
What about the people who AREN'T Netflix customers and DON'T want to pay for someone else's service? Why should my ISP fees be used to help someone else stream movies I can't access?!
Well, the benefit to their other customers would be that their connection to other servers outside of Verizon's network wouldn't be impeded by the congestion of their customers who would like to stream said movies. Keep in mind, the customer who wants to watch movies on Netflix have exactly as many rights as the customer who wants to play MMOs, or the one who wants to send emails. This benefits all their customers - just not their RedBox business.
If Netflix wants to solve this, they can talk to Cogent and help Cogent come up with a solution that isn't making Verizon and their non-Netflix subscribing customers foot the bill. There's absolutely no reason I should be footing the bill for a service I have no intention of using.
It must be a source of relief to you to know that all those services that you use are vitally important to all the other Verizon customers. Or just maybe those other customers' service fees pay for those services they use, on average.
Sure I'm paranoid, but am I paranoid enough?
Or they'd simply rather not spend time and money to solve someone else's problem?
Verizon's bandwidth is indeed Verizon's problem.
It's not like rack space is free, or electricity is free...
The backspace and electricity demands of an OpenConnect box are likely negligible in comparison to the overall strain placed on the network by Verizon customers using Netflix en masse.
...or ensuring that someone else's hardware isn't going to harm your network is free. If I were an ISP, Netflix would "get" to install hardware in my network over my dead body - simply because I DO NOT TRUST HARDWARE AND SOFTWARE I HAVEN'T VERIFIED.
Good. You sound like a capable admin. Now, what's to say you cannot verify the box?
What about the people who AREN'T Netflix customers and DON'T want to pay for someone else's service? Why should my ISP fees be used to help someone else stream movies I can't access?!
By having an ISP you are splitting the cost of using the network among X number of people. Since the cost of an OpenConnect box is rackspace + electricity + verification / customer base, the cost to you alone is exceedingly low.
There's absolutely no reason I should be footing the bill for a service I have no intention of using.
This mentality is destroying the country.
I work for a telco, and not too long ago I got to chat with one of our VPs about why this happens. I'm a total net neutrality guy, but after talking to him I understood his point of view a bit better.
With most large content providers, like google for example, ISPs can go to them and say "hey, we're getting a lot of traffic from you. It's cheaper for us if we can make arrangements that are beneficial to the both of us." and then the ISP and the content provider enter into an agreement where the ISP pays a bulk rate for trunks to a network, and the content provider remains on that network and gives plenty of warning before switching so the ISP can make sure that they have enough capacity in that direction.
Netflix however, doesn't make these kind of agreements. The switch providers and hosting at will. The ISP will pay for large trunks leading to where the majority of netflix traffic is coming from and then Netflix will suddenly drop that host and switch to another. Suddenly 20% of the ISPs traffic is coming from an entirely new network. But they are still locked into a contract with that other network.
Also, Netflix has no interest in the health of the ISPs network. If Netflix had a financial interest in the health of the network they could do some rather simple things to help the isp, like encourage users to queue up movies ahead of time, have them download at off peak times and then play when they wanted to watch them. This is was cable companies do after all... but netflix has no interest in this sort of thing and as far as the ISP is concerned is doing is best to be as damaging to the network as possible.
I'm still all for net neutrality, but its good to understand the ISPs concerns. They aren't just out to thwart Netflix. But Netflix is digging their own grave on this one.
Yes, it is. The job of the ISP is to provide their paying customers access to 'TheInternet'. That is still the promise they make, and still their obligation. If they can't meet that obligation they should go do something else.
They are using publicly subsidized infrastructure on publicly owned land to seek rent on a network they are not investing in or improving. So fuck them.
Samsung took back my unlocked bootloader because Google wants me to rent movies. They're both evil.
That is the heart of the matter. They're so used to huge profits for next to no effort that the notion of giving customers value for their money never enters their mind. And they'd laugh at the suggestion of "invest in your own network".
There really needs to be some anti-trust cases brought against the biggest telecoms. Threaten to do to them what was done to AT&T decades ago. You'd see service improve everywhere in a big hurry.
You are welcome on my lawn.
Are you sure? Why is the traffic being routed to Verizon? Because Verizon is the optimal path for that traffic. The bulk of that traffic is gong to Verizon's customers or the customer of Verizon's customers. No one is asking Verizon to do anything for free. There may be some case where Verizon has a shorter path to another provider than Cogent, but that will be a small fraction of the traffic.
Verizon chooses not too. Obviously, they cannot think their customers do not value Netflix. Clearly, they don't care much about their customers -- or there's an alterior motive; or just plain ignorance, blindness, and stupidity.
No Verizon chooses not to because they can't charge $100 a month for cable video in a free market with actual competition. Thus they stop delivering other video service over the internet eliminating the competition.
Who is John Galt?
There is a good chance it's more complicated than just this. Remember this is Cogent we are taking about here and they are famous for trying to get downstream isps to pay the entire cost of peering upgrades and have also been known to actively cut back on peering points with other providers.
They are also famous for causing most of the IPv6 routing problems that affect day to day useage.
Bingo, we have a winnar! This is just another case of a large corp fucking an audience that probably has NO say or choice in the matter (from what I've seen unless you live on the coasts you get one ISP with a useful speed and the other crap, in mine its 8-20Mbps for cable and 3Mbps for DSL) because God fucking forbid they actually provide what they were fricking PAID FOR by those customers instead of trying to fuck them for every cent they can squeeze!
What will happen is the vast majority, which don't read tech sites and most likely will never hear a word of this from their corporate kissing local MSM, will try to use netflix only to get a stuttering mess. They will say "Oh Netflix sucks" because they won't have any way of knowing its their ISP that is MAKING it suck so they will have to use the alternative...which is owned by Verizon who will make that much more money off all those people that have to use Redbox instead of the service they originally wanted to.
This is why we frankly shouldn't allow any company that owns ISPs to own content as they will always end up tilting things in favor of their content. Of course with our government being such corporate whores that every politician gets a free set of kneepads i doubt shit will be done, but that is why our prices put us in the top 5 but the quality and speed puts us behind countries like Romania.
ACs don't waste your time replying, your posts are never seen by me.
Verizon chooses not too. Obviously, they cannot think their customers do not value Netflix. Clearly, they don't care much about their customers -- or there's an alterior motive; or just plain ignorance, blindness, and stupidity.
That's the question. We have a similar situation in the UK where YouTube and iPlayer are unusable on Virgin Media between about 3:30PM and 11PM in many areas. If you switch over to a VPN or proxy that blocks their internal CDN everything is fine, indicating that the CDN cache boxes installed by Google and the BBC are inadequate for the demand. As a result people go on BitTorrent instead, causing more degradation of the network.
There are only two reasons I can think of for this being the case:
1. Virgin Media is incompetent and there is some really lame reason like running out of physical rack space or network ports that prevents them getting more cache boxes in.
2. Virgin Media is trying to sabotage streaming video services in order to drive people to their cable TV products instead.
I don't know much about Verizon. Care to speculate?
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
Capitalism. There are a pile of ISPs. If one sucks, vote with your dollars. Drop em and get a real ISP. Simple, done.
I'd love to live in your magical land that has a pile of ISPs. I get a choice between ATT and TimeWarner, which both suck.
He effected a bored affect.