Slashdot Mirror
Verizon's Accidental Mea Culpa
Barryke writes: Verizon has blamed Netflix for the streaming slowdowns their customers have been seeing. It seems the Verizon blog post defending this accusation has backfired in a spectacular way: The chief has clearly admitted that Verizon has capacity to spare, and is deliberately constraining throughput from network providers. Level3, a major ISP that interconnects with Verizon's networks, responded by showing a diagram that visualizes the underpowered interconnect problem and explaining why Verizon's own post indicates how it restricts data flow. Level3 also offered to pay for the necessary upgrades to Verizon hardware: "... these cards are very cheap, a few thousand dollars for each 10 Gbps card which could support 5,000 streams or more. If that's the case, we'll buy one for them. Maybe they can't afford the small piece of cable between our two ports. If that's the case, we'll provide it. Heck, we'll even install it." I'm curious to see Verizon's response to this straightforward accusation of throttling paying users (which tech-savvy readers were quick to confirm).
If people don't think bandwidth is a scarce commodity, how will we get them to pay through the nose for it?!?
https://www.eff.org/https-everywhere
Too big to fail, too arrogant to concede, too greedy to care. This news is all the more reason to regulate.
Was obvious people were going to figure out everything Verizon was saying is BS, and that they'd continue to get bad press about this. You'd think the PR droids spouting this stuff would talk to their tech people and listen. But they probably said "look, just give us a pretty graphic right?" "But, techs will see through your spin" "Leave that to us" "But it'll make us look even worse" "You don't get paid to deal with this" All too predictable, and the same techs are probably still being yelled at.
Waiting for an amusing sig.
Just connect to a VPN first and then use Netflix. You'll be able to clearly see how much Verizon is throttling. I've been using this as a workaround for a while now. I'm not sure why more people don't think of pointing this out when Verizon's tech support people claim there is no throttling.
Netflix has *yet* to pull up a dump-truck full of money to Verizon HQ.
SJW's don't eliminate discrimination. They just expropriate it for themselves.
Verizon's response was "Ok, but these cards tend to wear out pretty quickly so we'll need you to pay that amount each month. 5,000 streams may sound like a lot, but they don't last very long. A person watches a few movies a week, maybe a couple of youtube videos per day, that's like 20 streams in one week, and that's only one customer. Before you know it, you've used up all 5,000 of those streams and the card needs to be replaced."
"Oh yeah, and if it's coming from Netflix then we're using twice as many streams. We use one stream from Netflix to us, then another stream from us to our customers. Maybe you should really pay us that amount every week."
What happened was a bunch of salesmen and marketers at Verizon asked how they could explain the network throttling.
They obviously didn't understand the presentation so they assumed no one else would either.
Customers.
You're paying for a service, and nowhere does it say that they will discriminate against a particular service, such as Netflix.
It's obstructive business, against your customer's best interests, for no particular reason. It will also violate any given "net neutrality" laws that are / may come into effect.
Those laws are the answers. The reason for their existence is this sort of unnecessary posturing. And governments make companies do a lot of things against a company's best interests - all the time. It would be in the company's best interest to not pay tax, screw over its customers, not ship goods that have been paid for, be monopolistic, collude with others to enforce market prices, etc. The laws are brought in to stop that shit in the PEOPLE'S best interest, not the company's.
Not saying it's anywhere near perfect, but your post seems to want to back a corporation screwing over its customers and then (falsely) blaming its competitors and random third-party companies for that.
Sure. The content streaming from Netflix has been requested by Verizion customers. They've paid for access to the internet, which includes Netflix. They are the ones being throttled. Basically Verizon is trying to double dip here - get money from regular customers plus shaking down more from content providers. If Verizon really cannot handle the flood of Netflix content, shouldn't they raise the cost to the consumers to build out the Verizon network?
Part of the issue is that Verizon is a last-mile network, and does not sell symmetric bandwidth to its subscribers. So, the typical agreement between providers - where they each send about the same amount of traffic to each other and upgrade the interconnects to handle that traffic - will not work between Verizon and Level 3. Verizon (and the vast majority of other last-mile providers, including Comcast) will NEVER have a balanced interconnection with Level 3, because the home subscribers can all download far faster than they can upload.
Really, it's Verizon's customers who are causing all this bandwidth usage, so it should be Verizon ensuring that their interconnects can handle the requested bandwidth. If anything, Verizon (and Comcast) should be paying Level 3 for additional download capacity... but we all know that is never going to happen.
So when Netflix decided to pay Comcast, they were able to upgrade all of those remote trunks in ~24 hours, even though they cost of fortune?
Got anything better?
Remove the laws and regulations holding back community fiber projects.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
"I want it and my government friends have guns..." Is this the best we can do?
The reason Verizon can stay in business despite having "very limited interest in what their customers want" is because of municipal and state granted monopolies, federal grants and subsidies, and the reason they even exist at all is because of a government approved corporate charter. Why is "government friends with guns" an acceptable argument for them getting their way, but not an acceptable argument against it?
If you want a vision of the future, imagine a youtube comments section scrolling - forever.
Level 3 Communications is (or at least was) a really great company to work with. When the company I worked for was a huge customer of theirs, they did anything and everything to satisfy us. The claim of them volunteering to install 10GE cards really does sound like something they'd just do to make a large customer happy.
I really miss working with them.
Serious? Seriousness is well above my pay grade.
+1. Politicians at the state level have been paid off by cable companies and ISPs to squash competition from local municipalities. The cities who got in before the legislation are loving their services. It is absolutely insane what money can do in the political process.
Costs should be driven by the party responsible for the traffic being on the network. In the case of neflix traffic, that's _me_, the end recipient. And I've already ponied up to the cable company to cover their cost to transfer the bits to me. The cable co just wants to double dip.
Nice sentiment, but, unfortunately, a public corporation's responsibility is to its shareholders and their interests - which is simply $$$. (and probably executives and cushy bonuses, etc...)
And a government's responsibility is to take action against a company which is committing wholesale fraud against its customers by selling them Internet Service which promises bandwidth speeds which they are then purposefully not providing in order to shake down their customers and companies trying to provide services to those customers more money.
A government's responsibility is to ensure that companies that are given government licenses and franchise agreements which restrict competition in certain geographic areas are providing the service that the people of that area want and need at a fair price.
A government's responsibility is to ensure that companies which get too big, hold too much market share and are too horizontally or vertically integrated are broken up so that there can be real competition and a real free market.
Verizon to level3: "Our traffic from netflix moved over to Level3 last night... very strange, anyways we need to increase our capacity..."
Level3 to Verizon: "Ok, that will be $X"
Verizon to level3: "um... That's 300% higher than any other provider out there..."
Level3 to Verizon: "suck it... your monies are belong to us"
Except that this fictional exchange you've created, in which Level 3 is extorting Verizon for more, is easily refuted by using either blog post. For instance, from Verizon:
Netflix did not make arrangements to deliver this massive amount of traffic through connections that can handle it.
[...] Netflix is responsible for either using connections that can carry the volume of traffic it is sending, or working out arrangements with its suppliers so they can handle the volumes. As we’ve made clear before, we regularly negotiate reasonable commercial arrangements with transit providers or content providers to ensure a level of capacity that accommodates their volume of traffic.
Which is a nice way of saying, "Level 3 is refusing to negotiate rates for more capacity with us, so we've refused to give them more." Level 3's blog post also affirms that the issue is Verizon's refusal to act:
Verizon has confirmed that everything between that router in their network and their subscribers is uncongested – in fact has plenty of capacity sitting there waiting to be used. Above, I confirmed exactly the same thing for the Level 3 network. So in fact, we could fix this congestion in about five minutes simply by connecting up more 10Gbps ports on those routers. Simple. Something we’ve been asking Verizon to do for many, many months, and something other providers regularly do in similar circumstances. But Verizon has refused.
Even without the blog posts, it should be obvious your notion makes little business sense. Level 3 is in no business position to play hardball like you've suggested. If they sacrificed on performance as a ploy to double-dip (i.e. get both Netflix and a lower-tier ISP* like Verizon to pay), Netflix would simply take its traffic to a different Tier 1 ISP that doesn't play those sorts of games, since the double-dipping would be hurting their bottom line. Or, at the very least, they'd be calling out their own ISP, rather than calling out the customer's ISP.
On the other hand, as a lower-tier ISP, Verizon has a monopoly on its own end users: if you want to reach them, you MUST go through them. If Verizon tries to double-dip by getting money out of both the higher-tier ISP and its end users, the end users won't understand what's going on, and in many cases they lack any viable alternatives anyway. Meanwhile, the higher-tier ISP can't switch out for a different peer, since Verizon is the only way to get to those end users.
Besides which, it's not like Netflix's switch from Akamai to Level 3 took Verizon by surprise, as you suggest, since it happened way back in 2010 and has been working fine for most of that time. If there was a problem resulting from the switch, it would have come up before now. Which is to say, this isn't a "Wow! Level 3's traffic is suddenly skyrocketing and we can't keep up!" situation. Rather, it's almost certainly a, "Hey, that Comcast company had a good idea to try getting money out of both sides...let's see if we can do it too!" situation, given the timing of it all.
* A quick aside: I'm well aware that Verizon also maintains a Tier 1 network, but Tier 1 networks rarely connect directly to end users. That's what lower-tier networks do. Moreover, the defining characteristic of a Tier 1 network is that it enjoys free peering with other Tier 1 networks. As such, the Verizon network being discussed here is clearly not their Tier 1 network, but rather a lower-tier one they control (e.g. a Tier 2 or 3 network) that has direct access to their end customers.
If you are paying for 75/25 or 50/25 and they are throttling it at the borders of their network, then you aren't getting the bandwidth you are paying for... downgrade your service. That $10, $20 or more per month they aren't getting from you because of their throttling practices should get their attention.
Bingo. Well, almost; it's a little more nuanced than that. Costs should be driven by the party responsible for the traffic being on *your* network. For Verizon, that's Verizon's customer; for Level3, that's Netflix. And they both already pay their providers. Where Verizon and Level3 peer, it's a matter of recognizing that the imbalance of traffic across that link is caused by Verizon's customers requesting more traffic than they (can) return. Thus, Verizon caused the imbalance and should therefore pay for it. If Verizon primarily sold symmetrical access and allowed their users to run servers, there would likely be a balance, and if there was not, they'd have a leg to stand on here, but they don't sell symmetrical access to the end user and they don't have a leg to stand on in this debate; what they do have is a monopoly on Verizon customers, which they're attempting to abuse right now, which should warrant an anti-trust suit, if anything. No additional regulation needed.
APK quotes people (including myself) without context and should not be trusted. Just thought you should know.