First Detailed Data Analysis Shows Exactly How Comcast Jammed Netflix

An anonymous reader writes John Oliver calls it "cable company f*ckery" and we've all suspected it happens. Now on Steven Levy's new Backchannel publication on Medium, Susan Crawford delivers decisive proof, expertly dissecting the Comcast-Netflix network congestion controversy. Her source material is a detailed traffic measurement report (.pdf) released this week by Google-backed M-Lab — the first of its kind — showing severe degradation of service at interconnection points between Comcast, Verizon and other monopoly "eyeball networks" and "transit networks" such as Cogent, which was contracted by Netflix to deliver its bits. The report shows that interconnection points give monopoly ISPs all the leverage they need to discriminate against companies like Netflix, which compete with them in video services, simply by refusing to relieve network congestion caused by external traffic requested by their very own ISP customers. And the effects victimize not only companies targeted but ALL incoming traffic from the affected transit network. The report proves the problem is not technical, but rather a result of business decisions. This is not technically a Net neutrality problem, but it creates the very same headaches for consumers, and unfair business advantages for ISPs. In an accompanying article, Crawford makes a compelling case for FCC intervention.

Comments

Their answer to oversubscription as well

[Anonymous Coward]

Think twice next time you wonder why you aren't getting your advertised speed...

Re:Their answer to oversubscription as well

[ourlovecanlastforeve]

That's quite a loaded statement.

Business buggery is not the only reason you might not get the advertised speed.

First of all, ISPs advertise "up to" a specific speed, which means that's the maximum bandwidth you're allowed.

It doesn't state or imply that you'll receive that speed consistently.

It means that, assuming the network is capable of that speed, if you were capable of getting higher speeds you'd be capped at that speed.

Re:Their answer to oversubscription as well

[Anonymous Coward]

"First of all, ISPs advertise "up to" a specific speed, which means that's the maximum bandwidth you're allowed.

It doesn't state or imply that you'll receive that speed consistently."

Well, how about I pay "up to" the amount they want, then? I can throw that (in small print) on the back of a check I send them, to make it all legal and such.

Re:Their answer to oversubscription as well

[JWSmythe]

Sure you can do that. They'll (kind of) accept it too. Reducing your payment amount will result in reduction of your service.

When you pay less than your bill, they'll simply reduce your service to 0.

Re:Their answer to oversubscription as well

[David_Hart]

Think twice next time you wonder why you aren't getting your advertised speed...

And another article about Comcast throttling Netflix without any background or context.

Backbone providers have peering agreements. Usually, if both backbone providers (i.e. Comcast & Verizon) are sending close to the same traffic between each other, the peering agreement is free for both parties. In the Netflix case, Netflix went with a small backbone provider, likely due to cost. The problem is that the backbone provider they chose sends way more traffic than they accept. Typically, this type of peering agreement means that the smaller backbone provider (i.e. Cogent) pays fees to the larger backbone provider (Comcast). It's my understanding that Cogent wouldn't or couldn't pay these fees, so Comcast throttled them.

Because Cogent couldn't or wouldn't pay the fees and customers were complaining, Neflix agreed to pay the peering fees for Cogent. Though, this isn't how the media or Netflix presents it.

I hate Comcast as much as anyone. I think that they are essentially a monopoly that takes advantage of their customers by increasing prices where there is little to no competition (a toothless FCC doesn't help). But, in this one instance, it's my opinion that Comcast had a good case against the provider used by Netflix and that, by selecting the lowest cost provider and possible knowingly selecting one that is a bit sketchy on the peering side, Netflix had some responsibility.

Re:Their answer to oversubscription as well

[Anonymous Coward]

And another article about Comcast throttling Netflix without any background or context.

And yet the "study" shows that multiple ISPs that used Cogent for transit ALL SAW THE SAME DEGRADATION IN PERFORMANCE at the moment Netflix started dumping all its traffic onto Cogent's network.

The conclusions of the study and other commentators are uninformed.

The study shows what all the ISPs were saying was true: that Netflix found the cheapest carrier they could find and that cheap carrier couldn't handle it, instead expecting to dump it all onto other networks.

Re:Their answer to oversubscription as well

[thaylin]

Except you are confusing a transit and a consumer endpoint. Transit providers normally peer, but an endpoint is going to have more traffic coming in then going out because their consumers are requesting it, ALWAYS, but this is the first time they have been able to pressure people into these types of agreements.

Peering agreements between transit providers is fine, but not when an endpoint bullies a service providor.

Re:Their answer to oversubscription as well

[ShanghaiBill]

Think twice next time you wonder why you aren't getting your advertised speed...

It is unrealistic to expect them to build out enough bandwidth to let everyone max out at the same time. They would need far more infrastructure, and YOU would have to pay for it. Car analogy: If a highway carries 10,000 vehicles per day, do you complain that it doesn't have 10,000 lanes?

Re:Their answer to oversubscription as well

[Duhavid]

"If a highway carries 10,000 vehicles per day, do you complain that it doesn't have 10,000 lanes?"

I certainly do.

Re:Their answer to oversubscription as well

[David_Hart]

No, I'm not confusing transit and endpoints.

You are right that the end point connection isn't one sided. That's why we all pay monthly fees to our provider to pay for bandwidth. Our provider, though, has peering agreements with other providers. If this peering is unequal, some of the money that we pay our provider goes to pay for peering fees. In the case of Netflix, their provider refused to pay the peering fees. If our provider screws up, we suffer. We then have a choice to change providers. Netflix chose to pay Comcast instead (probably the cheaper option, factoring termination fees, etc.). But... they did have other options...

Re: Their answer to oversubscription as well

[GigaplexNZ]

The problem is that the backbone provider they chose sends way more traffic than they accept.

And consumer ISPs give asymmetric speeds most of the time with EULAs that forbid running servers. It's pretty obvious that they'll accept more data than they send by design, so it's unreasonable for their peering agreements to assume symmetric transfers.

Re:Their answer to oversubscription as well

[fibonacci8]

When a toll road sells 10.000 people a monthly pass for a lane each, and then offers up to 5 lanes at a time during peak hours due to "maintenance", and yet has another 9,995 lanes constructed, uncluttered, and for sale in advertisements to other bidders I would certainly complain.

Re:Their answer to oversubscription as well

[fibonacci8]

Making the toll road operator look even better, one of the competing toll road projects decides that those who car-pool should either pay more or drive at reduced speeds.

Re:Their answer to oversubscription as well

[jythie]

'How' is simple. Just be the party writing the contract rather then signing it. If you can not arrange such a thing, it is your own fault for being poor.

Re:Their answer to oversubscription as well

[Anonymous Coward]

First of all, ISPs advertise "up to" a specific speed, which means that's the maximum bandwidth you're allowed.

Then the ISP purposefully makes sure you can't use the bandwidth you're paying for. Bandwidth within your ISP and to it's peer should be congestion free at all times. If it's not, then they're not making a best effort.

Re:Their answer to oversubscription as well

[Anonymous Coward]

General rule, you pay for transit, not access, unless you're in the business of reselling access, like a CDN. Another general rule, last mile ISPs pay for their bandwidth, they don't have others pay them.

Re:Their answer to oversubscription as well

[Anonymous Coward]

It is unrealistic to expect them to build out enough bandwidth to let everyone max out at the same time.

Not true. Standard fiber equipment can easily handle this. It would be cheaper to build a fiber network that can handle all customer at 100% at the same time, than the cost of upgrading current copper networks. Current fiber consolidators can handle 3-4tb/s and modern core routers can handle hundreds of terabits up to nearly 1pb/s.

Re:Their answer to oversubscription as well

[morethanapapercert]

Given the way many broadband ISPs oversubscribe their services, I consider weasel words like "up to X speeds" in the fine print while all the headlines and banner texts say "Now surf at X*" or "Fastest Internet in Y county!*" with all those asterisk footnotes to be a form of corporate buggery.

Re:Their answer to oversubscription as well

[AK Marc]

The problem is that the backbone provider they chose sends way more traffic than they accept. Typically, this type of peering agreement means that the smaller backbone provider (i.e. Cogent) pays fees to the larger backbone provider (Comcast).

Typicaly, the small provider isn't a host, but is a small ISP. residential ISPs accept way more traffic than they send. They get the highest fees.

Part of the problem is that one ISP will download 10G and upload 1G and they'll be charged 10G rates, and the next ISP will upload 10G and download 1G and they'll be charged at 10G rates. The backbone will move 11G of traffic while being paid for 20G. And they'll complain about it all the way to the bank.

Re:Their answer to oversubscription as well

[AK Marc]

The requestor pays. That was the old system. They are now finding out ways of blackmailing the sender into paying. My fees to my ISP pay for all the bandwidth needed to get the bits from Netflix to me. Unless my ISP blackmails Netflix.

Re:Their answer to oversubscription as well

[tepples]

But how feasible is it for a typical U.S. resident to qualify for a work visa in your country?

Re:Their answer to oversubscription as well

[TangoMargarine]

But then I'm not in US.

Ah, so you're almost entirely irrelevant to this conversation. Thanks so much.

Re:Their answer to oversubscription as well

[Anonymous Coward]

No, I'm not confusing transit and endpoints.

Parent's choice of words here doesn't help, but he's talking about edge vs. transit networks. Comcast is mostly eyeballs/edge, so the relationship between Comcast and Cogent is not transit provider to transit provider but transit to eyeballs. The possible counterpoint to this that Comcast likes to go with is that they also are a transit provider (I don't personally know their ratio of transit to eyeball traffic, but my gut says it's skewed pretty heavily to the latter).

Regardless:
In the Netflix example, Comcast is the edge not transit: the final destination for the packet is within Comcast's network; period. Call them the "terminating network" if the other language doesn't suit. The point is that Comcast is the only way to reach those eyeballs since those eyeballs reside within their network. If the traffic were transiting Comcast's network but not terminating there, then you could start playing the ratio game, but that's not what's happening here.

The article takes issue with last mile providers holding their eyeball customers hostage. Ratios are irrelevant in this discussion and a bullshit line from the eyeball networks in question.

Netflix chose to pay Comcast instead (probably the cheaper option, factoring termination fees, etc.). But... they did have other options...

Sure:
1. Try another higher tier provider like Level 3. Oh...wait..that didn't work out so well either.
2. Provide caching boxes inside the eyeball network to reduce the amount of traffic traversing the eyeball network's edge. Tried that; the eyeball networks in question refused.
3. Peer with the eyeball network in the best interest of themselves and their mutual customers, paying for their own equipment and transport and meeting the eyeball network at common exchange points. Tried that too: Comcast and the other boys won't play ball and require payment.

Which led us to the conclusion of the saga: Netflix ends up in paid peering agreements with the eyeball networks in question (plus possibly some number of caching boxes, but not hosted within the eyeball network's facilities) because said networks have a sufficient number of captive eyeballs to be able to extract payment.

Re:Their answer to oversubscription as well

[repka]

You're welcome.

Re: Their answer to oversubscription as well

[cthulhu11]

Peering is done with symmetric connections. Last mile consumer isn't -- the wire's capabilities are generally allocated more for downstream than upstream traffic. This is sophistry.

Re: Their answer to oversubscription as well

[GigaplexNZ]

My point was that if the vast majority of the ISP customers only download, they can't expect to have any symmetrical peering as there's no traffic to balance it out.

I'd like to hear Bennett Haselton's opinion

[Anonymous Coward]

Bennett Haselton is a frequent contributor, and I tend to hold-off judgement on these things until I read his fine points on the topic. I was recently particularly moved by his work on line queues at Burning Man. It changed my entire life. I now piss sitting down.

Re:I'd like to hear Bennett Haselton's opinion

[Anonymous Coward]

I don't know whether to mod you down for merely mentioning the fucktard's name, or mod you up for the good use of sarcasm. Fuck. Modding down because someone can fall victim to Poe's law and trip themselves up something fierce. Haselton....become a Burning Man. Will ya?

Re: I'd like to hear Bennett Haselton's opinion

[Anonymous Coward]

He's a complete amateur, I get all my opinions from hughpickens.com, the true font of all wisdom !

Re: I'd like to hear Bennett Haselton's opinion

[Anonymous Coward]

Can you please repost the info from that site? I installed APK's hosts file and I can't seem to reach anything on the internet except slashdot and reddit.

Sigh...

[MightyMartian]

"In an accompanying article, Crawford makes a compelling case for FCC intervention."

That won't work unless it comes with a check with seven digits attached to it.

Re:Sigh...

[Krojack]

And that won't really work because an even larger check from the cable companies will always follow. Hell they most likely just give them a credit card with an unlimited balance that draws right from the cable companies bank account.

Re:Sigh...

[Em Adespoton]

And that won't really work because an even larger check from the cable companies will always follow. Hell they most likely just give them a credit card with an unlimited balance that draws right from the cable companies' customers' bank accounts.

FTFY

Re:Sigh...

[houstonbofh]

Someone needs to file a Sherman Anti-trust Act case here... It covers most of the egregious stuff, like paying cities not to lay fiber.

Re:Sigh...

[Anonymous Coward]

That's fine, Comcast already bought the Dems and the Obama administration so nothing will come of this.
How Comcast bought the democratic party http://www.nationalreview.com/article/375116/how-comcast-bought-democratic-party-matthew-continetti

Re:Sigh...

[Anonymous Coward]

Not to mention paying cities for exclusive use of it (though they don't actually use it).

Common Carrier

[Anon-Admin]

Once again, a call for net neutrality will ensue. All we really need is for the FCC to call them Common Carriers and apply the age old law.

It has already been applied to Telecoms and Utilities, just apply it to the ISP's and be done with this crap.

http://en.wikipedia.org/wiki/C...

Re:Common Carrier

[Burdell]

That would have zero impact. This is like the telephone company in city A have 96 channels to the telephone company in city B, but then 100 people try to make calls. Only some of them will go through, and that's a capacity issue, not regulated by Common Carrier status. They are not discriminating based on callers or anything, they are just "decliining" to upgrade capacity. In some cases, that could be regulated by state PUCs/PSCs, but AFAIK it is not normally. It is just up to the two carriers to reach an agreement.

This type of thing happened a lot in the early dialup ISP days, when telecom deregulation spawed a lot of CLECs that had to connect to ILECs to carry calls. The ILECs structured the contracts with settlement money for to flow to the destination of a call (thinking most of the CLEC calls would be _to_ ILEC users), but then the CLECs went and got all the dialup ISPs to move modem banks to them. Suddenly all the calls went _to_ the CLECs, and the ILECs had to pay (some did not and went to court instead).

Re:Common Carrier

[Krojack]

But with internet bandwidth, you have have overflow connections that are sitting dark and if needed just light them up and data will be flowing within seconds. This can even be automated. As traffic starts to fall off, turn those connections off.

Re:Common Carrier

[Jason Levine]

It's more like Phone Company A has 100 channels to Phone Company B. They notice that there is a constant level of 99 calls between A and B. They also see that 25 calls are constantly being made to Joe's Pizzeria who uses Phone Company B for phone service. Phone Company A gets upset that Joe's isn't paying them (Phone Company A) for this traffic, so they refuse to add more channels. (Even though doing so would be inexpensive to do.) So calls to Joe's Pizzeria begin being dropped and Joe's customers get mad that they can't get through. Joe's finally signs an agreement with Phone Company A paying them money and suddenly the calls go through just fine.

This is extortion plain and simple. Add in the fact that the ISPs doing this have an Internet monopoly/duopoly in their areas and also tend to provide video services - that Netflix competes against - and you have extortion plus the use of a monopoly to crush competition in another market. This deserves swift and severe action to show the ISPs that this is NOT to be tolerated. Unfortunately, at best we'll get a strongly worded statement and perhaps a fine that Comcast will make back in the time it took me to write this comment.

Re:Common Carrier

[sabri]

But with internet bandwidth, you have have overflow connections that are sitting dark and if needed just light them up and data will be flowing within seconds. This can even be automated. As traffic starts to fall off, turn those connections off.

This has got to be the biggest bullshit I have ever heard.

Sabri
JNCIE #261

Re:Common Carrier

[fustakrakich]

Tuesday! Make it so! Show them who's boss.

Baby steps... but it will wake people up

Re:Common Carrier

[mysidia]

That would have zero impact. This is like the telephone company in city A have 96 channels to the telephone company in city B, but then 100 people try to make calls. Only some of them will go through, and that's a capacity issue, not regulated by Common Carrier status.

That's not the scenario. It's not capacity between cities, in this case, it is capacity between networks. The problem is they are discriminating against some networks and refusing to build capacity at the same time as they are building capacity to other networks for free; that's not a common carrier.

A telco expands capacity based on utilization, and in building more capacity to other networks in the same area: it's not a case of some networks get capacity built to them for free and some have to pay, it's..... each telco pays their own costs to build that capacity needed by their customers AND asymmetric usage is settled through the USAGE fees associated with LD termination on each call.

Re:Common Carrier

[mysidia]

Since when were fibre cables, $20000 optics, Switch ports, and 40-Gigabit port licenses free when the link is turned off?

Re:Common Carrier

[suutar]

And the cable companies get their shills in congress to tell the FCC "they are not common carriers, try again"

Re:Common Carrier

[houstonbofh]

No help as they can work around that. What is needed is to allow local last mile competition. Right now the incumbents are paying off or suing cities that try to allow it. But look at the cities that got Google fiber. Suddenly, all of the incumbent carriers got so much better...

Re:Common Carrier

[Charliemopps]

Once again, a call for net neutrality will ensue. All we really need is for the FCC to call them Common Carriers and apply the age old law.

It has already been applied to Telecoms and Utilities, just apply it to the ISP's and be done with this crap.

http://en.wikipedia.org/wiki/C...

This has absolutely nothing to do with Net Neutrality. The ongoing calls for it highlight how you and people who argue similarly do not understand how networking works.
Netflix moved all of their content to a tier1 peering service that these companies did not have large contracts with. Then demanded that they to move there. These companies run their own tier1 networks and have absolutely no reason to move to these other peers.

Netflix moved to a peer, then expected these networks to follow, they did not. The networks did not have to break net neutrality to throttle Netflix, Netflix did it to themselves and the linked story flat out states that, though trys to word it as if the ISPs had some obligation to comply. They treated the traffic "neutrally", Netflix did not have peering agreements with the correct Tier1 networks to support the amount of traffic they were attracting. Why is it Comcasts problem that Netflix peered with the wrong company?!?!

If you and this story are correct, and the ISPs have an obligation to connect to wherever their customers favorite destinations are... lets take that to it's logical extreme conclusion then. Lets say Netflix gets an awesome deal on storage space and bandwidth from Star Joint Venture Co. North Koreas premier ISP. Should comcast be required to then trunk all the way to North Korea to handle the traffic? Seriously? Because that's what you're arguing.

Since the dawn of the internet, ISPs have controlled how peering works. Netflix tried to change that, and lost. If you think peering agreements should change (I certainly do) that's fine. But Netflix's plan was even worse than what we already have.
But most importantly: This has nothing to do with Net Neutrality. So stop saying that it does!

Re:Common Carrier

[Cramer]

Not "for free". Settlement Free Peering is based on a mostly balanced flow of traffic. The instant that ratio moves from 1:1 to 100:1 (as happened when Netflix switched to their in-house CDN), "free" isn't in the room anymore.

Re:Common Carrier

[budgenator]

An easier solution would be to move speedtest to cogent, speedtest always rocks, no matter how slow the rest of the net is!

Re:Common Carrier

[budgenator]

True enough, but Joe's is big enough to actually pay the extortion, which most of the colaterally damaged companies using company B's phone line can't; so it's really an example of "When Elephant's fight, the mice get trampled".

Re:Common Carrier

[Anonymous Coward]

More likely:

Phone Company A has 100 channels to Phone Company B. Phone company B operates 1-900 telephone and other paid Long Distance services, that are billed at $1/minute billed directly to the customer. Phone Company A offers similar Long Distance services, and some 2nd rate 1-900 services, at $1.25/minutes (with some horrible customer service I might add).

These are any simple calls to a pizza shop now and then, this is a whopping 34% of Phone Company A's capacity (https://www.google.com/search?q=percentage+of+netflix+traffic&sourceid=ie7&rls=com.microsoft:en-US:IE-Address&ie=&oe=&gws_rd=ssl) is taken up by Phone Company B's services.

Phone Company B has trouble keeping up with demand for their services, and needs more interconnects between Phone Company A and Phone Company B, so they pay for more interconnects, but Phone company B drags it's feet asking for some Quid Pro Quo to enable the connections.

So Phone Company B agrees to make a payment to Yelp! so their Pizza company doesn't get as many bad reviews. No... Wait... did I mix up stories here?

Re:Common Carrier

[Anonymous Coward]

My ISP has a bit over 6x peak usage worth of bandwidth. They have 6 links, each provisioned to handle 100% of peak capacity without congestion. It is 3 sets of redundant pairs. Each set is a teamed pair that supports one of the fibers going bad without losing the connection, but while both fibers are operating, bandwidth of both fibers can be utilized. Two of these sets go to Level 3 over different paths. So one acts as a fail over, but could be quickly configured for load balancing. The 3rd set is another team going to Global Crossing, owned by Level 3.

So even if 5 of the 6 links went down, I could still play my video games lag free on my dedicated 50/50 fiber connection that costs under $100/month. Sometimes they have to light up the fail over to handle DDOS attacks. I've had to call in a few times because my latency was breaking into the 30ms range while playing games. Tech said they were under a DDOS and were having their trunk bandwidth increased to handle the load. 10 minutes later, back in action.

Anyone that thinks Comcast, Verizon, et all, have it bad, are just apologists

Re:Common Carrier

[mysidia]

Not "for free". Settlement Free Peering is based on a mostly balanced flow of traffic. The instant that ratio moves from 1:1 to 100:1 .... "free" isn't in the room anymore.

And THIS is what makes them not common carriers; ISPs can do this. In the Telco world, interconnect fees are required to be symmetric, for example: if the agreement is that charges carrier A $0.05 per call record to terminate onto carrier B's network, then it must charge carrier B $0.05 a call to terminate onto carrier A's network, it's not allowed to charge carrier A $0.05 per call and give carrier B free service into carrier A's network. An interconnect agreement cannot be terminated or repriced to favor specific networks, just based on the ratio of calls in or out.

Re:Common Carrier

[riverat1]

Once again, a call for net neutrality will ensue. All we really need is for the FCC to call them Common Carriers and apply the age old law.

It has already been applied to Telecoms and Utilities, just apply it to the ISP's and be done with this crap.

http://en.wikipedia.org/wiki/C...

Can't mod you up so ... Amen brother! Amen!

Re:Common Carrier

[Anonymous Coward]

It's worse than that...not only are the calls of Joe's Pizzeria being dropped randomly but so are the calls to the other 99 recipients, again in some 'random' order unexplainable by the caller.

Re: Common Carrier

[Anonymous Coward]

Who the bleep is your ISP?

Re:Common Carrier

[Anonymous Coward]

In a sense, it's extortion. But someone has to pay for the network upgrade to support Joe's pizza. The question is whether it should be everyone with a telephone, or Joe. To pay for the network upgrades, either all of Comcast's customers will get a rate hike, or Netflix will pay more and pass the charges on only to its customers. Which outcome is more fair?

Re:Common Carrier

[Lisias]

please mod parent up.

Re:Common Carrier

[Anonymous Coward]

One more reason to move this country to a pure democracy. The republic model worked fine in colonial times because of the logistical nightmares of getting everyone's vote. But now in the digital age all those nightmares are effectively vanquished.

Re:Common Carrier

[Anonymous Coward]

Once again, a call for net neutrality will ensue. All we really need is for the FCC to call them Common Carriers and apply the age old law.

It has already been applied to Telecoms and Utilities, just apply it to the ISP's and be done with this crap.

http://en.wikipedia.org/wiki/C...

Placing all ISP and transit providers under CC rules would not help. Having once worked for a regulated telecom in the US, I can tell you with some certainty that even in those days there were "carrier settlement" departments...and we are talking about strictly "voice calls" in the old ("ancient"?) TDM world and where fiber usage was for long distance (where it started it's existence) and carrier interconnect only. Even in those days the regulated carriers had contracts between them, not regulated by a government, that expected payment from 1 carrier ot the other on a interconnect trunk circuit when 1 carrier sent more "voice minutes of usage" to the other carrier ("unbalanced exchange"). The settlements could be a few dollars to hundreds or thousands of dollars every month. The average balanced "minutes of use" were analyzed each month to determine if more trunk circuits were necessary; a basic "service level" for the interconnect was agreed upon initially between the 2 carriers based on a minimum level mandated by a regulator. If circuit expansion was necessary due to an increase in "balanced average minutes of use", the costs were shared as reasonably as possible (normally outside plant costs, not inside plant costs). This description is not much different than Internet peering agreements today, and did you ever wonder where Internet peering agreements came from? Guess what? Regulated telecom carrier interconnect trunk agreements.

Now "fast forward" to the data world.

In the US a never ending thirst for online video took placed with the advent of Netflix. As netflix filled it's libraries, we watched more and more. It is a like a drug and Netflix is a pusher whose sole interest is keeping it's users "high". the more we watch, the more traffic we generate on the Internet, and that traffic is "unbalanced"; we make a small request of a movie or TV episode and the result is a multi-megabit video stream. I think we all agree on that description.

Now place all ISPs and transit providers in a CC model. A regulator might mandate a minimum service level on interconnect points between networks. If that service level might be established on "average balanced levels", or "95th percentile balanced levels", I suspect it would be accepted without much complaint. Placing a demand to establish it on "peak levels" would generate "push back" because nobody has a crystal ball that works. Oh, the "balanced" aspect is critical, since such a mandate could be viewed as discriminatory by a court of law in the US if it were "unbalanced"; just because companies are regulated doesn't mean the companies can't complaint to a court in the US.

So what would a CC environment mean to a transit provider for Netflix? Pay up sucka! Yes, that's right. The flows between Netflix and it's providers are not "balanced", so Netflix or it's transit providers have to pay the "eyeball ISPs" for the imbalance in traffic exchange created by Netflix users. So long as the service levels at the interconnect points are maintained above the level set by the regulator, the regulator has little else to do.

The cable ISP model of today is "unbalanced"; about a 10:1 ratio in favor of download, but maybe more or less. Could that be changed without changing out every cable modem in the US? Perhaps, but I hear cable modem hardware vendors salivating. What if Netflix implemented their "upstream" model to subvert the "imbalance" in traffic? We could see cable ISPs implement upstream bandwidth usage caps with overage charges because blocking it would get them in trouble with regulators; it's all about making a buck. Could a regulator force "no caps" of any type on su

Re:Common Carrier

[dl_sledding]

One more reason to move this country to a pure democracy. The republic model worked fine in colonial times because of the logistical nightmares of getting everyone's vote. But now in the digital age all those nightmares are effectively vanquished.

Vanquished? Really!?

http://www.csmonitor.com/USA/E...

Re:Common Carrier

[Anonymous Coward]

Probably since it only cost a few hundred dollars for the fiber.

Their answer to oversubscription as well

[Anonymous Coward]

Well i get about 10% more than I pay for. Consistently between 102-110% o f the speed i pay. But then I'm not in US.

Netflix via sneakernet

[Anonymous Coward]

I prefer to get Netflix via sneakernet anyway.

Re:Netflix via sneakernet

[Anonymous Coward]

Receive, rip, and return!

Re:Netflix via sneakernet

[Anonymous Coward]

That's very Y2K. We've moved way beyond that.

Re: Netflix via sneakernet

[Anonymous Coward]

Not really given the stability of my Time Warner connection lately. To their credit it is really xo communications causing the problem, but TWC won't hear it or review the traceroutes they ask for.

WHy net neutrality doesn't work

[lgw]

Stuff like this is why I think Net Neutrality discussions miss the mark - you're not going to fix the problem that way, you're only going to cause the cable companies to achieve the same throttling through other technical means. Trying to close technical loopholes via the lawmaking process requires a body of law the size of the tax code.

The fundamental problem is that companies with a legally-granted monopoly for delivering high-speed internet are also allowed to sell content. In a free market, that wouldn't bother me - competition would sort it all out. But "last mile" is about as far from a free market as you can get in most of the country these days, and so we get this as a result.

Last mile needs to become a public utility. Let vendors compete for my business, and I'll pick "just a pipe" or a content company or whatever mix fits my needs.

Re:WHy net neutrality doesn't work

[MozeeToby]

Stuff like this is why I think Net Neutrality discussions miss the mark - you're not going to fix the problem that way, you're only going to cause the cable companies to achieve the same throttling through other technical means.

You can make crap like this illegal, in fact it arguably already is without net neutrality legislation.

Re:WHy net neutrality doesn't work

[Anonymous Coward]

They could just require that cable companies must allow companies to setup a relay point to serve the information within their own network at said companies own expense if they consume past a certain percentage of their access points bandwidth.

Then they have the choice of either fixing that point or being required to allow Netflix to pay to setup a point within their network to bypass that point entirely. The only thing they could do to try and get around it then would be to try and route their network through multiple congested links which would be obvious to prove in court.

Would fix a lot of this.

Re:WHy net neutrality doesn't work

[Anonymous Coward]

"Last mile needs to become a public utility. Let vendors compete for my business, and I'll pick "just a pipe" or a content company or whatever mix fits my needs."

I think you have that backward.

The backbone should be a public utility. The gov't should assume the responsibility for the massive infrastructure required to build the backbone. Then its easy for any small mom/pop ISP to connect to the public backbone for a nominal fee and provide last mile service to their area when they are unhappy with the current last mile providers as it requires orders of magnitude smaller investments in infrastructure.

Slashdot required car analogy, The Fed Gov't built the interstate system, the local gov't built/maintains the local roads and all the mom/pop stores are welcome to build brick n mortar business and connect to those roads for nominal fees. Meanwhile private parties are also welcome to build special roads/infrastructure at there own cost to serve special needs, and the public is sometimes welcome to use those items at additional cost EG: Toll roads/bridges/tunnels/ferries

Re:WHy net neutrality doesn't work

[alen]

there is no legal monopoly, the cost is too high.

FIOS only has on average 40% of potential customers in the markets they serve. it's impossible to borrow enough money to build out a last mile network and make a profit after you account for advertising and operational costs

Re:WHy net neutrality doesn't work

[sabri]

The gov't should assume the responsibility for the massive infrastructure required to build the backbone.

Ah yes, with fibertaps everywhere.

No thanks.

Re:WHy net neutrality doesn't work

[nine-times]

The fundamental problem is that companies with a legally-granted monopoly for delivering high-speed internet are also allowed to sell content.

I agree with this part of your post, at least, and have been making the same argument for years. If the companies providing the infrastructure were not making money from selling content, and were only serving as "dumb pipes", then their business incentive would be in pushing customers toward higher-bandwidth (and therefore more expensive) connections. In that business model, companies that can provide content to saturate slow connections become very important, and so it seems likely that they would be falling all over themselves to provide a better connection to Netflix.

Instead, the Infrastructure providers have no incentive to increase content availability, because any piece of available content becomes competition for the content that they are trying to sell. That's a bad system. Unless you have an effective regulatory system, the ISPs will find ways to push towards a walled garden AOL-style internet, charging for access outside of the walled garden.

However, I don't think this is an example of "net neutrality" missing the mark. Net neutrality is a concept, and divorcing infrastructure providers from content providers is one way in which net neutrality could be promoted.

Re:WHy net neutrality doesn't work

[nehumanuscrede]

So the fix sounds easy, but gets complicated quickly. :|

Break them up. Don't think about it, threaten or consider it. To borrow a phrase from Nike: JUST DO IT

Force the companies who provide the bandwidth to split from the part of the company who sells content so there isn't any grey area issues. In order for the bandwidth side of the house to remain competitive, they'll need to upgrade their network to ensure they're delivering what their customers want. This also prevents the content folks from tweaking the network to ensure their own services get priority over competing ones. Taken to an extreme, and without some sort of oversight, the big players can absolutely destroy the competition in this manner.

They've already proven they can't be trusted to do the right thing themselves. Time to step in with a big stick and start swinging.

Of course, true competition needs to be in place to help fuel that fire otherwise we'll still end up with crappy throughput with no realistic choices of switching to another provider. We really need to end this whole regional mono / duopoly thing.

On top of this, we'll need to purge the Congressional votes-for-contributions types to make sure the legislation has a fair chance of happening.

Re:WHy net neutrality doesn't work

[Gr8Apes]

I think you have that backward.

The backbone should be a public utility. The gov't should assume the responsibility for the massive infrastructure required to build the backbone.

The last mile is the most expensive part of the entire network. Backbones have few pricier components. Every connection requires a port on a switch, etc.

Re:WHy net neutrality doesn't work

[fustakrakich]

Last mile needs to become a public utility.

"Reform this! Reform that!"

Everybody keeps saying that like it's going to magically happen by some miracle.You think the politicians currently in office are going to let that happen? Especially when they have 95% chance of being reelected no matter what they do? When voting, know who/what your politician is representing, and then decide if you want the same thing. Their records are easy to find now, and you can stop believing the lies they tell you on the TV. When I see a big turnover (get rid of the entire institutional republican/democrat party), then I'll believe people are serious. But everybody please, quit crying about how the system is rigged. It is a social/psychological exploit, that needs to be patched on the individual level, like any other system update. This patch however, requires a firmware upgrade.

Re:WHy net neutrality doesn't work

[lgw]

Well, this is rigged at the local level, so it's possible to fix it at the state or federal level. Local government is often more responsive, but when it's corrupt it generally can't be fixed at that level. I love to see a state step up and do this - force "last mile" into a utility wherever it locally has legally-granted monopoly status. State governments can also be fairly responsive, at least when issues don't touch car dealerships.

Re:WHy net neutrality doesn't work

[fustakrakich]

It doesn't matter what level you're at. The politician represents his financiers, or he receives no financing. Our obligation is to seek out those that aren't on the organ grinder's leash between elections so we have somebody to put on the ballot. That's how they do it, and it shows.

Re:WHy net neutrality doesn't work

[LessThanObvious]

I'd be basically OK with it if they were allowed to charge the peer for the (actual and reasonable) cost of the upgrade and no additional fees were charged, even though they would be willing to meet half way were it not in competitive relationship. We'd almost need to force them to split their businesses into separate walled off cable TV and Internet businesses to end the large incentive to squelch competition for eyeballs on the screen. If they offer better peering deals to some transit providers and worse to another solely in order to screw their competition I'd have to believe there is some Antitrust legal provision being violated. Ever since cable speeds beat out common DSL speeds we have been screwed by lack of competition.

Re:WHy net neutrality doesn't work

[AK Marc]

Nope. The anti-trust cases have ended up (successfully) arguing that a monopoly isn't a monopoly, if you have choice. You can choose to move states and get different choices, so you aren't "locked in".

Re:WHy net neutrality doesn't work

[tepples]

You can choose to move states and get different choices

Which court in which case used this as an argument in its opinion to dismiss an antitrust action?

Re:WHy net neutrality doesn't work

[Anonymous Coward]

Really? Tell that to Google...I'm sure you're a better accountant & business mind then they have so I'm sure they'd be happy to pay for your insight on this topic.

Re:WHy net neutrality doesn't work

[TangoMargarine]

You can choose to move states and get different choices, so you aren't "locked in".

By that logic, we could have a single company providing a service to the entire world minus one podunk town in the middle of Siberia (directly financed by Putin probably) and it "wouldn't be a monopoly."

Fuck all you guys with your contortions to redefine the argument to "prove" yourself correct.

Re: WHy net neutrality doesn't work

[Anonymous Coward]

This!

This is how power is sold in the UK and it was awesome. The providers bought the power that was delivered over the municipal lines to cover their customer's usage. This bypassed the natural monopoly while allowing competition to keep prices reasonable and abuses low.

Re: WHy net neutrality doesn't work

[lgw]

I believe that's how power works in Texas as well. It's a good pattern. Sadly, we have a problem in the US with ideas that make too much sense.

Re:WHy net neutrality doesn't work

[Anonymous Coward]

You have that backwards. The backbone is cheap and does not have performance issues. It's the last mile where there is no competition and it's hard to break in because of capital cost and red tape.

Off with their heads!

[AttilaB]

Time to get out the pitchforks ...

Your'e still suspecting?

[Anonymous Coward]

and we've all suspected it happens

I don't have to "suspect" anything. If I uncheck the encryption checkbox of my torrent client, transfer speeds drop to 0 instantaneously. Re-check it and the speeds go back to normal instantaneously.

Re:Your'e still suspecting?

[Anonymous Coward]

BS. Encryption is negotiated at connect. Flipping that switch will have no effect on existing connections -- aside from possibly forcing them all to close, which would have a dramatic effect on speed. :-) I suspect, without encryption most of the clients in your swarms are refusing to talk to you.

How many engineers does it take to screw netflix?

[deathcloset]

OK, so people had to implement this blocking, right?

And it had to have been more than one person, right?

How many individuals would have to have been active and knowingly involved in order to implement such blocking?

From that number, how long until someone straight-up comes out and says they did it and exactly how they did it? You know, instead of having to rely on this external third-party reverse engineering.

Yes it is a peering problem ...

[jschultz410]

and not a net neutrality issue thankfully.

Settlement free peering between tier 1 carriers only happens when the flow of traffic is roughly balanced between the contracting peers.

When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network. Otherwise, networks would be monetarily incentivized to unload traffic they should carry on their own networks onto their peers' instead.

Re:Yes it is a peering problem ...

[0123456]

I'm sorry, but your post is too sensible and informative for today's Slashdot, where technological knowledge takes a back seat to 'stick it to the man!' outrage.

Re:Yes it is a peering problem ...

[Burdell]

Traffic balance is not the primary measure these days (from what I understand), it is just an economic decision. However, the Netflix case is interesting, because they were essentially used as a leverage tool by Cogent against the other carriers. Cogent has a long history of trying to get settlement-free peering, not meeting contract terms (whatever they are), getting dropped, and then blaming the other side. They have long wanted to be a settlement-free "tier 1" provider (which is a nebulous term, but go with it), but have generally not been. They sell bandwidth often at below-market rates in order to attract customers to leverage against the other "tier 1" providers. They saw Netflix on the rise and grabbed them, apparently selling bandwidth much cheaper than any other backbone (possibly at a loss even) in order to leverage settlement-free peering contracts out of other providers.

Any network engineering with a clue knows that you never buy bandwidth only from Cogent (or even Cogent and one other provider), because you _will_ get disconnected from somebody when Cogent gets in another peering dispute.

Re:Yes it is a peering problem ...

[PPH]

When one peer is pushing a lot more traffic onto the other network,

Simple fix: A Netflix client that echos the content back to the source server.

Problem fixed.

Re:Yes it is a peering problem ...

[Anonymous Coward]

When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network.

They're not pushing the traffic - the other network is pulling it. Netflix's traffic is not unsolicited - every goddamned packet is being sent in response to a specific request from the other network's customers, and it's not fucking transit - the Netflix packets will terminate within the receiving network. Are you seriously arguing that Comcast should be paid by Netflix because they're carrying gigabytes of Netflix traffic their own fucking customers requested?

Re:Yes it is a peering problem ...

[waldozer]

I think you have it backwards. Netflix does not "push data" to Comcast. Comcast customers "pull data from Netflix".

Re:Yes it is a peering problem ...

[Anonymous Coward]

I'm sorry, but your post does not repeat the talking points of any half-baked ideology, and is therefore unsuitable for today's Slashdot, where technological correctness takes a back seat to ill-informed political posturing.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Sensible?? More like nonsensical and completely misses the point of the article. As a great many people have pointed out, this is not unsolicited traffic, but traffic requested by Comcast's customer. So where on earth do you get the idea that

"When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network."

has ANY relevance to this story? Given what has been reported as the "trivial" costs of upgrading the interconnection points, by what other argument do you think the ISP's are in the right? Given that they are being paid by their customer's for internet access?

Re:Yes it is a peering problem ...

[AnotherBlackHat]

Settlement free peering between tier 1 carriers only happens when the flow of traffic is roughly balanced between the contracting peers.

When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network.

So you're saying if Netflix downloaded more data from Comcast than they sent, that Comcast should pay them?

I have no problem uploading an amount equal to what I download from Netflix, or even more, if you really think that will solve the problem. I don't really control the software on my Roku box, but I don't mind if Netflix puts some P2P software on it for help carrying their own traffic.

Re:Yes it is a peering problem ...

[RelaxedTension]

Got it, but where does the fact that the traffic has been requested by the users the target network play into it? The more appropriate term here is "puller" as opposed to "pusher". The traffic would not be there except for the end network requesting it in the first place.

Re:Yes it is a peering problem ...

[Anonymous Coward]

And that makes perfect sense in between tier 1 carriers. As soon as you bring in the monopoly last mile consumer grade carriers it makes no sense. Especially since these same monopoly carriers tend to offer asymmetric service (upload significantly slower than download) and tend to prohibit running "servers" (supplying really any services) on consumer networks. At that point what you have is an ISP that is "receiving" a lot of traffic. It happens to be the traffic that the customers asked for. The customers already paid for that traffic. They signed up for a certain bitrate, and many of them also have a cap. So the traffic is paid. Now that monopoly ISP carrier wants to be paid AGAIN for that traffic. Screw that! It is a peering problem. They need to upgrade their connection equipment to the peer. Network maintenance and upgrade costs are built into their billing model. If they aren't, fire the CEO, and redo the pricing tiers. But there is absolutely no reason for a tier 1 peer to need to pay these clowns or for a customer of the tier 1 peer to be paying these clowns.

Re:Yes it is a peering problem ...

[Obfuscant]

Netflix does not "push data" to Comcast. Comcast customers "pull data from Netflix".

It doesn't matter. There is an imbalance of the data flow through the peering connection.

Comcast doesn't guarantee data rates to off-network content providers. It can't, or you'd be paying even more outrageous rates for service from them.

Here's why. Imagine Comcast has 2 million subs. I don't know the numbers, but let's use that for argument. They each have 50Mbps service. That means that EVERY PEER CONNECTION that Comcast has would have to be at least 100,000,000 Mbps to guarantee full rate to every sub. That's 100 TERAbps. Why? Because all 2 million might want to connect to the same data provider at the same time. Very slim chance, but a guarantee is a guarantee, and we see how loudly people complain when implied promises aren't kept.

Service providers have ALWAYS used statistical methods to determine the most cost effective amount of hardware to meet the anticipated demand for service. You don't think a phone company with 100 users had 100 long distance trunks running from the central office to the nearest long distance center, do you? (Mandatory auto analogy: a city with 1000 residents with one car each doesn't have 1000-lane roads everywhere, does it? And a restaurant with 100 seats doesn't have 100 valets parking cars, does it?)

So now ISPs are doing the same thing. And every one of them does it. Where I work, I have gigabit to the desktop. There's about 400 of us in this college alone. Do you think we have a 400Gb line to the Uni? And does the Uni multiply 1 gigabit times the number of employees/students to figure out how much bandwidth they have to buy from their provider? Of course not. We have a /16 address block, so ignoring the large number of unroutable addresses we already use, that's 64,000Gbps, if the Uni provided a full-time full-rate connection at 1Gbps to every address.

This article is FUD and nonsense. Comcast isn't jamming Netflix. In fact, this article pretty much proves that Comcast is doing nothing special to Netflix, they're just not upgrading their peer connection as fasts as people want them to. Why? Because it costs money to do that.

Should they upgrade? They can't make everyone happy. They didn't promise you 24/7 full-rate access to anyplace off their net, and I don't think they did that even ON their net. It's not illegal not to upgrade. It's not even a net neutrality issue, because, as this article proved, the congestion impacts EVERY service the same.

Re:Yes it is a peering problem ...

[Em Adespoton]

and not a net neutrality issue thankfully.
Settlement free peering between tier 1 carriers only happens when the flow of traffic is roughly balanced between the contracting peers.
When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network. Otherwise, networks would be monetarily incentivized to unload traffic they should carry on their own networks onto their peers' instead.

What you're stating is the situation for transit peering. This issue has nothing to do with transit peering, as the packets all terminate inside the receiving network.

See, this is why it's a problem that we've got the same people providing Tier 1 trunk lines that are providing endpoint connects. It creates another class of peer where they're being paid by end users for transit to the interconnect, and then they turn around and want to charge peers for transit to the end users. That's called double dipping, and that's why there's an issue here. The entire point of peering with these Tier 1-to-endpoint providers is to unload traffic; there's no other way to actually route traffic to the destination!

Re:Yes it is a peering problem ...

[Anonymous Coward]

Are you seriously arguing that Comcast should be paid by Netflix because they're carrying gigabytes of Netflix traffic their own fucking customers requested?

Uh, I think the answer is "Yes, very serious."

Your way would turn the Internet into an overnight DoS as every crook dumped fake traffic on every other guy so he could get paid.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Who cares that its "requested traffic"? Sorry there is not special "recycled bits" discount or some other stupidity like that. There are packets and sender pays for the packet. Its the little weasels like that that make it impossible to run a business any more. Any excuse... If there is a contract that says they have to add connections for free then they should. If not then someone should get a contract that says they will add connections (and probably pay for it). Don't like it get a different ISP.

Re:Yes it is a peering problem ...

[Anonymous Coward]

It doesn't play into it. This is Internet Protocol. Packets are forwarded on to the next link the router decides it should be to. SENT. There is no concept of a packet have a life or existence outside of it's own packet header.

Re:Yes it is a peering problem ...

[suutar]

Except that the recipient (me) is already paying Comcast to deliver the bits. If they want to go to sender pays, then I get to start charging them for traffic that they send to my house.

Re:Yes it is a peering problem ...

[torkus]

Erm...but netflix will drop a cache server in the ISP's datacenter, configure it, and maintain it. Oh, and it's free. The ISP saves on bandwidth/interconnect at the cost of a few U of rackspace and a couple bucks in power/cooling.

But then why would they comcast or TW want to do something to help their competition even if it also benefits their customers. Monopoly and conflict of interest. Good job politicians.

Re:Yes it is a peering problem ...

[Anonymous Coward]

So, as a customer of an ISP that has unbalanced links, what exactly does the monthly fee that I pay for Internet access actually cover?

If I pay $80/month for Internet service and 90% of my use is Netflix traffic, then I'm already paying my ISP to handle the inbalance. It is wrong for my ISP to collect money from me for that access and the also collect money from Netflix for that access - that is double dipping.

I have FIOS service at my house, I do not pay for access to the FIOS network alone - I pay for Internet access - to access all of the Internet. If the bulk of traffic on the FIOS network is from Netflix at a certain time of day then FIOS should be using the bulk of customer payments to ensure I'm getting the service that I pay for and give me good access to Netflix. Instead, FIOS takes my money and does nothing to provide service to Netflix - they have failed me as an ISP. FIOS claims that I have full speed access within the FIOS network - but that is not what I pay for, I pay for service beyond the FIOS network and I'm not getting what I pay for.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Wholly crap batman. You know a way to pull a bit unwillingly from the other side of the internet? You should sell that to some government and get payed...

*pft* Pulling traffic. What other physically impossible moronic things are they going to come up with next. Sorry but you are responsible for your own shit. If you don't want to pay for the traffic you sent out how about you not send the traffic. Or maybe you should get a contract that says you don't have to. I bet for enough money they would promise you infinite bandwidth (probably just cheaper to pay your fair share however).

Re:Yes it is a peering problem ...

[Anonymous Coward]

you can parrot this nonsense all you want, but the fact is that every other video provider pays a CDN to get good performance, except netflix who whines about everyone having to pay their bills for them.

Netflix goes even further and will put their own caching servers inside your network at their expense. Guess which ISP decided they'd rather hold their customers hostage for a big fucking check?

Re:Yes it is a peering problem ...

[Obfuscant]

So you're saying if Netflix downloaded more data from Comcast than they sent, that Comcast should pay them?

If Comcast was the source of the data that was creating congestion on a Netflix-to-peer link, I'd say yes, I think Comcast should have a part in paying for the increased bandwidth required to carry their data.

I have no problem uploading an amount equal to what I download from Netflix, or even more, if you really think that will solve the problem.

Who said that would solve the problem? All it would do is make Netflix less responsible for paying for the upgrade, not that the upgrade would happen.

There are two problems here. The first is a limited bandwidth connection is being congested (and all connections are limited in some way.) The second problem is figuring out who should pay for the upgrade -- the company who is profiting from sending the data, or the company whose customers are assuming that a maximum data rate through their "last mile" means they should get the same rate to any site anywhere on the planet?

Re:Yes it is a peering problem ...

[mythosaz]

Yes, and Netflix pays their direct service providers for their bandwidth.

When bandwidth leaves their direct provides and reaches, say, Comcast, where Comcast's customers have paid for internet service as well, Comcast (and others) refuse to deliver that data, because they're only getting paid to do so at least twice.

Re:Yes it is a peering problem ...

[Anonymous Coward]

> ...400 of us...400Gb line to the Uni?

But that is a ridiculous example since most people have much less than a thousandth of that speed of a connection. From polling my employees here in Seattle, our average connection speed is 110 kbps. Mine at home is 64 kbps if I use one ISDN channel or 128 kbps if I use both channels which I don't usually do because the per minute charge doubles. I'm the president of my HOA and out of the ninety-seven houses in our neighborhood, I'm the only one with ISDN so I'm the only person with faster than 56k dial-up. With your 1 Gbps example, you can support almost 18,000 of the average connections in my neighborhood. Obviously your example is way out of line with reality.

Re:Yes it is a peering problem ...

[Anonymous Coward]

It is already sender pays. If you want to charge anybody to deliver packets to your house, you are free to do so. Except, nobody cares about you or your house, so you'll have no takers. But, if you happen to have an Amazon Glacier node in your house, then there is a very good chance you will get paid to receive all that data.

In so far as you are paying Comcast to deliver bits, you're paying them to deliver the bits that you send. Somebody else is paying for the bits you receive.

Re:Yes it is a peering problem ...

[MozeeToby]

I agree, why should Cogent need to supply all this throughput that Comcast's paying customers are asking for!? Waaaaaiit a minute....

Re:Yes it is a peering problem ...

[Anonymous Coward]

If Comcast simply doesn't have the backbone capacity to handle the traffic they get at the peers (almost certainly true, isn't every network like that?), then there isn't much else to say. However if they are throttling JUST Netflix, then yes, they are tariffing. From Comcasts perspective it is network management, from Netflixes perspective it is tariff. BOTH are true.

The solution is simple. Comcast needs to stop lying to their customers about capacity they don't have and throttle in a media agnostic way at the client end, like responsible network managers. Chances are this is what their NOC wants to do, but the execs refuse because they STILL don't understand the Internet. What is most unfortunate about this, is that it will invariably bring down the tower of ignorance (better known as the the United States Congress) on all of us, and so everybody will end up getting screwed because Comcasts CEO refuses to wear his big boy pants.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Netflix does not "push data" to Comcast. Comcast customers "pull data from Netflix".

It doesn't matter. There is an imbalance of the data flow through the peering connection.

...

Should they upgrade? They can't make everyone happy. They didn't promise you 24/7 full-rate access to anyplace off their net, and I don't think they did that even ON their net. It's not illegal not to upgrade. It's not even a net neutrality issue, because, as this article proved, the congestion impacts EVERY service the same.

Why does imbalance matter? It only matters because Netflix creates increased traffic on Comcast's network and presumably there's a cost associated with that (even if Netflix provides caching boxes, there's still the traffic - it's just not originating from the other side of the interconnect).

But this view ignores the most important point, that Comcast has explicitly promised its customers "internet access" at an advertised speed. There are no pharmaceutical-like disclaimers during those commercials that say "Speeds not achievable during peak hours. Speeds not achievable to Netflix/Hulu/YouTube and other video sites; or during peer-to-peer file file transfers. Speeds only available to certain business partners of Comcast" etc. etc. If Comcast says you're paying for "10Mbps internet", the assumption is that you get the advertised speed to the entire internet, provided there are no technical limitations outside of Comcast's control. It certainly implies that Comcast won't artificially limit speeds by refusing to upgrade relatively inexpensive network hardware.

TL;DR, if Comcast advertises a given internet speed, and the only reason it's not attainable is their refusal to upgrade equipment so they can extort money from a content provider, than that's false advertising and unfair business practice. Packets don't matter.

(AC to preserve mods)

Re:Yes it is a peering problem ...

[Anonymous Coward]

This is where the argument lies.

Netflix isn't 'pushing' data to Comcast. Comcast (or it's customers) are 'pulling' data FROM Netflix. The connection is initiated on Comcast's network, not Netflix's network.

In the peering world, Comcast should be paying the peers for requesting more data than is requested from them, not the other way around. Comcast is trying to double dip, and to change the way peering is done.

Re:Yes it is a peering problem ...

[Anonymous Coward]

You know a way to pull a bit unwillingly from the other side of the internet? You should sell that to some government and get payed.

Netflix is sending traffic in response to an application layer request from the network whose customers are requesting said traffic, colloquially known as "pulling" traffic. Is that better? You knew perfectly well what I was talking about, but chose to be a pedantic asshole instead of actually debating the point.

And it's "paid", not "payed".

Re:Yes it is a peering problem ...

[Anonymous Coward]

ybefore that the baby bells used to charge termination charges for calls ending on their networks

...and this is the basis of peering. I make a call from network A to network B, network B charges network A for the call.

Now, think about what you've just said and apply it to this situation. I make a 'call' from Comcast to Netflix...

You see where I'm going with this? The 'call' is initiated from the Comcast network, and the receiving network should 'bill' for the call. That's how peering works. The initiating network pays, not the receiving network.

Re:Yes it is a peering problem ...

[Anonymous Coward]

What you say is nice and logical, as long as you leave out the part where Netflix's alternatives (such as dropping a content box inside Comcast's network to solve the problem) were ignored by Comcast.

Comcast is fucking with Netflix in favor of it's own video streaming service. Whether or not they are doing so 'legally' does not change that fact.

Re:Yes it is a peering problem ...

[Anonymous Coward]

There is an imbalance of the data flow through the peering connection.

Which exists solely due to the actions of Comcast subscribers.

Remember back when everyone roundly castigated Ed Whitacre for saying "they can't use my pipes for free" in regards to trying to get Google et al to pay for serving data in response to SBC customers? This is exactly the same thing, except nowadays more people seem to agree with Whitacre's philosophy. What changed in the intervening years to make people think it's acceptable for the ISP to get paid from both ends?

Re:Yes it is a peering problem ...

[NormalVisual]

You can't just ignore the entire OSI model like that, and the fact there's an application level request being sent from Comcast to Netflix for the traffic received DOES most definitely factor into it. That a layer 7 request/response is being implemented by means of layer 3 activity doesn't change the fact that the layer 7 request from Comcast was the ONLY reason that the subsequent Netflix traffic was generated and received.

The fact is that traditional peering simply doesn't work when dealing with hugely asymmetrical networks like the average ISP. Comcast isn't transiting the Netflix traffic to another network, they're routing it directly to a Comcast end customer. IMO this wouldn't be an issue if Netflix weren't competing with Comcast's own content offerings.

Re:Yes it is a peering problem ...

[Xipher]

Eyeball networks didn't usually get settlement free peering to begin with. Until you had these huge eyeballs form like Comcast that kind of peering was between the transit ISPs themselves. Comcast used to be a customer of these ISPs, and didn't get the peering agreements until they started congesting links by dropping transit services. I found information on this discussed on the NANOG mailing list from 2010, so this has been going on for a number of years already.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Of course. ISPs offer entirely symmetric connections to customers so any imbalances in traffic flow is in no way engineered into the system.

Re:Yes it is a peering problem ...

[Obfuscant]

Why does imbalance matter?

Because when peering agreements were created, the assumption was that trying to keep track of how much data you wanted us to carry and you keeping track of how much we gave you to carry was not necessary because we'd be charging each other the same amount if we did keep track.

Large amounts of data going one way breaks that basic tenet of peering. Now it makes sense to charge the other guy for data they want you to handle.

But this view ignores the most important point, that Comcast has explicitly promised its customers "internet access" at an advertised speed.

No. From here:

Performance Starter: Offer ends 01/04/15. Restrictions apply. Not available in all areas. Limited to new residential customers. Requires subscription to Performance Starter Internet service. Equipment, installation, taxes and fees, including regulatory recovery fees, Broadcast TV Fee (up to $3.50/mo.), Regional Sports Fee (up to $1/mo.) and other applicable charges extra, and subject to change during and after the promotion. After applicable promotional period, regular rates apply. Comcastâ(TM)s current monthly service charge for Performance Starter Internet is $49.95 (pricing subject to change). Service limited to a single outlet. May not be combined with other offers. Actual speeds vary and are not guaranteed.

Emphasis mine. The same emphasized text appears in the details for all three residential service levels. My statement stands: they did not promise you 24/7 full-rate access to anyplace off their net.

There are no pharmaceutical-like disclaimers during those commercials

There are when you actually go to sign up. And common sense tells you that they cannot guarantee those speeds to every site on the planet. They can't even guarantee those speeds to every site on the Comcast network. That's why they don't.

If Comcast says you're paying for "10Mbps internet", the assumption is that you get the advertised speed to the entire internet, provided there are no technical limitations outside of Comcast's control.

That's what some people assume, but that's not backed up by the service agreement.

It's not even backed up by common sense. Suppose you buy the Blast service and get 105Mbps download. You want to connect to my system and I've got Performance Starter (6 Mbps down, God knows what it is up). You ain't getting anywhere near 105Mbps from my stream. Even trying to connect to your next door neighbor who has the same service, you ain't getting faster than his upload allows. If you think Comcast could promise anything faster, then you must think they'll upgrade MY service to Blast for free because they promised YOU that you'd get data from me that fast, and you're paying them for my data at that speed.

Re:Yes it is a peering problem ...

[Obfuscant]

What you say is nice and logical, as long as you leave out the part where Netflix's alternatives (such as dropping a content box inside Comcast's network to solve the problem) were ignored by Comcast.

Well, I see this claim here a lot, but I also see things from other people that say that Comcast didn't ignore the offer, it was that Netflix wanted hosting without paying for it like the other CDNs do. And I'm not sure why Netflix should get hosted on Comcast's network without paying for it... after all, Netflix is making a profit selling the data and they'd have to pay any other network provider for the connection.

Re:Yes it is a peering problem ...

[Anonymous Coward]

but comcast is not a peer in that sense, they are an end user internet provider -- they're supposed to be paying for their links to 'the internet' in sufficient units to cover their customers' usage.

Re: Yes it is a peering problem ...

[Anonymous Coward]

Hello, 90's

Re:Yes it is a peering problem ...

[AK Marc]

When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network.

When did this change? In the Early Days, the puller paid, not the pusher. Hosting servers in datacenters was almost free, as upload bandwidth was "free" to the ISP. I pay my ISP to get the Internet. If their connection to the content I want isn't good enough, my ISP, not the other side, should pay to fix it. Puller pays. Otherwise, my ISP should be paying me to be their customer for generating demand that they make money from.

Re:Yes it is a peering problem ...

[AK Marc]

So there's no session information in a packet? TCP is stateless? I'm going to have to find some brain bleach to fix all my bad info. Thanks A/C. I thought TCP was stateful and that packets can contain information that pertains to port sequence and other things that demonstrate a life beyond that single packet.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Not only do they note guarantee your speed, they will actively make sure you don't get it. An ISP should be able to guarantee no congestion within their own network and over links to peering networks. If they can't, then they should not be selling the rates they do. The ISP cannot guarantee speeds through other networks, but they have full control of everything that touches their network.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Your claims go completely out the window as soon as Comcast accepts a caching server from Netflix. This "peering dispute" is completely on Comcast.

Re:Yes it is a peering problem ...

[Anonymous Coward]

it was that Netflix wanted hosting without paying for it

Last mile ISPs pay for bandwidth. They should just turn this into a law. A last mile ISP should never be able to charge peers for bandwidth. Companies should not be able to mingle infrastructure and accounting between last mile and non last mile ISPs. They should have to be separate business entities. This would make the business decision of hosting CDNs a lot simpler if Comcast Residential had to choose between purchasing transit for $1/mbit or hosting a 20gb/s Netflix box for the cost of a 125watt 2u server spot in their datacenter.

Re:Yes it is a peering problem ...

[tepples]

My statement stands: they did not promise you 24/7 full-rate access to anyplace off their net.

So why don't ISPs accept Netflix's offer to provide a caching server without charge that keeps the most popular traffic on their net?

Re:Yes it is a peering problem ...

[Solandri]

It is not a peering problem. Netflix provides streaming servers to ISPs for free so they can stream Netflix videos locally, and completely avoid the bandwidth bottleneck of going through their peering networks. Comcast, Verizon, et al refused to accept these free servers, to artificially degrade Netflix's quality specifically so they could extort money from Netflix.

It is a monopoly problem, plain and simple. If Comcast, Verizon, et al had had competition, this tactic would've caused them to hemorrhage customers who would've fled to an ISP which actually provided decent service. But because they have a monopoly, their customers have nowhere to flee, and this extortion tactic works

Re:Yes it is a peering problem ...

[Anonymous Coward]

A shill for Comcast & their 'ilk' I see. The fact is that the parent is 'sensible' AND we have direct offers by the "pushers" here to pay the CAPITAL costs of the 'unbalanced peering arrangement'...what they are NOT willing to do and what Netflix should NEVER have had to do is pay excessive fees that have nothing to do with upgrading the hardware involved in the peering arrangement...What the major last mile monopolies are doing is NOT just trying to get fairness in 'who pays for the upgrade' but rather they are extorting money over and above the costs of the upgrade...but I'm sure you know that, you're just a shill...I trust your traitors gold will be well spent.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Really? So Cogent offering to pay for the capital costs of the upgrade to the connection points is because Cogent wants to have 'settlement free' connections? Strange way for them to negotiate NOT paying for the connections by offering to pay for them!

Re:Yes it is a peering problem ...

[Anonymous Coward]

Really? And you think your pile of excrement is remotely valid? Right, so Comcast can't 'guarantee' that every path through to other providers will have 'max simultaneous' bandwidth but when their customers CLEARLY demonstrate they have a desire for content on another network that is saturating the link they have an OBLIGATION by your own analysis to upgrade it. If that restaurant with 100 seats did not have enough Valet's to park cars to keep their restaurant filled to capacity at peak times they WOULD hire more Valet's...they may not need 100 Valets but they certainly WOULD hire enough to stay at maximum capacity...so even by your own argument Comcast et.al. have an obligation to upgrade the peering hardware to be able to service that particular link to the other network to the maximum needed to avoid congestion at peak times...ANY good network engineer WOULD do this...trust me...Comcast certainly would upgrade their network gear to their XFinity service if they saw congestion on the path from their customers to that service...& no that wouldn't require 100,000,000 Mbps unless of course all their customers were doing nothing but downloading sufficient content from XFinity that they would need that capacity but then I'm pretty damn sure that Comcast would LOVE to have that problem...money, money & more money is all they'd be seeing.

Re:Yes it is a peering problem ...

[perryizgr8]

Is this serious? kbps? per-minute charges? ISDN? huh?

Re:Yes it is a peering problem ...

[Anonymous Coward]

And yet more 'shilling'...noone expects or assumes that they should get the 'maximum data rather through their "last mile" to every site on the planet'...at least no one that reads Slashdot & has even an INKLING of how the internet works...your making up strawman arguments that don't exist.

Heck, Cogent & Level-3 aren't adverse to paying for the capital costs of upgrading the networking equipment at the 'choke point' but they are NOT willing to pay extortionist fees over & above that cost...Comcast's et.als. disclaimer's on 'not being able to guarantee speed' has NOTHING whatsoever to do with this dispute since just upgrading the peering connections will alleviate the issue and ensure that the content is being provided 'at the bandwidth necessary as requested by their customers at peak times'...it is Comcast's OBLIGATION to do this...any good network engineer would have no issue with identifying the bottleneck & upgrading the equipment at the choke point & would be ecstatic if someone else agreed to pay for the entire cost of upgrading that choke point...

Re:Yes it is a peering problem ...

[Anonymous Coward]

Because of some sort of ideological bias on your part, you fail to see why your statements are so illogical.
Firstly,
1. "sender pays for the packet" - Yes, but irrelevant, Netflix paid their ISP already. Why should they pay Comcast? Comcast is not Netflix's ISP. Are you saying that for any company on the internet, that suddenly begins to send lots of packets via its internet connection to a connected ISP, then that company needs to pay that other ISP?
2. "don't like it, get a different ISP" - Did you even read the article? If that were an option, more people would do it. Since it is NOT an option, sensible people discuss solutions.

Honestly, I don't really get what you are arguing, you seem to be upset at some personal experience that this situation is similar to. But I don't really see how anyone who is fair of free markets working correctly can support the "Eyeball" networks in this case. Deliberately letting your own customers have a bad experience for MONTHS in order to bully Netflix into paying you money is completely immoral and unethical even if it is legal.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Because ISPs charge for colocation on their networks. They always have. They always will. Every CDN pays for this. Every directly hosted company pays for this. Only NetFlix seems to think that the agreement is supposed to work the other way. If NetFlix wants to pay for the colocation they are free to do so. They used to do so before they dumped Akamai (with whom they never experienced these problems) and went with Cogent. NetFlix is forcing these problems very intentionally as a political bargaining chip. They want to redefine decades of standard operations in order to get a free ride.

Re:Yes it is a peering problem ...

[Anonymous Coward]

Fine, but then service providers have to establish a level of service in which the average peak levels are met with consistency.

To push a bad analogy (but better than yours) nobody will scream if an airline oversells all their flights but only one flight in fifty is actually overcapacity by a seat. Everyone will rightfully scream if forty flights out of fifty is overbooked by ten seats.

And then Netflix comes along and says, "Hey, we'll buy and build extra terminal space and even pay for a brand new airplane, just so all passengers can fly comfortably on time.," and the airline says, "No, thanks...."

Either they meet the average peak demands, or they need to stop promising what they can't deliver.

Re:Yes it is a peering problem ...

[Anonymous Coward]

What a stupid troll. Pulling traffic is exactly how the Internet works.

Re:Yes it is a peering problem ...

[Anonymous Coward]

So Comcast can install a 56k modem on their end and demand even more money, because now everything is even more congested?

Re:Yes it is a peering problem ...

[Anonymous Coward]

Both the Netflix will pay for a caching server at the ISP which will reduce bandwidth to almost 0%. And Netflix's ISP offered to pay for the hardware upgrades that Comcast needed to do to get enough bandwidth for Comcast's customers.

Both offers were rejected because Netflix is in direct competition with Comcast's video on demand services.

Re:Yes it is a peering problem ...

[Anonymous Coward]

The funny thing is someone asked Netflix why they were using Cogent when Cogent has such a bad history, and Netflix said they were forced to use Cogent because some of the ISPs refused to accept Netflix traffic via any other means. Then those ISPs would point at Netflix and be like "See, Netflix is using a crappy provider". No shit, you forced them to.

Re:Yes it is a peering problem ...

[mgcarley]

When they are running their links (peered or not) at 100% for 24 hours a day, you bet your ass they should upgrade. ESPECIALLY if someone else offers to foot the bill.

Will it be a permanent fix as bandwidth demands increase? Hell no, but that's what operational expenses are.

In some countries, if an ISP's links are running at more than a certain percentage of their capacity for more than a certain percentage of the day and they are doing nothing about it, they would be found to be falling afoul of the regulations and could very well be fined. Additionally, in some countries if an ISPs contention ratio gets too high, that's also going to cause an issue with the regulator (even though it's a bullshit measurement for so many reasons).

If US ISPs here were told they couldn't have a ratio of more than 50:1 (1mbit/s per 50 mbit/s sold) it might help - you get sold a 50mbit/s line, so your ISP has to have at least 1mbit/s of bandwidth at the border.

For example: 2 million subscribers each with 50mbit/s means only 2 million mbit/s (2 terabits) at the network border, which is very doable - even a pair of Juniper T1600s will do that - and that's a 3 or 4 year old model. An ISP with that many subscribers should definitely be present at more than 1 IXP, so if we assume 8-10, you're only looking at 200-250gbit/s in each.

Comcast in particular already announces that it has traffic levels exceeding 1Tbit/s, so they surely already have the equipment to do it (just not the incentive, I guess).

Re:Yes it is a peering problem ...

[mgcarley]

Now *this* post, actually makes sense and I fully agree with it. We have to explain this to people //all//the//time//.

Re:Netflix, in the parlor, with the fireplace poke

[Anonymous Coward]

Yeah, I'm sure netflix just goes around dumping truckloads of data on the information superhighway just at random, and picked on Comcast like a bully.

Oh wait, every one of those streams were requested from users of Comcast's network who thought that those awesome 150mbit internet speeds comcast advertised were real.

Net neutrality is a constituional right!

[Anonymous Coward]

I should be able to use my bandwidth any way I please! It's freedom of speech! They can't throttle netflix! George Bush would be rolling in his grave!

This is a ploy from Obama to help spread misinformation about ebola (Obola) on his path to further to destroy the country!!1!

Duh!

[gstoddart]

The report proves the problem is not technical, but rather a result of business decisions.

Is anybody actually surprised by this?

This has always been about maximizing profits, and preventing a competitor from gaining access to your customers.

Because cable companies are ran by assholes.

Re:How many engineers does it take to screw netfli

[MozeeToby]

Engineer - "Hey Boss, we need some cash to upgrade the connection to these networks."
Boss - "What?! We just upgraded those connections a couple years ago"
Engineer - *rolls eyes* "Well the link is saturated, looks like lots of people watching online video... Netflix comes in over this connection so it makes sense"
Boss - "First they take our subscribers now they're forcing us to upgrade our equipment... well fuck em!"
Engineer - "Waaah?"
Boss - "You heard me, fuck em!"
Engineer - "But... our customers will get terrible service when they try to watch Netflix, or do anything else on that network for that matter"
Boss - "Exactly!"

Too bad

[Anonymous Coward]

the hipsteriffic site is unreadable to me.

If you made a properly formatted petition ...

[Anonymous Coward]

I would sign it...
The slashdot effect would likely cause it to be signed hundreds of thousands of times...

Would you mind?

Re:If you made a properly formatted petition ...

[suman28]

You are correct. The slashdot effect would be great, but no one cares about the opinions of the common folk. We already saw that with the Net Neutrality debate on the FCC comments debacle. In the end, if you are not greasing Tom Wheeler's cock, he will not be very nice when he gives you the rim job.

Re:Netflix, in the parlor, with the fireplace poke

[putaro]

The traffic isn't transiting Comcast going to another network. It's going to a Comcast subscriber who wants to watch a movie. So, yes, the subscriber is requesting a movie and the data is being delivered to them. There's no other route to the subscriber than through their ISP.

Re:Netflix, in the parlor, with the fireplace poke

[Anonymous Coward]

OMG, you are so right, and those request HTTP packets that started the whole SEND SEND SEND also came from the people seeking "free dumping rights". Damn, I wish I was as smart as you with regards to layer3 and TCP Connections.

Why rely on peering?

[Anonymous Coward]

I'm confused why Netflix, as a content provider, would only be peered with a single ISP. When I used to work for a content provider we would directly connect to all the Tier 1 service providers and had our own BGP ASN for multi-homing to avoid the peering problems that caused poor performance for our customers. Netflix seems big enough that they should be doing something similar.

Re:Why rely on peering?

[Anonymous Coward]

You're right! And that is the proof that this was a Netflix & Cogent stunt.

They intentionally dumped this traffic on the connections of the ISPs who wouldn't give them free expansions, in hopes that the customer complaints would force their hand.

Netflix, the sender of this traffic, has always been the one in control of the distribution method of their data. They intentionally sent it down an overloaded path.

Re:Why rely on peering?

[robbyb20]

If I read correctly, Netflix is hosted with Amazon. So its Amazons connection that needs to be looked at, no? I could be completely wrong as well, so not really pushing this.

http://en.wikipedia.org/wiki/N...

Re:Why rely on peering?

[Cramer]

They have (had?) more than one provider. They have their own ASN - AS2906. It's readily apparent they suck at traffic engineering. (or they let it happen to try to push Open Connect.)

Re:Why rely on peering?

[NormalVisual]

They intentionally dumped this traffic on the connections of the ISPs who wouldn't give them free expansions, in hopes that the customer complaints would force their hand.

Except you're not telling the full story. Netflix was well aware of the potential congestion issues, and offered their to place their own CDN boxes at no cost to any ISP that asked in order to alleviate this. Comcast chose not to avail themselves of this offer, as they couldn't double-dip if they did.

Re:Why rely on peering?

[Anonymous Coward]

They intentionally dumped this traffic on the connections of the ISPs who wouldn't give them free expansions, in hopes that the customer complaints would force their hand.

Except you're not telling the full story. Netflix was well aware of the potential congestion issues, and offered their to place their own CDN boxes at no cost to any ISP that asked in order to alleviate this. Comcast chose not to avail themselves of this offer, as they couldn't double-dip if they did.

And you do not understand that providers like Comcast charge to host cache servers in their network. Do you really think that Comcast hosts Akamai servers for free? The offer from Netflix was purely a publicity stunt to make it seem like they were trying to be reasonable to their customers. The only reason why this worked from a publicity standpoint is because the vast majority have no idea how ISP peering works, how cache server hosting works, etc.

Re:Why rely on peering?

[NormalVisual]

Do you really think that Comcast hosts Akamai servers for free?

I've read that they do charge, and I've read that they don't. It's in their best interest to do so, unless you have a citation to the contrary.

Multiple CDN contracts?

[CaptSlaq]

I assume there's a reason why Netflix hasn't perused the idea of load balancing across CDNs? Yes, it'd be a pain in the ass, but I know both Akamai and Limelight will read from your source to deliver bits to an end user.

It'd be a hell of a lot better than buckling to ISPs. At least you're in control of your costs at that point.

Re: Multiple CDN contracts?

[bill_mcgonigle]

Netflix is its own CDN - they will give, for free, one or more caches to any ISP, causing any one movie to transit the ISP's nonfree network connections only once.

But this is about competition for video services, not caching.

Re:Multiple CDN contracts?

[Talderas]

They have but they didn't want to pay anything close to the costs of what Akamai charges or anything close to the costs Akamai has to host their CDN.

Admittedly, they may have some weird stipulations going on with copyright and all that jazz.

Re: Multiple CDN contracts?

[killfixx]

This is a very salient point. Netflix already has these arrangements with other ISPs. Only Comcast, AT&T, and Verizon (surprise surprise) refused to host local caching servers. Of course, this precedes their demands for more money because, "Waaaahhhh...they're stealing our customers, they need to pay!".

Netflix tried to be the better entity (within reason) and were told, in no uncertain terms, "Go fuck yourself."

Yay, free market!

*sigh*

Re: Multiple CDN contracts?

[alen]

except CDN's have paid ISP's for years for hosting costs. netflix refuses to pay and wants their CDN hosted for free

Re:Multiple CDN contracts?

[PRMan]

All kinds of wrong. They will provide a server box to be housed in the ISP containing their most popular content FOR FREE.

Re: Multiple CDN contracts?

[Anonymous Coward]

That's not always the case as we most certainly had an Akamai server hosted within the CO I worked at for free.

Re: Multiple CDN contracts?

[Anonymous Coward]

No, if netflix was its own CDN they would HAVE caches in all ISPs they needed. What netflix is doing is allowing you to run a CDN for them and in return they don't mess you up.

Re: Multiple CDN contracts?

[Anonymous Coward]

This is complete nonsense. Google and Akamai both place hardware just inside ISP borders at no cost. They don't pay the ISPs to do this, the ISPs are saving a shit-ton on transit and are happy to set aside a little space and electricity for the hardware. This includes Comcast. Try doing a traceroute to 23.79.61.240 for example, this is one of at least a dozen Akamai servers sitting in a Comcast cage in Atlanta. Akamai is paying zero dollars for that.

Netflix offered the same arrangement and Comcast said no, the only reason being that Netflix sells a competing product to Comcast's own video on demand. It's anticompetitive behavior, period, plain and simple.

Re: Multiple CDN contracts?

[Cramer]

So I should be able to demand Verizon install ("host") my server(s) for free as well? Not going to happen.

Netflix is a FOR PROFIT BUSINESS. They can pay for services just like everybody else. (speaking as Verizon) Why should I bear the cost of hosting their business? It isn't costing me customers. And I'm sure as hell not going to give those asshats at Cogent anything; they're being paid boatloads by Netflix but won't buy the interconnects to support 'em.

Yes, there are small(ish) ISPs hopping on the Open Connect bandwagon. For them, it's a cost effective solution vs. the alternatives -- lost customers, or additional expensive bandwidth. Verizon (et. al) simply aren't going to play those games: Cogent can buy the bandwidth necessary to support their customers, or Netflix can find a different (preferably direct) path.

Re: Multiple CDN contracts?

[jcam2]

1 - What evidence do you have for how much Comcast charges Google or Akamai for placing hardware at their facilities?

2 - Google is just as much a competitor to Comcast via Youtube and the Google Play movies store. Why is only Netflix being targeted?

Re: Multiple CDN contracts?

[tepples]

Google is just as much a competitor to Comcast via Youtube

Since when does Comcast offer a platform for amateurs and small-time professionals to publish their videos? I thought Comcast was for the Disneys, Scrippses, and Discoverys of the media world.

Re: Multiple CDN contracts?

[Anonymous Coward]

Just wait...2 will happen 'soon enough' if & when Google video traffic gets large enough...but I'm not sure that Comcast et.al. what to play THAT game especially when Google has a SHITTON of users daily on their services and can easily post a message something to the effect 'experiencing poor search performance...call your internet provider as you are not likely getting what you pay for!'

Netflix is big...Google is HUGE (not in bandwidth needs in company size & importance to the internet)...but not to worry the monopolies will get around to Google, Facebook & the rest of the 'providers' to extract extortion when they think their safe to do so...

This is kind of relevant...

[Svartalf]

The Very First Honest Cable Internet Provider

WHy net neutrality doesn't work

[KingMotley]

You are totally correct here. ISPs should only be allowed to be content producers, or content distributors, IF they relinquish all their monopoly statuses with local municipalities. Comcast, Time Warner, etc should be taken to court under anti-monopoly laws in the US. As they are guaranteed monopolies and their behavior is definitely harming consumers and they are trying to leverage their monopoly in one sector to give them an unfair advantage in a different sector, this seems a rather simple case, but well... lobbying... money... corruption... self-serving politicians... yeah.

Re:How many engineers does it take to screw netfli

[Jason Levine]

Replace the Boss' last line with:

Boss: "Oh, boo hoo! What're they gonna do? Leave us and go to another ISP? We've got a monopoly in the area! They want Internet? They need to come to us. Besides, if we make Netflix look bad, maybe people will dump them and pay us $200 a month for cable again!"

How is this not a neutrality issue?

[Rujiel]

Regardless of the mechanism for throttling traffic, Comcast's customers are being disallowed access to a certain network because Comcast allows congestion only for traffic to those sites. Comcast and Verizon have a direct conflict of interest, providing services similar to those of competitors that they are throttling.
The idea that there is a bandwidth "shortage" caused by services like Netflix is fucking laughable, and a total spoon-fed excuse to rob paying customers of the service they deserve.

Re:How is this not a neutrality issue?

[Obfuscant]

Regardless of the mechanism for throttling traffic, Comcast's customers are being disallowed access to a certain network because Comcast allows congestion only for traffic to those sites.

You didn't even read the summary, did you?

And the effects victimize not only companies targeted but ALL incoming traffic from the affected transit network.

It isn't congestion only to Netflix sites, it's every site that routes through the congested peer connection. And it isn't "disallowed", it's only slowed down.

and a total spoon-fed excuse to rob paying customers of the service they deserve.

"Deserve" is a very subjective term. What did they pay for? And if you say "100MBps (or any specific number) from every service anywhere in the world to their client", you're wrong.

The fact that the issue is congestion at a peering point means it is a technical problem. Who pays to upgrade that connection is a business decision, and there is more than one party involved in that game. Should the service that is being paid for the content and sending large amounts of data pay, or should the company that hasn't promised full-time full-rate data anywhere off-net have to pay when their customers demand full-time full-rate data streams from off-net? Is that fair to the customers who haven't asked for that level of service?

Re:How is this not a neutrality issue?

[Anonymous Coward]

The backbone of the Internet is just fine. If your ISP can't handle getting your 100mb to everywhere in the world, then your ISP should pay someone who can. It's not that expensive. Heck, it's cheaper than what Comcast charges. Until the backbone of the Internet starts having congestion, your argument is a complete strawman. The only places in the world with connection issues are places like Africa or India, but they have other pressing matters to worry about first. Other areas with connection problems are because of technical limitations, but those are about to be "fixed" with long distance multi-teribit links.

The island of Madagascar has about 3tb/s of Internet access, enough bandwidth for the entire USA, and that island has a GDP of $10.025 billion with an average per capita of $458. I see they can afford better Internet. Maybe that's what we need to do, go back into a farming culture

Re:How is this not a neutrality issue?

[tepples]

Who pays to upgrade that connection is a business decision

Cogent offered half a year ago to pay the actual costs of the upgrade. The ISPs in question appear to want to extract a markup on top of that.

Re:How is this not a neutrality issue?

[Obfuscant]

The relevant bit of your link says:

To be clear, Cogent is not offering to enter into paid peering arrangements with these or any other networks. Rather, Cogent is simply willing, at this time, to incur the capital costs associated with augmenting its interconnections with these networks to address the current level of traffic congestion.

So Cogent isn't offering peering to anyone that doesn't have it, and is paying only the capital costs, not the ongoing fees. It's also only covering the "current level" of congestion.

To use an amazingly appropriate analogy, that would be like Comcast offering to provide you free installation and a free digital decoder so it could charge you another $40/month for enhanced television service. Sounds like a great deal, Comcast paying the capital costs of your upgrade, huh?

Re:How is this not a neutrality issue?

[tepples]

What "ongoing fees" are just and why?

Re:How is this not a neutrality issue?

[Obfuscant]

You're either a troll, kidding, or just don't know what "operating expenses" are.

Re:How is this not a neutrality issue?

[tepples]

Then please explain what increased ongoing expenses Comcast would incur by upgrading its link with Cogent.

Comcast fanboy detected

[Rujiel]

Wait, did I say comcast fanboy? I forgot, those don't exist. Guess I should have said:

Paid shill detected

Re:How many engineers does it take to screw netfli

[operagost]

FFFT

*rubs nipples*

Another misleading headline

[jbmartin6]

On the summary, not the article. Comcast wasn't "jamming' Netflix. Jamming is an active response. What Comcast did was nothing. Now whether they should have done something about the overflowing links is the argument, but that's a far cry from "jamming."

Re:Another misleading headline

[Anonymous Coward]

jam1
jam/
verb
gerund or present participle: jamming

        1.
        squeeze or pack tightly into a specified space.
        "four of us were jammed in one compartment"
        synonyms: stuff, shove, force, ram, thrust, press, push, stick, squeeze, cram More
        "he jammed a finger in each ear"
        crowd, pack, pile, press, squeeze, squish, cram, wedge;
        throng, mob, occupy, fill, overcrowd, obstruct, block, congest
        "hundreds of people jammed into the hall"
                push (something) roughly and forcibly into position or a space.
                "he jammed his hat on"
                synonyms: stuff, shove, force, ram, thrust, press, push, stick, squeeze, cram
                "he jammed a finger in each ear"
                crowd onto (a road) so as to block it.
                "the roads were jammed with traffic"
                cause (telephone lines) to be continuously busy with a large number of calls.
                "listeners jammed WBOQ's switchboard with calls"
        2.
        become or make unable to move or work due to a part seizing or becoming stuck.
        "the photocopier jammed"
        synonyms: stick, become stuck, catch, seize (up), become trapped More
        "the rudder had jammed"
        immobilize, paralyze, disable, cripple, put out of action, bring to a standstill;
        clog
        "dust can jam the mechanism"
                make (a broadcast or other electronic signal) unintelligible by causing interference.
                "GPS signals are weak and easily jammed"
        3.
        informal
        improvise with other musicians, especially in jazz or blues.
        "the opportunity to jam with Atlanta blues musicians"
        synonyms: improvise, play (music), extemporize, ad lib
        "we were just jamming and his amp blew"

Hm... Definition #2 looks suspiciously like it fits. There's no discussion of passive or active on this- because it isn't specific to one or the other. You can effectively jam a communications channel by literally doing nothing when you should do something to prevent it.

If it's misleading...it's because you don't understand the very English you're reading and writing. Ah, but this is /., after all.

Re:Another misleading headline

[Anonymous Coward]

Sorry. For all practical purposes #1 is the most appropriate definition, but it's Netflix is the one doing the jamming. You can play bullshit semantics games all you want,but its the sender that jams a communications channel, not the receiver. You are not being intellectually honest.

Re:Another misleading headline

[jbmartin6]

Being snide isn't useful. Since Netflix/Cogent would be the one jamming traffic into the pipe, not Comcast, my point still stands. One cannot "jam" something by any of these definitions by doing nothing. The jamming is done by something or someone else.

Re:How many engineers does it take to screw netfli

[torkus]

Actually netflix offered to foot the bill for upgrading the bandwidth - it's literally a couple cross-connects in a datacenter, maybe a fiber card or two.

Oh, and netflix ALSO offers to drop a server in your datacenter *free* which caches all the common netflix streams. This reduces the internet bandwidth demands by something like 90+% since it lives within the ISP's datacenter and just needs to download each stream once.

But the last line is exactly the point. The ISPs are also TV providers and they don't want you to have a good netflix experience. If they can passively let that happen...well of course they will. No one can accuse them of taking any action to damage your netflix streaming...it's their complete inaction that's resulting in it.

Techdirt Article on Same Story

[carrier lost]

"Similarly, M-Lab's data shows that the problem wasn't a lack of bandwidth on the part of the ISPs (i.e., no actual technical congestion), because those same ISPs had no problem connecting to a different transit provider, Internap. So the only logical culprit was the interconnection points. There was more than enough bandwidth to go around. There's just the single bottleneck of the interconnection border router (which, again, is trivially simple to get rid of by opening up more ports)."

link

Re:Techdirt Article on Same Story

[Svartalf]

Basically, it was a pissing match like people claim it was. Something that USED to be called out on the carpet over- because it's violating common carrier status that the jokers in question all have and alternately want and don't want. (They don't want the regulation, but they want the shield from vicarious liability from their customers' actions...)

Re:Techdirt Article on Same Story

[carrier lost]

Ha, your sig

I keep telling my GF I don't want to be a citizen of California (we live in Nevada)

Re:Netflix, in the parlor, with the fireplace poke

[Obfuscant]

Oh wait, every one of those streams were requested from users of Comcast's network who thought that those awesome 150mbit internet speeds comcast advertised were real.

150mbps isn't very fast. Did you mean 150Mbps?

The issue is that people think the 150Mbps "last mile" speed was some kind of promise of speed to every other site on the planet. I can request a data stream from a service that needs 100Mbps to function reliably, but that doesn't mean I'm guaranteed 100Mbps from end to end. If an ISP promises you that, you know they're lying. If you assume the last mile speed applies to every connection you make anywhere, then you're the one who's being foolish, not the ISP being dishonest.

Sarbanes-Oxley

[Anonymous Coward]

The executives oversubscribed their infrastructure? Has this been reported as part of their SOX audit?

Food for thought.

All this amounts to....

[arbiter1]

This is nothing more then Anti-competitive practices against competing products that are cheaper then theirs.

End the ISP monopolies

[Jodka]

from wikipedia

Franchise fees are governed under Section 622 of the Cable Communications Act of 1984.[2] Section 622, states that municipalities are entitled to a maximum of 5% of gross revenues derived from the operation of the cable system for the provision of cable services such as Public, educational, and government access (PEG) TV channels.

Franchise fees are fixed at a maximum of 5% of gross revenues. So how do municipalities maximize revenues from franchise fees? By maximizing cable company gross revenues. And how do municipalities maximize cable company gross revenues? By creating monopolies! By awarding exclusive license to one provider to extract monopolist profits from the public.

Note that there is nothing inherently wrong with permitting local governments to charge cable companies fees. That is justifiable to the extent that local governments incur costs of infrastructure repair with damage from cable installation. All that is needed is a single addendum to the law, one prohibiting local governments from creating monopolies. The law could simply mandate that municipalities must offer franchise licenses to all ISPs if they offer licenses to one and that all licencees must be be charged at the same rate.

The only reason we have cable monopolies in the U.S. is because the Cable Communications Act of 1984 created that perverse incentive. Other countries without such laws have much faster service at much lower prices.

If federal law permitted local governments to do this sort of thing with groceries, computers and cars we would have regional monopolies for those products as well. Be grateful that your town council is not permitted to sell grocery, computer and car franchises.

Re:End the ISP monopolies

[Svartalf]

The biggest problem without them is you'd end up having a tragedy of the commons. How would any of them intercommunicate to allow the Internet to be.

One of the things you can do is harshly punish the ISPs in question when they play games like this. One of the things they have right now is "common carrier" status. That's a liability shield against all sorts of things that your customers might do that's illegal. You could be held vicariously liable if you don't have that status and they commit acts of sedition, copyright infringement, etc. You used to run the very real risk of losing that status as a provider of services if you pulled a stunt like this- which kept them mostly from pulling crap like this. We need to bring that back, to be honest.

Public Utility Regulation =Yes, Net Neutrality =No

[Anonymous Coward]

This is what people should be upset about. "Net neutrality" is unnecessary at this point, and can potentially do more harm than good. What really needs to happen is to break up the ownership of the physical lines and the ownership of the network traffic being routed in and out of them so that companies that have practical or absolute monopolies over the lines cannot leave these speed bumps in place without someone else offering to sell a quicker route.

Re:Netflix, in the parlor, with the fireplace poke

[Cramer]

Netflix has other ISPs, and significant traffic engineering power at their disposal to get traffic to flow how they like. The simple fact that people could "VPN around" the congestion is proof Netflix could have used a different path to that user. However, it's easier to whine to the media in lame attempts to get Open Connect in the door. (i.e. host our business for free.)

Plus, they bought bandwidth from some of the cheapest providers around, who everyone knows isn't going to care when they fail to deliver. (Cogent is infamous for "peering disputes".)

Fuck this im going dialup

[Anonymous Coward]

anybody recommend a good dialup isp?

Split them up, please

[pellik]

Going with the notion that excessive regulation has it's own cost, maybe the solution here is to split up the cable giants? Comcast could easily be three separate companies- one that manages infrastructure and charges for it, one that sells TV channels to customers, and one that sells internet. The two later companies would just lease bandwidth from the former.

Re:How many engineers does it take to screw netfli

[fibonacci8]

There's too much revisionist history in the Boss's statement for my tolerance. Boss- "First they take our subscribers now they're forcing to upgrade our equipment... well fuck em!" First they had subscribers, then Netflix introduced the online version of their service (circa 2008) bringing in more subscribers, then Xfinity got added (circa 2010) offering the speeds demonstrated by Netflix carried by Comcast but with a selection of Comcast's cable offerings. fast forward a few years and Netflix is still the better offering so Comcast strategically decides to upgrade in ways that improve Xfinity on-demand services without accepting offers from Netflix to likewise maintain the quality of service Netflix customers experienced in the past and presumably would still get to experience if the dollars spent on monthly connection fees to Comcast actually went into improving total network quality instead of just promoting Xfinity services. It would be nice if it were just friendly competition going on, but it's Comcast trying to cut in on the streaming market after Netflix showed it was profitable.

Careful not to turn them into utilities

[Karmashock]

Listen, we will get what we want right now by turning big monopoly ISPs into utilities.

However, we have to think strategically. If we only think tactically we will get out maneuvered.

Play chess for a moment. What happens when the big ISPs are utilities? Yes, they will be regulated more heavily but they will also have their protection from competition enshrined more deeply in law.

What is more, the federal government will gain increasing control over the internet. Consider the NSA for example. Do you for an instant think they won't exploit this situation? If the FCC starts dictating things to the ISPs how hard would it be for the NSA to go to the FCC and get them to put an NSA box at the ISP? Child's play.

And that is just the beginning.

The wild free days of the internet are over if we turn the ISPs into utilities. Will the ISPs fuck us on occasion? Yes. But I'll take that on an ongoing basis rather then give the feds total control. Because at least with the ISPs there is hope. With the feds it is gone. You'll start seeing regulations on free speech, requirements that people use real names on the internet, micromanaging of peer to peer content.

I mean seriously what do you think is going to happen once the feds get this power?

The solution has always been greater competition. it is the only way to eat our cake and have it too. With greater competition the ISPs can't fuck over their consumers without losing market share and they aren't being micromanaged by politicians.

Best of both worlds.

Competition. Please.

Re:How many engineers does it take to screw netfli

[UnknownSoldier]

/Oblg.

South Park - Cable Company
https://www.youtube.com/watch?...

Nationalize Internet Service

[Anonymous Coward]

Internet service should be nationalized with a structured upgrade schedule. It is essential to becoming a first rate country.

No detailed report required

[Anonymous Coward]

I've mentioned this in other threads involving peer points.

My company has Cogent, Verizon, and various other carriers with 100mbit or larger business connections at our offices around the US.

Every office we have that has Cogent has problems with speed with every office that has Verizon. One of them to squeeze 15-30 mbit/sec for extended periods between them but that is the best it gets (and that is between a 1gb Verizon to 1gb Cogent connection). Most are FAR worse like under 200kb/sec for hours on end. One of them even dips down to under 50kb/sec. For one of the offices we investigated further looking for answers. Cogent did speed tests for us one hop before the peer point with Verizon and we achieved near line speed. Cogent also showed us metrics they see at that peering point and that they have offered to increase the speed at that point but Verizon has not considered it. During the troubleshooting Verizon basically told us there is nothing wrong with our Verizon end of the line or their network and it's not their problem, call Cogent. We got an excuse along the lines the internet is unpredictable and there are many things beyond their control, as if I was a home customer calling Verizon level 1 support because the interwebs were slow.

Cogent to XO, XO to Level3, Level 3 to Verizon, XO to Verizon, Sprint to Verizon, and so on all all perfectly fine. Just Verizon to Cogent is the problem.

This is happening, peering points ARE the new defacto way for companies to create fast and slow lanes and cheery pick traffic. When you have one company like Comcast or Verizon acting as a long haul T1 carrier AND acting as the last mile this is bound to happen. They can use tricks like this at the T1 peering level to manage their load at the last mile.
In my companies case, we are getting dedicated ethernet from Verizon in managed data centers and we suffer the same last mile effects.

Cogent is willing to pay these costs

[tepples]

Since when were fibre cables, $20000 optics, Switch ports, and 40-Gigabit port licenses free when the link is turned off?

Not free, but Cogent is willing to pay these costs itself. Verizon and Comcast won't take Cogent's offer; they want to charge Cogent an arguably excessive markup on top of Cogent's costs

Summary: Netflix breaks Internet

[Dr J. keeps the nerd]

The data the article presents really just shows a ton of Netflix traffic breaking the Internet for other users. Shouldn't all that adaptive bitrate stuff make it NOT break other flows? Apparently, not so much. Did Netflix respond by making their video delivery less aggressive, the way Bittorrent did with LEDBAT? No.

What did we learn?
1) Netflix breaks any link it's on. Period. Full stop. The rest of the Internet only gets through when Netflix isn't peered together with it.
2) Therefore ISPs -- ALL ISPs -- bad.
3) Therefore Net Neutrality so Netflix can break the Internet.

Of course, one might be tempted to conclude that big data users should work out their own peering and financial arrangements so that they don't mess up the Internet, but that would make one a corporate shill.

Ask about a Netflix exclusive show

[tepples]

The ISPs are also TV providers and they don't want you to have a good netflix experience.

Then ask about a specific work to which Netflix has the rights and the TV provider division of the ISP does not. "I'm having trouble watching House of Cards at home. It works fine on $different_isp_next_town_over. Might this be a problem with Comcast?"

Packets != circuits

[tepples]

I make a 'call' from Comcast to Netflix...

And Netflix "calls" you back with the data.

A circuit-switched network such as the PSTN allows sending information in both directions over one "call". A packet-switched network such as the Internet, on the other hand, doesn't see "calls"; it sees "datagrams". Except for last mile customers, each side pays for how many packets it sends. Otherwise, it'd be possible to manipulate billing by doing the equivalent of the difference between PORT and PASV in FTP.

Internet Protocol is stateless

[tepples]

You are correct that TCP is stateful. But the fact that TCP is stateful is irrelevant. ISPs are Internet Service Providers, and Internet Protocol is stateless. From the point of view of an ISP's infrastructure, TCP is just an application that runs on Internet Protocol. Otherwise, it'd be possible to manipulate billing through the equivalent of switching between FTP's PORT and PASV commands, which change only who sends the SYN.

Re:Internet Protocol is stateless

[AK Marc]

IP isn't a protocol. It's called TCP/IP for a reason. You are too dumb to educate. You probably know the answer, but are lying to yourself to win an argument that everyone reading this knows you already lost. IP is a suite of protocols, TCP being one of them. THat you don't even know what IP is, yet feel the need to lecture others on it proves you are too dumb to understand it.

Re:Internet Protocol is stateless

[tepples]

If you feel so strongly that I am lying to myself, then please explain what is incorrect in the following three statements: TCP is a connection-oriented protocol in the Internet Protocol Suite. TCP connections have two halves, one in each direction. Traffic is billed based on who sends more data down each half of the connection.

Re:Internet Protocol is stateless

[AK Marc]

Traffic is billed based on who sends more data down each half of the connection.

In the "old days" traffic was billed by who received the most. User pays.

So you are still wrong. If you were right, I could start a small ISP that only had residential connections (essentially "receive only"). And, since I sent nothing, and the other guy sent me more, I could charge my users $0 and still make money from all the people paying me to take their traffic.

But it doesn't work that way. The small ISP pays a lot to receive data, not send it.

Reality proves you wrong.

Comcast's private property

[tepples]

The ISP's premises is the private property of the ISP. If Netflix wants its box sitting in Comcast's private property, it can pay rent, just as you would have to pay rent to lawfully live on private property owned by a landlord.

Only works for a monopoly

[Anonymous Coward]

The insanity of local monopolies on display again... I have the choice between several different retail ISPs for ADSL or cable or wireless (3G etc). The backend infrastructure runs over one of six networks and it is possible to determine which of those networks is being offered by each retail ISP. If an ISP blocked, shaped or otherwise interfered with a media service I want, it would be very simple to just moved to another ISP that doesn't.

Where customers have a choice of ISP it would be simple for a large media provider, like Netflix, to blacklist or otherwise not support any given ISP and that ISP loses customers to their competition.

A free market allows customers the choice of providers but that only happens in countries were freedom is actually valued and the entire political system isn't corrupted by commercial lobbying...

I use apk's hosts file engine

[Anonymous Coward]

Its sources don't block hughpickens.com. In fact, it's not even in the hosts file his program creates, blocked or otherwise.I've used his host file program here http://start64.com/index.php?o... for about 2 years now for creating my initial custom hosts file and then using it for updating my hosts file daily with it. I go faster since I don't see ads and I am secured against known malicious sites too. I can't complain especially when it's free, and works great!

What if the conclusions are wrong?

[Anonymous Coward]

May be the hotpsots where the data is getting lagged is not because of intentional jamming of competition. The report clearly states that ALL traffic is lagged at these hotspots. In this paranoid world one could hypothethise that those hotspots are the government backdoor filters forced on big ISPs to monitor traffic. The result would be similar, such monitoring must have its cost on speed as you have to filter/archive data, ALL packets.

If you ask why then the ISPs own traffic is not altered, that can be explaind as well, they probably have the hook at a different central site for the ISPs stuff so the penalty is done for them at difefrent places.

Re:Netflix, in the parlor, with the fireplace poke

[perryizgr8]

The simple fact that people could "VPN around" the congestion is proof Netflix could have used a different path to that user.

That's not how it works.

Netflix Cache Servers Resolve This

[Anonymous Coward]

An ISP I used to work for has let Netflix install a caching server in their server farm. This thing does something like 1TB of updates during the night and has cleaned up their Netflix usage a lot. Netflix installed this system themselves and are apparently doing most of the management. There are real options for these companies to operate with ISP's that don't involve double dipping against growing competition. It's insane that this is happening like this.

Re:How many engineers does it take to screw netfli

[TangoMargarine]

Engineer - "Hey Boss, we need some cash to upgrade the connection to these networks."
Boss - "How about we raise their rates, just say we upgraded the network, and pocket the difference?"
Catbert - "I like the way you think."

FTFY

Nintendo was caught in this crossfire

[Anonymous Coward]

Nintendo's online services and download servers fell/are falling victim to this intentional congestion. The Level 3 to Verizon interconnect was left saturated, causing all sorts of trouble for Nintendo customers trying to download games and play online. Verizon's lower bandwidth services usually worked OK, but their FIOS customers were hit hard, sometimes to the point of un-usability.

It really really sucks to be told to take your new Wii U system to a friends house (with a different ISP) in order to download a multi-gig game since Verizon refuses to help and Nintendo can't do anything to make them provide the service that you are paying for.

If it hasn't been fixed, I expect Verizon's help lines will catch fire once Super Smash for Wii U launches.

Re:Netflix, in the parlor, with the fireplace poke

[Anonymous Coward]

They are honoring the REQUEST REQUEST REQUEST. You really don't know how networks work do you?

Sigh...

[Anonymous Coward]

Did you know Comcast pays more for lobby groups than any other industry? Second biggest lobby group is a military/industrial player.

research sponsored by Google

[Anonymous Coward]

who says "this won't happen if you switch to Google Fiber and tell your politicians to subsidize Our Product!"

If receiver pays

[tepples]

You make a good point. I guess my misconception was that sender pays for long haul transit and the endpoint pays for the last mile. But if receiver pays, even for long haul, then you can DDoS someone's billing by flooding his connection with packets. And if receiver pays, even for long haul, then why does Comcast slow down Netflix? All Comcast's customers are already paying.

Re:If receiver pays

[AK Marc]

And if receiver pays, even for long haul, then why does Comcast slow down Netflix? All Comcast's customers are already paying.

That's why people are angry. Comcast is already paid by its users for content. And Comcast tries to not deliver competing content. Making up reasons why, because "we are monopolistic douchebags" is not a good reason.

thanks

[sup4mak386]

thanks for useful article ..keep it up