Slashdot Mirror


First Detailed Data Analysis Shows Exactly How Comcast Jammed Netflix

An anonymous reader writes John Oliver calls it "cable company f*ckery" and we've all suspected it happens. Now on Steven Levy's new Backchannel publication on Medium, Susan Crawford delivers decisive proof, expertly dissecting the Comcast-Netflix network congestion controversy. Her source material is a detailed traffic measurement report (.pdf) released this week by Google-backed M-Lab — the first of its kind — showing severe degradation of service at interconnection points between Comcast, Verizon and other monopoly "eyeball networks" and "transit networks" such as Cogent, which was contracted by Netflix to deliver its bits. The report shows that interconnection points give monopoly ISPs all the leverage they need to discriminate against companies like Netflix, which compete with them in video services, simply by refusing to relieve network congestion caused by external traffic requested by their very own ISP customers. And the effects victimize not only companies targeted but ALL incoming traffic from the affected transit network. The report proves the problem is not technical, but rather a result of business decisions. This is not technically a Net neutrality problem, but it creates the very same headaches for consumers, and unfair business advantages for ISPs. In an accompanying article, Crawford makes a compelling case for FCC intervention.

153 of 243 comments (clear)

  1. Their answer to oversubscription as well by Anonymous Coward · · Score: 1

    Think twice next time you wonder why you aren't getting your advertised speed...

    1. Re:Their answer to oversubscription as well by ourlovecanlastforeve · · Score: 2

      That's quite a loaded statement.

      Business buggery is not the only reason you might not get the advertised speed.

      First of all, ISPs advertise "up to" a specific speed, which means that's the maximum bandwidth you're allowed.

      It doesn't state or imply that you'll receive that speed consistently.

      It means that, assuming the network is capable of that speed, if you were capable of getting higher speeds you'd be capped at that speed.

    2. Re:Their answer to oversubscription as well by Anonymous Coward · · Score: 2, Interesting

      "First of all, ISPs advertise "up to" a specific speed, which means that's the maximum bandwidth you're allowed.

      It doesn't state or imply that you'll receive that speed consistently."

      Well, how about I pay "up to" the amount they want, then? I can throw that (in small print) on the back of a check I send them, to make it all legal and such.

    3. Re:Their answer to oversubscription as well by JWSmythe · · Score: 1

      Sure you can do that. They'll (kind of) accept it too. Reducing your payment amount will result in reduction of your service.

      When you pay less than your bill, they'll simply reduce your service to 0.

      --
      Serious? Seriousness is well above my pay grade.
    4. Re:Their answer to oversubscription as well by David_Hart · · Score: 2

      Think twice next time you wonder why you aren't getting your advertised speed...

      And another article about Comcast throttling Netflix without any background or context.

      Backbone providers have peering agreements. Usually, if both backbone providers (i.e. Comcast & Verizon) are sending close to the same traffic between each other, the peering agreement is free for both parties. In the Netflix case, Netflix went with a small backbone provider, likely due to cost. The problem is that the backbone provider they chose sends way more traffic than they accept. Typically, this type of peering agreement means that the smaller backbone provider (i.e. Cogent) pays fees to the larger backbone provider (Comcast). It's my understanding that Cogent wouldn't or couldn't pay these fees, so Comcast throttled them.

      Because Cogent couldn't or wouldn't pay the fees and customers were complaining, Neflix agreed to pay the peering fees for Cogent. Though, this isn't how the media or Netflix presents it.

      I hate Comcast as much as anyone. I think that they are essentially a monopoly that takes advantage of their customers by increasing prices where there is little to no competition (a toothless FCC doesn't help). But, in this one instance, it's my opinion that Comcast had a good case against the provider used by Netflix and that, by selecting the lowest cost provider and possible knowingly selecting one that is a bit sketchy on the peering side, Netflix had some responsibility.

    5. Re:Their answer to oversubscription as well by thaylin · · Score: 4, Interesting

      Except you are confusing a transit and a consumer endpoint. Transit providers normally peer, but an endpoint is going to have more traffic coming in then going out because their consumers are requesting it, ALWAYS, but this is the first time they have been able to pressure people into these types of agreements.

      Peering agreements between transit providers is fine, but not when an endpoint bullies a service providor.

      --
      When you cant win, ad hominem.
    6. Re:Their answer to oversubscription as well by ShanghaiBill · · Score: 1

      Think twice next time you wonder why you aren't getting your advertised speed...

      It is unrealistic to expect them to build out enough bandwidth to let everyone max out at the same time. They would need far more infrastructure, and YOU would have to pay for it. Car analogy: If a highway carries 10,000 vehicles per day, do you complain that it doesn't have 10,000 lanes?

    7. Re:Their answer to oversubscription as well by Duhavid · · Score: 1

      "If a highway carries 10,000 vehicles per day, do you complain that it doesn't have 10,000 lanes?"

      I certainly do.

      --
      emt 377 emt 4
    8. Re:Their answer to oversubscription as well by David_Hart · · Score: 2

      No, I'm not confusing transit and endpoints.

      You are right that the end point connection isn't one sided. That's why we all pay monthly fees to our provider to pay for bandwidth. Our provider, though, has peering agreements with other providers. If this peering is unequal, some of the money that we pay our provider goes to pay for peering fees. In the case of Netflix, their provider refused to pay the peering fees. If our provider screws up, we suffer. We then have a choice to change providers. Netflix chose to pay Comcast instead (probably the cheaper option, factoring termination fees, etc.). But... they did have other options...

    9. Re: Their answer to oversubscription as well by GigaplexNZ · · Score: 3, Insightful

      The problem is that the backbone provider they chose sends way more traffic than they accept.

      And consumer ISPs give asymmetric speeds most of the time with EULAs that forbid running servers. It's pretty obvious that they'll accept more data than they send by design, so it's unreasonable for their peering agreements to assume symmetric transfers.

    10. Re:Their answer to oversubscription as well by fibonacci8 · · Score: 2

      When a toll road sells 10.000 people a monthly pass for a lane each, and then offers up to 5 lanes at a time during peak hours due to "maintenance", and yet has another 9,995 lanes constructed, uncluttered, and for sale in advertisements to other bidders I would certainly complain.

      --
      Inheritance is the sincerest form of nepotism.
    11. Re:Their answer to oversubscription as well by fibonacci8 · · Score: 1

      Making the toll road operator look even better, one of the competing toll road projects decides that those who car-pool should either pay more or drive at reduced speeds.

      --
      Inheritance is the sincerest form of nepotism.
    12. Re:Their answer to oversubscription as well by jythie · · Score: 1

      'How' is simple. Just be the party writing the contract rather then signing it. If you can not arrange such a thing, it is your own fault for being poor.

    13. Re:Their answer to oversubscription as well by morethanapapercert · · Score: 1

      Given the way many broadband ISPs oversubscribe their services, I consider weasel words like "up to X speeds" in the fine print while all the headlines and banner texts say "Now surf at X*" or "Fastest Internet in Y county!*" with all those asterisk footnotes to be a form of corporate buggery.

      --
      I need a wheelchair van for my son. Help me get the word out. https://www.gofundme.com/wheelchair-van-for-jj
    14. Re:Their answer to oversubscription as well by AK+Marc · · Score: 1

      The problem is that the backbone provider they chose sends way more traffic than they accept. Typically, this type of peering agreement means that the smaller backbone provider (i.e. Cogent) pays fees to the larger backbone provider (Comcast).

      Typicaly, the small provider isn't a host, but is a small ISP. residential ISPs accept way more traffic than they send. They get the highest fees.

      Part of the problem is that one ISP will download 10G and upload 1G and they'll be charged 10G rates, and the next ISP will upload 10G and download 1G and they'll be charged at 10G rates. The backbone will move 11G of traffic while being paid for 20G. And they'll complain about it all the way to the bank.

    15. Re:Their answer to oversubscription as well by AK+Marc · · Score: 2

      The requestor pays. That was the old system. They are now finding out ways of blackmailing the sender into paying. My fees to my ISP pay for all the bandwidth needed to get the bits from Netflix to me. Unless my ISP blackmails Netflix.

    16. Re:Their answer to oversubscription as well by tepples · · Score: 1

      But how feasible is it for a typical U.S. resident to qualify for a work visa in your country?

    17. Re:Their answer to oversubscription as well by TangoMargarine · · Score: 1

      But then I'm not in US.

      Ah, so you're almost entirely irrelevant to this conversation. Thanks so much.

      --
      Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
    18. Re:Their answer to oversubscription as well by repka · · Score: 1

      You're welcome.

    19. Re: Their answer to oversubscription as well by cthulhu11 · · Score: 1

      Peering is done with symmetric connections. Last mile consumer isn't -- the wire's capabilities are generally allocated more for downstream than upstream traffic. This is sophistry.

    20. Re: Their answer to oversubscription as well by GigaplexNZ · · Score: 1

      My point was that if the vast majority of the ISP customers only download, they can't expect to have any symmetrical peering as there's no traffic to balance it out.

  2. I'd like to hear Bennett Haselton's opinion by Anonymous Coward · · Score: 5, Funny

    Bennett Haselton is a frequent contributor, and I tend to hold-off judgement on these things until I read his fine points on the topic. I was recently particularly moved by his work on line queues at Burning Man. It changed my entire life. I now piss sitting down.

  3. Sigh... by MightyMartian · · Score: 5, Insightful

    "In an accompanying article, Crawford makes a compelling case for FCC intervention."

    That won't work unless it comes with a check with seven digits attached to it.

    --
    The world's burning. Moped Jesus spotted on I50. Details at 11.
    1. Re:Sigh... by Krojack · · Score: 2

      And that won't really work because an even larger check from the cable companies will always follow. Hell they most likely just give them a credit card with an unlimited balance that draws right from the cable companies bank account.

    2. Re:Sigh... by Em+Adespoton · · Score: 5, Insightful

      And that won't really work because an even larger check from the cable companies will always follow. Hell they most likely just give them a credit card with an unlimited balance that draws right from the cable companies' customers' bank accounts.

      FTFY

    3. Re:Sigh... by houstonbofh · · Score: 2

      Someone needs to file a Sherman Anti-trust Act case here... It covers most of the egregious stuff, like paying cities not to lay fiber.

    4. Re:Sigh... by Anonymous Coward · · Score: 1

      That's fine, Comcast already bought the Dems and the Obama administration so nothing will come of this.
      How Comcast bought the democratic party http://www.nationalreview.com/article/375116/how-comcast-bought-democratic-party-matthew-continetti

  4. Common Carrier by Anon-Admin · · Score: 5, Insightful

    Once again, a call for net neutrality will ensue. All we really need is for the FCC to call them Common Carriers and apply the age old law.

    It has already been applied to Telecoms and Utilities, just apply it to the ISP's and be done with this crap.

    http://en.wikipedia.org/wiki/C...

    1. Re:Common Carrier by Burdell · · Score: 4, Informative

      That would have zero impact. This is like the telephone company in city A have 96 channels to the telephone company in city B, but then 100 people try to make calls. Only some of them will go through, and that's a capacity issue, not regulated by Common Carrier status. They are not discriminating based on callers or anything, they are just "decliining" to upgrade capacity. In some cases, that could be regulated by state PUCs/PSCs, but AFAIK it is not normally. It is just up to the two carriers to reach an agreement.

      This type of thing happened a lot in the early dialup ISP days, when telecom deregulation spawed a lot of CLECs that had to connect to ILECs to carry calls. The ILECs structured the contracts with settlement money for to flow to the destination of a call (thinking most of the CLEC calls would be _to_ ILEC users), but then the CLECs went and got all the dialup ISPs to move modem banks to them. Suddenly all the calls went _to_ the CLECs, and the ILECs had to pay (some did not and went to court instead).

    2. Re:Common Carrier by Krojack · · Score: 1

      But with internet bandwidth, you have have overflow connections that are sitting dark and if needed just light them up and data will be flowing within seconds. This can even be automated. As traffic starts to fall off, turn those connections off.

    3. Re:Common Carrier by Jason+Levine · · Score: 5, Insightful

      It's more like Phone Company A has 100 channels to Phone Company B. They notice that there is a constant level of 99 calls between A and B. They also see that 25 calls are constantly being made to Joe's Pizzeria who uses Phone Company B for phone service. Phone Company A gets upset that Joe's isn't paying them (Phone Company A) for this traffic, so they refuse to add more channels. (Even though doing so would be inexpensive to do.) So calls to Joe's Pizzeria begin being dropped and Joe's customers get mad that they can't get through. Joe's finally signs an agreement with Phone Company A paying them money and suddenly the calls go through just fine.

      This is extortion plain and simple. Add in the fact that the ISPs doing this have an Internet monopoly/duopoly in their areas and also tend to provide video services - that Netflix competes against - and you have extortion plus the use of a monopoly to crush competition in another market. This deserves swift and severe action to show the ISPs that this is NOT to be tolerated. Unfortunately, at best we'll get a strongly worded statement and perhaps a fine that Comcast will make back in the time it took me to write this comment.

      --
      My sci-fi novel, Ghost Thief, is now available from Amazon.com.
    4. Re:Common Carrier by fustakrakich · · Score: 1

      Tuesday! Make it so! Show them who's boss.

      Baby steps... but it will wake people up

      --
      “He’s not deformed, he’s just drunk!”
    5. Re:Common Carrier by mysidia · · Score: 1

      That would have zero impact. This is like the telephone company in city A have 96 channels to the telephone company in city B, but then 100 people try to make calls. Only some of them will go through, and that's a capacity issue, not regulated by Common Carrier status.

      That's not the scenario. It's not capacity between cities, in this case, it is capacity between networks. The problem is they are discriminating against some networks and refusing to build capacity at the same time as they are building capacity to other networks for free; that's not a common carrier.

      A telco expands capacity based on utilization, and in building more capacity to other networks in the same area: it's not a case of some networks get capacity built to them for free and some have to pay, it's..... each telco pays their own costs to build that capacity needed by their customers AND asymmetric usage is settled through the USAGE fees associated with LD termination on each call.

    6. Re:Common Carrier by mysidia · · Score: 2

      Since when were fibre cables, $20000 optics, Switch ports, and 40-Gigabit port licenses free when the link is turned off?

    7. Re:Common Carrier by suutar · · Score: 1

      And the cable companies get their shills in congress to tell the FCC "they are not common carriers, try again"

    8. Re:Common Carrier by houstonbofh · · Score: 1

      No help as they can work around that. What is needed is to allow local last mile competition. Right now the incumbents are paying off or suing cities that try to allow it. But look at the cities that got Google fiber. Suddenly, all of the incumbent carriers got so much better...

    9. Re:Common Carrier by Cramer · · Score: 1

      Not "for free". Settlement Free Peering is based on a mostly balanced flow of traffic. The instant that ratio moves from 1:1 to 100:1 (as happened when Netflix switched to their in-house CDN), "free" isn't in the room anymore.

    10. Re:Common Carrier by budgenator · · Score: 1

      An easier solution would be to move speedtest to cogent, speedtest always rocks, no matter how slow the rest of the net is!

      --
      Apocalypse Cancelled, Sorry, No Ticket Refunds
    11. Re:Common Carrier by budgenator · · Score: 1

      True enough, but Joe's is big enough to actually pay the extortion, which most of the colaterally damaged companies using company B's phone line can't; so it's really an example of "When Elephant's fight, the mice get trampled".

      --
      Apocalypse Cancelled, Sorry, No Ticket Refunds
    12. Re:Common Carrier by mysidia · · Score: 2

      Not "for free". Settlement Free Peering is based on a mostly balanced flow of traffic. The instant that ratio moves from 1:1 to 100:1 .... "free" isn't in the room anymore.

      And THIS is what makes them not common carriers; ISPs can do this. In the Telco world, interconnect fees are required to be symmetric, for example: if the agreement is that charges carrier A $0.05 per call record to terminate onto carrier B's network, then it must charge carrier B $0.05 a call to terminate onto carrier A's network, it's not allowed to charge carrier A $0.05 per call and give carrier B free service into carrier A's network. An interconnect agreement cannot be terminated or repriced to favor specific networks, just based on the ratio of calls in or out.

    13. Re:Common Carrier by riverat1 · · Score: 1

      Once again, a call for net neutrality will ensue. All we really need is for the FCC to call them Common Carriers and apply the age old law.

      It has already been applied to Telecoms and Utilities, just apply it to the ISP's and be done with this crap.

      http://en.wikipedia.org/wiki/C...

      Can't mod you up so ... Amen brother! Amen!

    14. Re:Common Carrier by Lisias · · Score: 1

      please mod parent up.

      --
      Lisias@Earth.SolarSystem.OrionArm.MilkyWay.Local.Virgo.Universe.org
    15. Re:Common Carrier by dl_sledding · · Score: 1

      One more reason to move this country to a pure democracy. The republic model worked fine in colonial times because of the logistical nightmares of getting everyone's vote. But now in the digital age all those nightmares are effectively vanquished.

      Vanquished? Really!?

      http://www.csmonitor.com/USA/E...

  5. Their answer to oversubscription as well by Anonymous Coward · · Score: 1, Funny

    Well i get about 10% more than I pay for. Consistently between 102-110% o f the speed i pay. But then I'm not in US.

  6. WHy net neutrality doesn't work by lgw · · Score: 5, Insightful

    Stuff like this is why I think Net Neutrality discussions miss the mark - you're not going to fix the problem that way, you're only going to cause the cable companies to achieve the same throttling through other technical means. Trying to close technical loopholes via the lawmaking process requires a body of law the size of the tax code.

    The fundamental problem is that companies with a legally-granted monopoly for delivering high-speed internet are also allowed to sell content. In a free market, that wouldn't bother me - competition would sort it all out. But "last mile" is about as far from a free market as you can get in most of the country these days, and so we get this as a result.

    Last mile needs to become a public utility. Let vendors compete for my business, and I'll pick "just a pipe" or a content company or whatever mix fits my needs.

    --
    Socialism: a lie told by totalitarians and believed by fools.
    1. Re:WHy net neutrality doesn't work by MozeeToby · · Score: 1

      Stuff like this is why I think Net Neutrality discussions miss the mark - you're not going to fix the problem that way, you're only going to cause the cable companies to achieve the same throttling through other technical means.

      You can make crap like this illegal, in fact it arguably already is without net neutrality legislation.

    2. Re:WHy net neutrality doesn't work by Anonymous Coward · · Score: 2, Insightful

      "Last mile needs to become a public utility. Let vendors compete for my business, and I'll pick "just a pipe" or a content company or whatever mix fits my needs."

      I think you have that backward.

      The backbone should be a public utility. The gov't should assume the responsibility for the massive infrastructure required to build the backbone. Then its easy for any small mom/pop ISP to connect to the public backbone for a nominal fee and provide last mile service to their area when they are unhappy with the current last mile providers as it requires orders of magnitude smaller investments in infrastructure.

      Slashdot required car analogy, The Fed Gov't built the interstate system, the local gov't built/maintains the local roads and all the mom/pop stores are welcome to build brick n mortar business and connect to those roads for nominal fees. Meanwhile private parties are also welcome to build special roads/infrastructure at there own cost to serve special needs, and the public is sometimes welcome to use those items at additional cost EG: Toll roads/bridges/tunnels/ferries

    3. Re:WHy net neutrality doesn't work by alen · · Score: 1

      there is no legal monopoly, the cost is too high.

      FIOS only has on average 40% of potential customers in the markets they serve. it's impossible to borrow enough money to build out a last mile network and make a profit after you account for advertising and operational costs

    4. Re:WHy net neutrality doesn't work by sabri · · Score: 1

      The gov't should assume the responsibility for the massive infrastructure required to build the backbone.

      Ah yes, with fibertaps everywhere.

      No thanks.

      --
      I'm not a complete idiot... Some parts are missing.
    5. Re:WHy net neutrality doesn't work by nine-times · · Score: 4, Insightful

      The fundamental problem is that companies with a legally-granted monopoly for delivering high-speed internet are also allowed to sell content.

      I agree with this part of your post, at least, and have been making the same argument for years. If the companies providing the infrastructure were not making money from selling content, and were only serving as "dumb pipes", then their business incentive would be in pushing customers toward higher-bandwidth (and therefore more expensive) connections. In that business model, companies that can provide content to saturate slow connections become very important, and so it seems likely that they would be falling all over themselves to provide a better connection to Netflix.

      Instead, the Infrastructure providers have no incentive to increase content availability, because any piece of available content becomes competition for the content that they are trying to sell. That's a bad system. Unless you have an effective regulatory system, the ISPs will find ways to push towards a walled garden AOL-style internet, charging for access outside of the walled garden.

      However, I don't think this is an example of "net neutrality" missing the mark. Net neutrality is a concept, and divorcing infrastructure providers from content providers is one way in which net neutrality could be promoted.

    6. Re:WHy net neutrality doesn't work by nehumanuscrede · · Score: 5, Interesting

      So the fix sounds easy, but gets complicated quickly. :|

      Break them up. Don't think about it, threaten or consider it. To borrow a phrase from Nike: JUST DO IT

      Force the companies who provide the bandwidth to split from the part of the company who sells content so there isn't any grey area issues. In order for the bandwidth side of the house to remain competitive, they'll need to upgrade their network to ensure they're delivering what their customers want. This also prevents the content folks from tweaking the network to ensure their own services get priority over competing ones. Taken to an extreme, and without some sort of oversight, the big players can absolutely destroy the competition in this manner.

      They've already proven they can't be trusted to do the right thing themselves. Time to step in with a big stick and start swinging.

      Of course, true competition needs to be in place to help fuel that fire otherwise we'll still end up with crappy throughput with no realistic choices of switching to another provider. We really need to end this whole regional mono / duopoly thing.

      On top of this, we'll need to purge the Congressional votes-for-contributions types to make sure the legislation has a fair chance of happening.

    7. Re:WHy net neutrality doesn't work by Gr8Apes · · Score: 1

      I think you have that backward.

      The backbone should be a public utility. The gov't should assume the responsibility for the massive infrastructure required to build the backbone.

      The last mile is the most expensive part of the entire network. Backbones have few pricier components. Every connection requires a port on a switch, etc.

      --
      The cesspool just got a check and balance.
    8. Re:WHy net neutrality doesn't work by fustakrakich · · Score: 2

      Last mile needs to become a public utility.

      "Reform this! Reform that!"

      Everybody keeps saying that like it's going to magically happen by some miracle.You think the politicians currently in office are going to let that happen? Especially when they have 95% chance of being reelected no matter what they do? When voting, know who/what your politician is representing, and then decide if you want the same thing. Their records are easy to find now, and you can stop believing the lies they tell you on the TV. When I see a big turnover (get rid of the entire institutional republican/democrat party), then I'll believe people are serious. But everybody please, quit crying about how the system is rigged. It is a social/psychological exploit, that needs to be patched on the individual level, like any other system update. This patch however, requires a firmware upgrade.

      --
      “He’s not deformed, he’s just drunk!”
    9. Re:WHy net neutrality doesn't work by lgw · · Score: 2

      Well, this is rigged at the local level, so it's possible to fix it at the state or federal level. Local government is often more responsive, but when it's corrupt it generally can't be fixed at that level. I love to see a state step up and do this - force "last mile" into a utility wherever it locally has legally-granted monopoly status. State governments can also be fairly responsive, at least when issues don't touch car dealerships.

      --
      Socialism: a lie told by totalitarians and believed by fools.
    10. Re:WHy net neutrality doesn't work by fustakrakich · · Score: 2

      It doesn't matter what level you're at. The politician represents his financiers, or he receives no financing. Our obligation is to seek out those that aren't on the organ grinder's leash between elections so we have somebody to put on the ballot. That's how they do it, and it shows.

      --
      “He’s not deformed, he’s just drunk!”
    11. Re:WHy net neutrality doesn't work by LessThanObvious · · Score: 1

      I'd be basically OK with it if they were allowed to charge the peer for the (actual and reasonable) cost of the upgrade and no additional fees were charged, even though they would be willing to meet half way were it not in competitive relationship. We'd almost need to force them to split their businesses into separate walled off cable TV and Internet businesses to end the large incentive to squelch competition for eyeballs on the screen. If they offer better peering deals to some transit providers and worse to another solely in order to screw their competition I'd have to believe there is some Antitrust legal provision being violated. Ever since cable speeds beat out common DSL speeds we have been screwed by lack of competition.

    12. Re:WHy net neutrality doesn't work by AK+Marc · · Score: 1

      Nope. The anti-trust cases have ended up (successfully) arguing that a monopoly isn't a monopoly, if you have choice. You can choose to move states and get different choices, so you aren't "locked in".

    13. Re:WHy net neutrality doesn't work by tepples · · Score: 1

      You can choose to move states and get different choices

      Which court in which case used this as an argument in its opinion to dismiss an antitrust action?

    14. Re:WHy net neutrality doesn't work by TangoMargarine · · Score: 1

      You can choose to move states and get different choices, so you aren't "locked in".

      By that logic, we could have a single company providing a service to the entire world minus one podunk town in the middle of Siberia (directly financed by Putin probably) and it "wouldn't be a monopoly."

      Fuck all you guys with your contortions to redefine the argument to "prove" yourself correct.

      --
      Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
    15. Re: WHy net neutrality doesn't work by lgw · · Score: 1

      I believe that's how power works in Texas as well. It's a good pattern. Sadly, we have a problem in the US with ideas that make too much sense.

      --
      Socialism: a lie told by totalitarians and believed by fools.
  7. How many engineers does it take to screw netflix? by deathcloset · · Score: 1

    OK, so people had to implement this blocking, right?

    And it had to have been more than one person, right?

    How many individuals would have to have been active and knowingly involved in order to implement such blocking?

    From that number, how long until someone straight-up comes out and says they did it and exactly how they did it? You know, instead of having to rely on this external third-party reverse engineering.

  8. Yes it is a peering problem ... by jschultz410 · · Score: 4, Informative

    and not a net neutrality issue thankfully.

    Settlement free peering between tier 1 carriers only happens when the flow of traffic is roughly balanced between the contracting peers.

    When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network. Otherwise, networks would be monetarily incentivized to unload traffic they should carry on their own networks onto their peers' instead.

    1. Re:Yes it is a peering problem ... by 0123456 · · Score: 1, Funny

      I'm sorry, but your post is too sensible and informative for today's Slashdot, where technological knowledge takes a back seat to 'stick it to the man!' outrage.

    2. Re:Yes it is a peering problem ... by Burdell · · Score: 4, Informative

      Traffic balance is not the primary measure these days (from what I understand), it is just an economic decision. However, the Netflix case is interesting, because they were essentially used as a leverage tool by Cogent against the other carriers. Cogent has a long history of trying to get settlement-free peering, not meeting contract terms (whatever they are), getting dropped, and then blaming the other side. They have long wanted to be a settlement-free "tier 1" provider (which is a nebulous term, but go with it), but have generally not been. They sell bandwidth often at below-market rates in order to attract customers to leverage against the other "tier 1" providers. They saw Netflix on the rise and grabbed them, apparently selling bandwidth much cheaper than any other backbone (possibly at a loss even) in order to leverage settlement-free peering contracts out of other providers.

      Any network engineering with a clue knows that you never buy bandwidth only from Cogent (or even Cogent and one other provider), because you _will_ get disconnected from somebody when Cogent gets in another peering dispute.

    3. Re:Yes it is a peering problem ... by PPH · · Score: 2, Insightful

      When one peer is pushing a lot more traffic onto the other network,

      Simple fix: A Netflix client that echos the content back to the source server.

      Problem fixed.

      --
      Have gnu, will travel.
    4. Re:Yes it is a peering problem ... by Anonymous Coward · · Score: 5, Insightful

      When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network.

      They're not pushing the traffic - the other network is pulling it. Netflix's traffic is not unsolicited - every goddamned packet is being sent in response to a specific request from the other network's customers, and it's not fucking transit - the Netflix packets will terminate within the receiving network. Are you seriously arguing that Comcast should be paid by Netflix because they're carrying gigabytes of Netflix traffic their own fucking customers requested?

    5. Re:Yes it is a peering problem ... by waldozer · · Score: 3, Insightful

      I think you have it backwards. Netflix does not "push data" to Comcast. Comcast customers "pull data from Netflix".

    6. Re:Yes it is a peering problem ... by Anonymous Coward · · Score: 2, Informative

      Sensible?? More like nonsensical and completely misses the point of the article. As a great many people have pointed out, this is not unsolicited traffic, but traffic requested by Comcast's customer. So where on earth do you get the idea that

      "When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network."

      has ANY relevance to this story? Given what has been reported as the "trivial" costs of upgrading the interconnection points, by what other argument do you think the ISP's are in the right? Given that they are being paid by their customer's for internet access?

    7. Re:Yes it is a peering problem ... by AnotherBlackHat · · Score: 1

      Settlement free peering between tier 1 carriers only happens when the flow of traffic is roughly balanced between the contracting peers.

      When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network.

      So you're saying if Netflix downloaded more data from Comcast than they sent, that Comcast should pay them?

      I have no problem uploading an amount equal to what I download from Netflix, or even more, if you really think that will solve the problem. I don't really control the software on my Roku box, but I don't mind if Netflix puts some P2P software on it for help carrying their own traffic.

    8. Re:Yes it is a peering problem ... by RelaxedTension · · Score: 2

      Got it, but where does the fact that the traffic has been requested by the users the target network play into it? The more appropriate term here is "puller" as opposed to "pusher". The traffic would not be there except for the end network requesting it in the first place.

    9. Re:Yes it is a peering problem ... by Obfuscant · · Score: 2

      Netflix does not "push data" to Comcast. Comcast customers "pull data from Netflix".

      It doesn't matter. There is an imbalance of the data flow through the peering connection.

      Comcast doesn't guarantee data rates to off-network content providers. It can't, or you'd be paying even more outrageous rates for service from them.

      Here's why. Imagine Comcast has 2 million subs. I don't know the numbers, but let's use that for argument. They each have 50Mbps service. That means that EVERY PEER CONNECTION that Comcast has would have to be at least 100,000,000 Mbps to guarantee full rate to every sub. That's 100 TERAbps. Why? Because all 2 million might want to connect to the same data provider at the same time. Very slim chance, but a guarantee is a guarantee, and we see how loudly people complain when implied promises aren't kept.

      Service providers have ALWAYS used statistical methods to determine the most cost effective amount of hardware to meet the anticipated demand for service. You don't think a phone company with 100 users had 100 long distance trunks running from the central office to the nearest long distance center, do you? (Mandatory auto analogy: a city with 1000 residents with one car each doesn't have 1000-lane roads everywhere, does it? And a restaurant with 100 seats doesn't have 100 valets parking cars, does it?)

      So now ISPs are doing the same thing. And every one of them does it. Where I work, I have gigabit to the desktop. There's about 400 of us in this college alone. Do you think we have a 400Gb line to the Uni? And does the Uni multiply 1 gigabit times the number of employees/students to figure out how much bandwidth they have to buy from their provider? Of course not. We have a /16 address block, so ignoring the large number of unroutable addresses we already use, that's 64,000Gbps, if the Uni provided a full-time full-rate connection at 1Gbps to every address.

      This article is FUD and nonsense. Comcast isn't jamming Netflix. In fact, this article pretty much proves that Comcast is doing nothing special to Netflix, they're just not upgrading their peer connection as fasts as people want them to. Why? Because it costs money to do that.

      Should they upgrade? They can't make everyone happy. They didn't promise you 24/7 full-rate access to anyplace off their net, and I don't think they did that even ON their net. It's not illegal not to upgrade. It's not even a net neutrality issue, because, as this article proved, the congestion impacts EVERY service the same.

    10. Re:Yes it is a peering problem ... by Em+Adespoton · · Score: 1

      and not a net neutrality issue thankfully.
      Settlement free peering between tier 1 carriers only happens when the flow of traffic is roughly balanced between the contracting peers.
      When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network. Otherwise, networks would be monetarily incentivized to unload traffic they should carry on their own networks onto their peers' instead.

      What you're stating is the situation for transit peering. This issue has nothing to do with transit peering, as the packets all terminate inside the receiving network.

      See, this is why it's a problem that we've got the same people providing Tier 1 trunk lines that are providing endpoint connects. It creates another class of peer where they're being paid by end users for transit to the interconnect, and then they turn around and want to charge peers for transit to the end users. That's called double dipping, and that's why there's an issue here. The entire point of peering with these Tier 1-to-endpoint providers is to unload traffic; there's no other way to actually route traffic to the destination!

    11. Re:Yes it is a peering problem ... by suutar · · Score: 1

      Except that the recipient (me) is already paying Comcast to deliver the bits. If they want to go to sender pays, then I get to start charging them for traffic that they send to my house.

    12. Re:Yes it is a peering problem ... by torkus · · Score: 1

      Erm...but netflix will drop a cache server in the ISP's datacenter, configure it, and maintain it. Oh, and it's free. The ISP saves on bandwidth/interconnect at the cost of a few U of rackspace and a couple bucks in power/cooling.

      But then why would they comcast or TW want to do something to help their competition even if it also benefits their customers. Monopoly and conflict of interest. Good job politicians.

      --
      You can get rich if you own a politician, but you have to be rich to buy one in the first place.
    13. Re:Yes it is a peering problem ... by Obfuscant · · Score: 1

      So you're saying if Netflix downloaded more data from Comcast than they sent, that Comcast should pay them?

      If Comcast was the source of the data that was creating congestion on a Netflix-to-peer link, I'd say yes, I think Comcast should have a part in paying for the increased bandwidth required to carry their data.

      I have no problem uploading an amount equal to what I download from Netflix, or even more, if you really think that will solve the problem.

      Who said that would solve the problem? All it would do is make Netflix less responsible for paying for the upgrade, not that the upgrade would happen.

      There are two problems here. The first is a limited bandwidth connection is being congested (and all connections are limited in some way.) The second problem is figuring out who should pay for the upgrade -- the company who is profiting from sending the data, or the company whose customers are assuming that a maximum data rate through their "last mile" means they should get the same rate to any site anywhere on the planet?

    14. Re:Yes it is a peering problem ... by mythosaz · · Score: 2

      Yes, and Netflix pays their direct service providers for their bandwidth.

      When bandwidth leaves their direct provides and reaches, say, Comcast, where Comcast's customers have paid for internet service as well, Comcast (and others) refuse to deliver that data, because they're only getting paid to do so at least twice.

    15. Re:Yes it is a peering problem ... by MozeeToby · · Score: 1

      I agree, why should Cogent need to supply all this throughput that Comcast's paying customers are asking for!? Waaaaaiit a minute....

    16. Re:Yes it is a peering problem ... by NormalVisual · · Score: 1

      You can't just ignore the entire OSI model like that, and the fact there's an application level request being sent from Comcast to Netflix for the traffic received DOES most definitely factor into it. That a layer 7 request/response is being implemented by means of layer 3 activity doesn't change the fact that the layer 7 request from Comcast was the ONLY reason that the subsequent Netflix traffic was generated and received.

      The fact is that traditional peering simply doesn't work when dealing with hugely asymmetrical networks like the average ISP. Comcast isn't transiting the Netflix traffic to another network, they're routing it directly to a Comcast end customer. IMO this wouldn't be an issue if Netflix weren't competing with Comcast's own content offerings.

      --
      Please stand clear of the doors, por favor mantenganse alejado de las puertas
    17. Re:Yes it is a peering problem ... by Xipher · · Score: 1

      Eyeball networks didn't usually get settlement free peering to begin with. Until you had these huge eyeballs form like Comcast that kind of peering was between the transit ISPs themselves. Comcast used to be a customer of these ISPs, and didn't get the peering agreements until they started congesting links by dropping transit services. I found information on this discussed on the NANOG mailing list from 2010, so this has been going on for a number of years already.

      --
      I don't know everything.
    18. Re:Yes it is a peering problem ... by Obfuscant · · Score: 4, Informative

      Why does imbalance matter?

      Because when peering agreements were created, the assumption was that trying to keep track of how much data you wanted us to carry and you keeping track of how much we gave you to carry was not necessary because we'd be charging each other the same amount if we did keep track.

      Large amounts of data going one way breaks that basic tenet of peering. Now it makes sense to charge the other guy for data they want you to handle.

      But this view ignores the most important point, that Comcast has explicitly promised its customers "internet access" at an advertised speed.

      No. From here:

      Performance Starter: Offer ends 01/04/15. Restrictions apply. Not available in all areas. Limited to new residential customers. Requires subscription to Performance Starter Internet service. Equipment, installation, taxes and fees, including regulatory recovery fees, Broadcast TV Fee (up to $3.50/mo.), Regional Sports Fee (up to $1/mo.) and other applicable charges extra, and subject to change during and after the promotion. After applicable promotional period, regular rates apply. Comcastâ(TM)s current monthly service charge for Performance Starter Internet is $49.95 (pricing subject to change). Service limited to a single outlet. May not be combined with other offers. Actual speeds vary and are not guaranteed.

      Emphasis mine. The same emphasized text appears in the details for all three residential service levels. My statement stands: they did not promise you 24/7 full-rate access to anyplace off their net.

      There are no pharmaceutical-like disclaimers during those commercials

      There are when you actually go to sign up. And common sense tells you that they cannot guarantee those speeds to every site on the planet. They can't even guarantee those speeds to every site on the Comcast network. That's why they don't.

      If Comcast says you're paying for "10Mbps internet", the assumption is that you get the advertised speed to the entire internet, provided there are no technical limitations outside of Comcast's control.

      That's what some people assume, but that's not backed up by the service agreement.

      It's not even backed up by common sense. Suppose you buy the Blast service and get 105Mbps download. You want to connect to my system and I've got Performance Starter (6 Mbps down, God knows what it is up). You ain't getting anywhere near 105Mbps from my stream. Even trying to connect to your next door neighbor who has the same service, you ain't getting faster than his upload allows. If you think Comcast could promise anything faster, then you must think they'll upgrade MY service to Blast for free because they promised YOU that you'd get data from me that fast, and you're paying them for my data at that speed.

    19. Re:Yes it is a peering problem ... by Obfuscant · · Score: 1

      What you say is nice and logical, as long as you leave out the part where Netflix's alternatives (such as dropping a content box inside Comcast's network to solve the problem) were ignored by Comcast.

      Well, I see this claim here a lot, but I also see things from other people that say that Comcast didn't ignore the offer, it was that Netflix wanted hosting without paying for it like the other CDNs do. And I'm not sure why Netflix should get hosted on Comcast's network without paying for it... after all, Netflix is making a profit selling the data and they'd have to pay any other network provider for the connection.

    20. Re:Yes it is a peering problem ... by AK+Marc · · Score: 1

      When one peer is pushing a lot more traffic onto the other network, then that usually goes out the window and the pusher is required to pay the receiving network.

      When did this change? In the Early Days, the puller paid, not the pusher. Hosting servers in datacenters was almost free, as upload bandwidth was "free" to the ISP. I pay my ISP to get the Internet. If their connection to the content I want isn't good enough, my ISP, not the other side, should pay to fix it. Puller pays. Otherwise, my ISP should be paying me to be their customer for generating demand that they make money from.

    21. Re:Yes it is a peering problem ... by AK+Marc · · Score: 1

      So there's no session information in a packet? TCP is stateless? I'm going to have to find some brain bleach to fix all my bad info. Thanks A/C. I thought TCP was stateful and that packets can contain information that pertains to port sequence and other things that demonstrate a life beyond that single packet.

    22. Re:Yes it is a peering problem ... by tepples · · Score: 1

      My statement stands: they did not promise you 24/7 full-rate access to anyplace off their net.

      So why don't ISPs accept Netflix's offer to provide a caching server without charge that keeps the most popular traffic on their net?

    23. Re:Yes it is a peering problem ... by Solandri · · Score: 1

      It is not a peering problem. Netflix provides streaming servers to ISPs for free so they can stream Netflix videos locally, and completely avoid the bandwidth bottleneck of going through their peering networks. Comcast, Verizon, et al refused to accept these free servers, to artificially degrade Netflix's quality specifically so they could extort money from Netflix.

      It is a monopoly problem, plain and simple. If Comcast, Verizon, et al had had competition, this tactic would've caused them to hemorrhage customers who would've fled to an ISP which actually provided decent service. But because they have a monopoly, their customers have nowhere to flee, and this extortion tactic works

    24. Re:Yes it is a peering problem ... by perryizgr8 · · Score: 1

      Is this serious? kbps? per-minute charges? ISDN? huh?

      --
      Wealth is the gift that keeps on giving.
    25. Re:Yes it is a peering problem ... by mgcarley · · Score: 1

      When they are running their links (peered or not) at 100% for 24 hours a day, you bet your ass they should upgrade. ESPECIALLY if someone else offers to foot the bill.

      Will it be a permanent fix as bandwidth demands increase? Hell no, but that's what operational expenses are.

      In some countries, if an ISP's links are running at more than a certain percentage of their capacity for more than a certain percentage of the day and they are doing nothing about it, they would be found to be falling afoul of the regulations and could very well be fined. Additionally, in some countries if an ISPs contention ratio gets too high, that's also going to cause an issue with the regulator (even though it's a bullshit measurement for so many reasons).

      If US ISPs here were told they couldn't have a ratio of more than 50:1 (1mbit/s per 50 mbit/s sold) it might help - you get sold a 50mbit/s line, so your ISP has to have at least 1mbit/s of bandwidth at the border.

      For example: 2 million subscribers each with 50mbit/s means only 2 million mbit/s (2 terabits) at the network border, which is very doable - even a pair of Juniper T1600s will do that - and that's a 3 or 4 year old model. An ISP with that many subscribers should definitely be present at more than 1 IXP, so if we assume 8-10, you're only looking at 200-250gbit/s in each.

      Comcast in particular already announces that it has traffic levels exceeding 1Tbit/s, so they surely already have the equipment to do it (just not the incentive, I guess).

      --
      Founder & COO, Hayai India (hayai.in) / USA (hayaibroadband.com) // t: @mgcarley
    26. Re:Yes it is a peering problem ... by mgcarley · · Score: 1

      Now *this* post, actually makes sense and I fully agree with it. We have to explain this to people //all//the//time//.

      --
      Founder & COO, Hayai India (hayai.in) / USA (hayaibroadband.com) // t: @mgcarley
  9. Re:Netflix, in the parlor, with the fireplace poke by Anonymous Coward · · Score: 3, Insightful

    Yeah, I'm sure netflix just goes around dumping truckloads of data on the information superhighway just at random, and picked on Comcast like a bully.

    Oh wait, every one of those streams were requested from users of Comcast's network who thought that those awesome 150mbit internet speeds comcast advertised were real.

  10. Net neutrality is a constituional right! by Anonymous Coward · · Score: 5, Funny

    I should be able to use my bandwidth any way I please! It's freedom of speech! They can't throttle netflix! George Bush would be rolling in his grave!

    This is a ploy from Obama to help spread misinformation about ebola (Obola) on his path to further to destroy the country!!1!

  11. Duh! by gstoddart · · Score: 1

    The report proves the problem is not technical, but rather a result of business decisions.

    Is anybody actually surprised by this?

    This has always been about maximizing profits, and preventing a competitor from gaining access to your customers.

    Because cable companies are ran by assholes.

    --
    Lost at C:>. Found at C.
  12. Re:How many engineers does it take to screw netfli by MozeeToby · · Score: 5, Insightful

    Engineer - "Hey Boss, we need some cash to upgrade the connection to these networks."
    Boss - "What?! We just upgraded those connections a couple years ago"
    Engineer - *rolls eyes* "Well the link is saturated, looks like lots of people watching online video... Netflix comes in over this connection so it makes sense"
    Boss - "First they take our subscribers now they're forcing us to upgrade our equipment... well fuck em!"
    Engineer - "Waaah?"
    Boss - "You heard me, fuck em!"
    Engineer - "But... our customers will get terrible service when they try to watch Netflix, or do anything else on that network for that matter"
    Boss - "Exactly!"

  13. Re:Netflix via sneakernet by Anonymous Coward · · Score: 1

    Receive, rip, and return!

  14. Too bad by Anonymous Coward · · Score: 1

    the hipsteriffic site is unreadable to me.

  15. Re:Netflix, in the parlor, with the fireplace poke by putaro · · Score: 2

    The traffic isn't transiting Comcast going to another network. It's going to a Comcast subscriber who wants to watch a movie. So, yes, the subscriber is requesting a movie and the data is being delivered to them. There's no other route to the subscriber than through their ISP.

  16. Re:Netflix, in the parlor, with the fireplace poke by Anonymous Coward · · Score: 1

    OMG, you are so right, and those request HTTP packets that started the whole SEND SEND SEND also came from the people seeking "free dumping rights". Damn, I wish I was as smart as you with regards to layer3 and TCP Connections.

  17. Multiple CDN contracts? by CaptSlaq · · Score: 1
    I assume there's a reason why Netflix hasn't perused the idea of load balancing across CDNs? Yes, it'd be a pain in the ass, but I know both Akamai and Limelight will read from your source to deliver bits to an end user.

    It'd be a hell of a lot better than buckling to ISPs. At least you're in control of your costs at that point.

    1. Re: Multiple CDN contracts? by bill_mcgonigle · · Score: 5, Informative

      Netflix is its own CDN - they will give, for free, one or more caches to any ISP, causing any one movie to transit the ISP's nonfree network connections only once.

      But this is about competition for video services, not caching.

      --
      My God, it's Full of Source!
      OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
    2. Re:Multiple CDN contracts? by Talderas · · Score: 1

      They have but they didn't want to pay anything close to the costs of what Akamai charges or anything close to the costs Akamai has to host their CDN.

      Admittedly, they may have some weird stipulations going on with copyright and all that jazz.

      --
      "Lack of speed can be overcome. In the worst case by patience." --Znork
    3. Re: Multiple CDN contracts? by killfixx · · Score: 5, Insightful

      This is a very salient point. Netflix already has these arrangements with other ISPs. Only Comcast, AT&T, and Verizon (surprise surprise) refused to host local caching servers. Of course, this precedes their demands for more money because, "Waaaahhhh...they're stealing our customers, they need to pay!".

      Netflix tried to be the better entity (within reason) and were told, in no uncertain terms, "Go fuck yourself."

      Yay, free market!

      *sigh*

      --
      "Helping to keep you two steps ahead of the Thought Police!"
    4. Re: Multiple CDN contracts? by alen · · Score: 2

      except CDN's have paid ISP's for years for hosting costs. netflix refuses to pay and wants their CDN hosted for free

    5. Re:Multiple CDN contracts? by PRMan · · Score: 2

      All kinds of wrong. They will provide a server box to be housed in the ISP containing their most popular content FOR FREE.

      --
      Peter predicted that you would "deliberately forget" creation 2000 years ago...
    6. Re: Multiple CDN contracts? by Anonymous Coward · · Score: 2, Interesting

      This is complete nonsense. Google and Akamai both place hardware just inside ISP borders at no cost. They don't pay the ISPs to do this, the ISPs are saving a shit-ton on transit and are happy to set aside a little space and electricity for the hardware. This includes Comcast. Try doing a traceroute to 23.79.61.240 for example, this is one of at least a dozen Akamai servers sitting in a Comcast cage in Atlanta. Akamai is paying zero dollars for that.

      Netflix offered the same arrangement and Comcast said no, the only reason being that Netflix sells a competing product to Comcast's own video on demand. It's anticompetitive behavior, period, plain and simple.

    7. Re: Multiple CDN contracts? by Cramer · · Score: 1

      So I should be able to demand Verizon install ("host") my server(s) for free as well? Not going to happen.

      Netflix is a FOR PROFIT BUSINESS. They can pay for services just like everybody else. (speaking as Verizon) Why should I bear the cost of hosting their business? It isn't costing me customers. And I'm sure as hell not going to give those asshats at Cogent anything; they're being paid boatloads by Netflix but won't buy the interconnects to support 'em.

      Yes, there are small(ish) ISPs hopping on the Open Connect bandwagon. For them, it's a cost effective solution vs. the alternatives -- lost customers, or additional expensive bandwidth. Verizon (et. al) simply aren't going to play those games: Cogent can buy the bandwidth necessary to support their customers, or Netflix can find a different (preferably direct) path.

    8. Re: Multiple CDN contracts? by jcam2 · · Score: 1

      1 - What evidence do you have for how much Comcast charges Google or Akamai for placing hardware at their facilities?

      2 - Google is just as much a competitor to Comcast via Youtube and the Google Play movies store. Why is only Netflix being targeted?

    9. Re: Multiple CDN contracts? by tepples · · Score: 1

      Google is just as much a competitor to Comcast via Youtube

      Since when does Comcast offer a platform for amateurs and small-time professionals to publish their videos? I thought Comcast was for the Disneys, Scrippses, and Discoverys of the media world.

  18. This is kind of relevant... by Svartalf · · Score: 5, Funny
    --
    I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
  19. WHy net neutrality doesn't work by KingMotley · · Score: 1

    You are totally correct here. ISPs should only be allowed to be content producers, or content distributors, IF they relinquish all their monopoly statuses with local municipalities. Comcast, Time Warner, etc should be taken to court under anti-monopoly laws in the US. As they are guaranteed monopolies and their behavior is definitely harming consumers and they are trying to leverage their monopoly in one sector to give them an unfair advantage in a different sector, this seems a rather simple case, but well... lobbying... money... corruption... self-serving politicians... yeah.

  20. Re:How many engineers does it take to screw netfli by Jason+Levine · · Score: 2

    Replace the Boss' last line with:

    Boss: "Oh, boo hoo! What're they gonna do? Leave us and go to another ISP? We've got a monopoly in the area! They want Internet? They need to come to us. Besides, if we make Netflix look bad, maybe people will dump them and pay us $200 a month for cable again!"

    --
    My sci-fi novel, Ghost Thief, is now available from Amazon.com.
  21. How is this not a neutrality issue? by Rujiel · · Score: 1

    Regardless of the mechanism for throttling traffic, Comcast's customers are being disallowed access to a certain network because Comcast allows congestion only for traffic to those sites. Comcast and Verizon have a direct conflict of interest, providing services similar to those of competitors that they are throttling.
    The idea that there is a bandwidth "shortage" caused by services like Netflix is fucking laughable, and a total spoon-fed excuse to rob paying customers of the service they deserve.

    1. Re:How is this not a neutrality issue? by Obfuscant · · Score: 1

      Regardless of the mechanism for throttling traffic, Comcast's customers are being disallowed access to a certain network because Comcast allows congestion only for traffic to those sites.

      You didn't even read the summary, did you?

      And the effects victimize not only companies targeted but ALL incoming traffic from the affected transit network.

      It isn't congestion only to Netflix sites, it's every site that routes through the congested peer connection. And it isn't "disallowed", it's only slowed down.

      and a total spoon-fed excuse to rob paying customers of the service they deserve.

      "Deserve" is a very subjective term. What did they pay for? And if you say "100MBps (or any specific number) from every service anywhere in the world to their client", you're wrong.

      The fact that the issue is congestion at a peering point means it is a technical problem. Who pays to upgrade that connection is a business decision, and there is more than one party involved in that game. Should the service that is being paid for the content and sending large amounts of data pay, or should the company that hasn't promised full-time full-rate data anywhere off-net have to pay when their customers demand full-time full-rate data streams from off-net? Is that fair to the customers who haven't asked for that level of service?

    2. Re:How is this not a neutrality issue? by tepples · · Score: 1

      Who pays to upgrade that connection is a business decision

      Cogent offered half a year ago to pay the actual costs of the upgrade. The ISPs in question appear to want to extract a markup on top of that.

    3. Re:How is this not a neutrality issue? by Obfuscant · · Score: 1
      The relevant bit of your link says:

      To be clear, Cogent is not offering to enter into paid peering arrangements with these or any other networks. Rather, Cogent is simply willing, at this time, to incur the capital costs associated with augmenting its interconnections with these networks to address the current level of traffic congestion.

      So Cogent isn't offering peering to anyone that doesn't have it, and is paying only the capital costs, not the ongoing fees. It's also only covering the "current level" of congestion.

      To use an amazingly appropriate analogy, that would be like Comcast offering to provide you free installation and a free digital decoder so it could charge you another $40/month for enhanced television service. Sounds like a great deal, Comcast paying the capital costs of your upgrade, huh?

    4. Re:How is this not a neutrality issue? by tepples · · Score: 1

      What "ongoing fees" are just and why?

    5. Re:How is this not a neutrality issue? by Obfuscant · · Score: 1

      You're either a troll, kidding, or just don't know what "operating expenses" are.

    6. Re:How is this not a neutrality issue? by tepples · · Score: 1

      Then please explain what increased ongoing expenses Comcast would incur by upgrading its link with Cogent.

  22. Comcast fanboy detected by Rujiel · · Score: 1

    Wait, did I say comcast fanboy? I forgot, those don't exist. Guess I should have said:

    Paid shill detected

  23. Re:How many engineers does it take to screw netfli by operagost · · Score: 1

    FFFT

    *rubs nipples*

    --

    Gamingmuseum.com: Give your 3D accelerator a rest.
  24. Another misleading headline by jbmartin6 · · Score: 1

    On the summary, not the article. Comcast wasn't "jamming' Netflix. Jamming is an active response. What Comcast did was nothing. Now whether they should have done something about the overflowing links is the argument, but that's a far cry from "jamming."

    --
    This posting is provided 'AS IS' without warranty of any kind, implied or otherwise.
    1. Re:Another misleading headline by jbmartin6 · · Score: 1

      Being snide isn't useful. Since Netflix/Cogent would be the one jamming traffic into the pipe, not Comcast, my point still stands. One cannot "jam" something by any of these definitions by doing nothing. The jamming is done by something or someone else.

      --
      This posting is provided 'AS IS' without warranty of any kind, implied or otherwise.
  25. Re:How many engineers does it take to screw netfli by torkus · · Score: 5, Insightful

    Actually netflix offered to foot the bill for upgrading the bandwidth - it's literally a couple cross-connects in a datacenter, maybe a fiber card or two.

    Oh, and netflix ALSO offers to drop a server in your datacenter *free* which caches all the common netflix streams. This reduces the internet bandwidth demands by something like 90+% since it lives within the ISP's datacenter and just needs to download each stream once.

    But the last line is exactly the point. The ISPs are also TV providers and they don't want you to have a good netflix experience. If they can passively let that happen...well of course they will. No one can accuse them of taking any action to damage your netflix streaming...it's their complete inaction that's resulting in it.

    --
    You can get rich if you own a politician, but you have to be rich to buy one in the first place.
  26. Techdirt Article on Same Story by carrier+lost · · Score: 2

    "Similarly, M-Lab's data shows that the problem wasn't a lack of bandwidth on the part of the ISPs (i.e., no actual technical congestion), because those same ISPs had no problem connecting to a different transit provider, Internap. So the only logical culprit was the interconnection points. There was more than enough bandwidth to go around. There's just the single bottleneck of the interconnection border router (which, again, is trivially simple to get rid of by opening up more ports)."

    link

    1. Re:Techdirt Article on Same Story by Svartalf · · Score: 1

      Basically, it was a pissing match like people claim it was. Something that USED to be called out on the carpet over- because it's violating common carrier status that the jokers in question all have and alternately want and don't want. (They don't want the regulation, but they want the shield from vicarious liability from their customers' actions...)

      --
      I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
    2. Re:Techdirt Article on Same Story by carrier+lost · · Score: 1

      Ha, your sig

      I keep telling my GF I don't want to be a citizen of California (we live in Nevada)

  27. Re:Netflix, in the parlor, with the fireplace poke by Obfuscant · · Score: 2

    Oh wait, every one of those streams were requested from users of Comcast's network who thought that those awesome 150mbit internet speeds comcast advertised were real.

    150mbps isn't very fast. Did you mean 150Mbps?

    The issue is that people think the 150Mbps "last mile" speed was some kind of promise of speed to every other site on the planet. I can request a data stream from a service that needs 100Mbps to function reliably, but that doesn't mean I'm guaranteed 100Mbps from end to end. If an ISP promises you that, you know they're lying. If you assume the last mile speed applies to every connection you make anywhere, then you're the one who's being foolish, not the ISP being dishonest.

  28. Re:Why rely on peering? by robbyb20 · · Score: 1

    If I read correctly, Netflix is hosted with Amazon. So its Amazons connection that needs to be looked at, no? I could be completely wrong as well, so not really pushing this.

    http://en.wikipedia.org/wiki/N...

  29. All this amounts to.... by arbiter1 · · Score: 1

    This is nothing more then Anti-competitive practices against competing products that are cheaper then theirs.

  30. End the ISP monopolies by Jodka · · Score: 4, Interesting

    from wikipedia

    Franchise fees are governed under Section 622 of the Cable Communications Act of 1984.[2] Section 622, states that municipalities are entitled to a maximum of 5% of gross revenues derived from the operation of the cable system for the provision of cable services such as Public, educational, and government access (PEG) TV channels.

    Franchise fees are fixed at a maximum of 5% of gross revenues. So how do municipalities maximize revenues from franchise fees? By maximizing cable company gross revenues. And how do municipalities maximize cable company gross revenues? By creating monopolies! By awarding exclusive license to one provider to extract monopolist profits from the public.

    Note that there is nothing inherently wrong with permitting local governments to charge cable companies fees. That is justifiable to the extent that local governments incur costs of infrastructure repair with damage from cable installation. All that is needed is a single addendum to the law, one prohibiting local governments from creating monopolies. The law could simply mandate that municipalities must offer franchise licenses to all ISPs if they offer licenses to one and that all licencees must be be charged at the same rate.

    The only reason we have cable monopolies in the U.S. is because the Cable Communications Act of 1984 created that perverse incentive. Other countries without such laws have much faster service at much lower prices.

    If federal law permitted local governments to do this sort of thing with groceries, computers and cars we would have regional monopolies for those products as well. Be grateful that your town council is not permitted to sell grocery, computer and car franchises.

    --
    Ceci n'est pas une signature.
    1. Re:End the ISP monopolies by Svartalf · · Score: 1

      The biggest problem without them is you'd end up having a tragedy of the commons. How would any of them intercommunicate to allow the Internet to be.

      One of the things you can do is harshly punish the ISPs in question when they play games like this. One of the things they have right now is "common carrier" status. That's a liability shield against all sorts of things that your customers might do that's illegal. You could be held vicariously liable if you don't have that status and they commit acts of sedition, copyright infringement, etc. You used to run the very real risk of losing that status as a provider of services if you pulled a stunt like this- which kept them mostly from pulling crap like this. We need to bring that back, to be honest.

      --
      I am not merely a "consumer" or a "taxpayer". I am a Citizen of the State of Texas
  31. Re:Netflix, in the parlor, with the fireplace poke by Cramer · · Score: 1

    Netflix has other ISPs, and significant traffic engineering power at their disposal to get traffic to flow how they like. The simple fact that people could "VPN around" the congestion is proof Netflix could have used a different path to that user. However, it's easier to whine to the media in lame attempts to get Open Connect in the door. (i.e. host our business for free.)

    Plus, they bought bandwidth from some of the cheapest providers around, who everyone knows isn't going to care when they fail to deliver. (Cogent is infamous for "peering disputes".)

  32. Re:Why rely on peering? by Cramer · · Score: 1

    They have (had?) more than one provider. They have their own ASN - AS2906. It's readily apparent they suck at traffic engineering. (or they let it happen to try to push Open Connect.)

  33. Split them up, please by pellik · · Score: 1

    Going with the notion that excessive regulation has it's own cost, maybe the solution here is to split up the cable giants? Comcast could easily be three separate companies- one that manages infrastructure and charges for it, one that sells TV channels to customers, and one that sells internet. The two later companies would just lease bandwidth from the former.

  34. Re:Why rely on peering? by NormalVisual · · Score: 1

    They intentionally dumped this traffic on the connections of the ISPs who wouldn't give them free expansions, in hopes that the customer complaints would force their hand.

    Except you're not telling the full story. Netflix was well aware of the potential congestion issues, and offered their to place their own CDN boxes at no cost to any ISP that asked in order to alleviate this. Comcast chose not to avail themselves of this offer, as they couldn't double-dip if they did.

    --
    Please stand clear of the doors, por favor mantenganse alejado de las puertas
  35. Re:How many engineers does it take to screw netfli by fibonacci8 · · Score: 1

    There's too much revisionist history in the Boss's statement for my tolerance. Boss- "First they take our subscribers now they're forcing to upgrade our equipment... well fuck em!" First they had subscribers, then Netflix introduced the online version of their service (circa 2008) bringing in more subscribers, then Xfinity got added (circa 2010) offering the speeds demonstrated by Netflix carried by Comcast but with a selection of Comcast's cable offerings. fast forward a few years and Netflix is still the better offering so Comcast strategically decides to upgrade in ways that improve Xfinity on-demand services without accepting offers from Netflix to likewise maintain the quality of service Netflix customers experienced in the past and presumably would still get to experience if the dollars spent on monthly connection fees to Comcast actually went into improving total network quality instead of just promoting Xfinity services. It would be nice if it were just friendly competition going on, but it's Comcast trying to cut in on the streaming market after Netflix showed it was profitable.

    --
    Inheritance is the sincerest form of nepotism.
  36. Careful not to turn them into utilities by Karmashock · · Score: 1

    Listen, we will get what we want right now by turning big monopoly ISPs into utilities.

    However, we have to think strategically. If we only think tactically we will get out maneuvered.

    Play chess for a moment. What happens when the big ISPs are utilities? Yes, they will be regulated more heavily but they will also have their protection from competition enshrined more deeply in law.

    What is more, the federal government will gain increasing control over the internet. Consider the NSA for example. Do you for an instant think they won't exploit this situation? If the FCC starts dictating things to the ISPs how hard would it be for the NSA to go to the FCC and get them to put an NSA box at the ISP? Child's play.

    And that is just the beginning.

    The wild free days of the internet are over if we turn the ISPs into utilities. Will the ISPs fuck us on occasion? Yes. But I'll take that on an ongoing basis rather then give the feds total control. Because at least with the ISPs there is hope. With the feds it is gone. You'll start seeing regulations on free speech, requirements that people use real names on the internet, micromanaging of peer to peer content.

    I mean seriously what do you think is going to happen once the feds get this power?

    The solution has always been greater competition. it is the only way to eat our cake and have it too. With greater competition the ISPs can't fuck over their consumers without losing market share and they aren't being micromanaged by politicians.

    Best of both worlds.

    Competition. Please.

    --
    I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
  37. Re:How many engineers does it take to screw netfli by UnknownSoldier · · Score: 1

    /Oblg.

    South Park - Cable Company
    https://www.youtube.com/watch?...

  38. No detailed report required by Anonymous Coward · · Score: 1

    I've mentioned this in other threads involving peer points.

    My company has Cogent, Verizon, and various other carriers with 100mbit or larger business connections at our offices around the US.

    Every office we have that has Cogent has problems with speed with every office that has Verizon. One of them to squeeze 15-30 mbit/sec for extended periods between them but that is the best it gets (and that is between a 1gb Verizon to 1gb Cogent connection). Most are FAR worse like under 200kb/sec for hours on end. One of them even dips down to under 50kb/sec. For one of the offices we investigated further looking for answers. Cogent did speed tests for us one hop before the peer point with Verizon and we achieved near line speed. Cogent also showed us metrics they see at that peering point and that they have offered to increase the speed at that point but Verizon has not considered it. During the troubleshooting Verizon basically told us there is nothing wrong with our Verizon end of the line or their network and it's not their problem, call Cogent. We got an excuse along the lines the internet is unpredictable and there are many things beyond their control, as if I was a home customer calling Verizon level 1 support because the interwebs were slow.

    Cogent to XO, XO to Level3, Level 3 to Verizon, XO to Verizon, Sprint to Verizon, and so on all all perfectly fine. Just Verizon to Cogent is the problem.

    This is happening, peering points ARE the new defacto way for companies to create fast and slow lanes and cheery pick traffic. When you have one company like Comcast or Verizon acting as a long haul T1 carrier AND acting as the last mile this is bound to happen. They can use tricks like this at the T1 peering level to manage their load at the last mile.
    In my companies case, we are getting dedicated ethernet from Verizon in managed data centers and we suffer the same last mile effects.

  39. Cogent is willing to pay these costs by tepples · · Score: 2

    Since when were fibre cables, $20000 optics, Switch ports, and 40-Gigabit port licenses free when the link is turned off?

    Not free, but Cogent is willing to pay these costs itself. Verizon and Comcast won't take Cogent's offer; they want to charge Cogent an arguably excessive markup on top of Cogent's costs

  40. Summary: Netflix breaks Internet by Dr+J.+keeps+the+nerd · · Score: 1

    The data the article presents really just shows a ton of Netflix traffic breaking the Internet for other users. Shouldn't all that adaptive bitrate stuff make it NOT break other flows? Apparently, not so much. Did Netflix respond by making their video delivery less aggressive, the way Bittorrent did with LEDBAT? No.

    What did we learn?
    1) Netflix breaks any link it's on. Period. Full stop. The rest of the Internet only gets through when Netflix isn't peered together with it.
    2) Therefore ISPs -- ALL ISPs -- bad.
    3) Therefore Net Neutrality so Netflix can break the Internet.

    Of course, one might be tempted to conclude that big data users should work out their own peering and financial arrangements so that they don't mess up the Internet, but that would make one a corporate shill.

  41. Ask about a Netflix exclusive show by tepples · · Score: 1

    The ISPs are also TV providers and they don't want you to have a good netflix experience.

    Then ask about a specific work to which Netflix has the rights and the TV provider division of the ISP does not. "I'm having trouble watching House of Cards at home. It works fine on $different_isp_next_town_over. Might this be a problem with Comcast?"

  42. Packets != circuits by tepples · · Score: 1

    I make a 'call' from Comcast to Netflix...

    And Netflix "calls" you back with the data.

    A circuit-switched network such as the PSTN allows sending information in both directions over one "call". A packet-switched network such as the Internet, on the other hand, doesn't see "calls"; it sees "datagrams". Except for last mile customers, each side pays for how many packets it sends. Otherwise, it'd be possible to manipulate billing by doing the equivalent of the difference between PORT and PASV in FTP.

  43. Internet Protocol is stateless by tepples · · Score: 1

    You are correct that TCP is stateful. But the fact that TCP is stateful is irrelevant. ISPs are Internet Service Providers, and Internet Protocol is stateless. From the point of view of an ISP's infrastructure, TCP is just an application that runs on Internet Protocol. Otherwise, it'd be possible to manipulate billing through the equivalent of switching between FTP's PORT and PASV commands, which change only who sends the SYN.

    1. Re:Internet Protocol is stateless by AK+Marc · · Score: 1

      IP isn't a protocol. It's called TCP/IP for a reason. You are too dumb to educate. You probably know the answer, but are lying to yourself to win an argument that everyone reading this knows you already lost. IP is a suite of protocols, TCP being one of them. THat you don't even know what IP is, yet feel the need to lecture others on it proves you are too dumb to understand it.

    2. Re:Internet Protocol is stateless by tepples · · Score: 1

      If you feel so strongly that I am lying to myself, then please explain what is incorrect in the following three statements: TCP is a connection-oriented protocol in the Internet Protocol Suite. TCP connections have two halves, one in each direction. Traffic is billed based on who sends more data down each half of the connection.

    3. Re:Internet Protocol is stateless by AK+Marc · · Score: 1

      Traffic is billed based on who sends more data down each half of the connection.

      In the "old days" traffic was billed by who received the most. User pays.

      So you are still wrong. If you were right, I could start a small ISP that only had residential connections (essentially "receive only"). And, since I sent nothing, and the other guy sent me more, I could charge my users $0 and still make money from all the people paying me to take their traffic.

      But it doesn't work that way. The small ISP pays a lot to receive data, not send it.

      Reality proves you wrong.

  44. Comcast's private property by tepples · · Score: 1

    The ISP's premises is the private property of the ISP. If Netflix wants its box sitting in Comcast's private property, it can pay rent, just as you would have to pay rent to lawfully live on private property owned by a landlord.

  45. Re:Netflix, in the parlor, with the fireplace poke by perryizgr8 · · Score: 1

    The simple fact that people could "VPN around" the congestion is proof Netflix could have used a different path to that user.

    That's not how it works.

    --
    Wealth is the gift that keeps on giving.
  46. Re:Why rely on peering? by NormalVisual · · Score: 1

    Do you really think that Comcast hosts Akamai servers for free?

    I've read that they do charge, and I've read that they don't. It's in their best interest to do so, unless you have a citation to the contrary.

    --
    Please stand clear of the doors, por favor mantenganse alejado de las puertas
  47. Re:If you made a properly formatted petition ... by suman28 · · Score: 1

    You are correct. The slashdot effect would be great, but no one cares about the opinions of the common folk. We already saw that with the Net Neutrality debate on the FCC comments debacle. In the end, if you are not greasing Tom Wheeler's cock, he will not be very nice when he gives you the rim job.

  48. Re:How many engineers does it take to screw netfli by TangoMargarine · · Score: 1

    Engineer - "Hey Boss, we need some cash to upgrade the connection to these networks."
    Boss - "How about we raise their rates, just say we upgraded the network, and pocket the difference?"
    Catbert - "I like the way you think."

    FTFY

    --
    Unity? Screw that: XFCE. Slashdot Beta? Screw that: SoylentNews. Australis? Screw that: Pale Moon. UX developers DIAF
  49. If receiver pays by tepples · · Score: 1

    You make a good point. I guess my misconception was that sender pays for long haul transit and the endpoint pays for the last mile. But if receiver pays, even for long haul, then you can DDoS someone's billing by flooding his connection with packets. And if receiver pays, even for long haul, then why does Comcast slow down Netflix? All Comcast's customers are already paying.

    1. Re:If receiver pays by AK+Marc · · Score: 1

      And if receiver pays, even for long haul, then why does Comcast slow down Netflix? All Comcast's customers are already paying.

      That's why people are angry. Comcast is already paid by its users for content. And Comcast tries to not deliver competing content. Making up reasons why, because "we are monopolistic douchebags" is not a good reason.

  50. thanks by sup4mak386 · · Score: 1

    thanks for useful article ..keep it up

    --
    http://watchamoviesonline2k.blogspot.com