Slashdot Mirror


Georgia Tech's ShaperProbe Detects ISP Traffic Manipulation

An anonymous reader writes with this excerpt from Ars Technica: "Two researchers at Georgia Tech can tell you exactly how American ISPs shape Internet traffic, and which ones do so. Bottom line: of the five largest Internet providers in the country, the three cable companies (Comcast, Time Warner, Cox) employ shaping while the telephone companies (AT&T, Verizon) do not — though that fact is less significant for the user experience than it might first sound."

22 of 113 comments (clear)

  1. Re:Cue the cable company bashing in 3...2...1.... by Squiddie · · Score: 5, Insightful

    Their network runs on public land. They are also granted exclusivity by local governments. I think that regulation is in order.

  2. not all shaping / policing is bad. by Anonymous Coward · · Score: 5, Informative

    I work for a college, and we shape / police traffic to / from the Internet.

    This was a necessity on our 3Mb link of many years ago, but has still been useful on our 1Gb link of today.

    This policy has greatly improved the user experience. Interactive protocols have low latency, bulk transfer protocols get sent to the end of the line. Where we do slow down things, it isn't really noticed by most folks. After first implementing this many years ago, we immediately got positive feedback. Now it is just "how things are."

    Hell, I shape / police traffic at home to my cable modem. VOIP and interactive ssh are still usable even with huge downloads going on now, and users hammering the public wifi I provide to my neighborhood.

    1. Re:not all shaping / policing is bad. by Idbar · · Score: 3, Interesting

      Hum... that's the issue. You're providing for a bunch of students that don't have a contractual agreement with you. Now imagine that the company you contracted the 1Gb link to, started shaping your traffic. That eventually comes to you, not receiving the 1Gb you paid for.

      I understand you sometimes need to shape the traffic to prioritize services you want to perform better. But these companies are getting their money offering services they don't completely provide: First they charge you for access they can shape. And second, they put a cap just in case you get away with it.

      Again, I understand your work case, because if you don't pay for it, normally you tend to abuse it. But if you pay for it, why would they need to mess with your traffic?

    2. Re:not all shaping / policing is bad. by Dhalka226 · · Score: 2

      But these companies are getting their money offering services they don't completely provide

      I hesitate to use words like "every," but I can think of no ISPs who advertise their plans as anything other than "up to X Mbps." Why you believe that not getting that is somehow a service they don't provide eludes me, particularly when the reason you are not receiving it is legitimate shaping activity*.

      And second, they put a cap just in case you get away with it.

      I tend to agree with you on this one, for a lot of reasons. In particular I object to the limits being buried in the terms and conditions somewhere rather than being open about it. Stil, your conclusion is that you're paying for it and they're not delivering it which is false. You're paying for what your contract says you're paying for, and somewhere in that fine print--at least nowadays--is a limit.

      There are a lot of things to object about the state of Internet service in the US. The limits are going to quickly become a burden on growing Internet-based industries. The monopoly status of most providers means they are uninterested in serving their customers or competing for their business. They oversell their services far more than they should and pocket dollars intended to improve their networks. That's off the top of my head. "I'm not getting what I pay for," meh. Not so much.

      * Admitting that not all shaping activity is legitimate, especially based on source/destination.

  3. Re:Question by Anonymous Coward · · Score: 4, Informative

    no, shaping is done per-protocol, and throttling is done per pipe

  4. Re:Question by ozmanjusri · · Score: 5, Insightful

    Is shaping the same as throttling?

    Sort of.

    Online, shaping and throttling are something network companies do to customers. In meatspace, throttling is what customers want to to to network company executives.

    HTH.

    --
    "I've got more toys than Teruhisa Kitahara."
  5. Re:Question by Ungrounded+Lightning · · Score: 5, Informative

    Is shaping the same as throttling?

    Shaping is when they give you a rate and enforce it. (The faster burst at startup is because you had accumulated some credit by not using your bandwidth in the immediately preceeding time.) There may be separate shaping mechanisms for different protocol families and there may also be shaping on aggregates - like total bandwidth across multiple users of a common DSLAM.

    Throttling is when, after they notice that you've used a lot of bandwidth lately, they turn down the rate on the shaper ("traffic manager").

    Shaping is mainly about things like keeping protocols from interfering with each other (by giving different classes of them separate allowances) and avoiding congestion and queue-too-full latency (by limiting the traffic sent to a following box to the amount it can handle.)

    Throttling is about keeping a user's resource consumption down by slowing him down after he's run fast for a while.

    --
    Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
  6. Re:Cue the cable company bashing in 3...2...1.... by Squiddie · · Score: 2

    I guess we should also deregulate food and drugs. You can take care of yourself, right?

  7. Re: Is shaping the same as throttling? by AK+Marc · · Score: 3, Informative

    I find that Wikipedia is good at giving a few people's opinions of terms, but not actually backing up the vernacular definition.

    Shaping: controlling bandwidth among various protocols (whether DPI or QoS, port number, etc.). This can be enforced by throttling some traffic or by prioritization.

    Throttling: capping or reducing the bandwidth available to some identifiable clump of traffic (I use clump because all the other appropriate terms I can think of have some technical definition more strict than what I want to say). It can be done solely in response to congestion, or in the absence of congestion. It can be done on some subset of a subscriber's traffic, or to the entirety of it. Throttling is a slowing or capping of traffic. Most shaping is a subset of throttling. Oversubscription could be considered a form of throttling. Throttling is much more general of a term than shaping.

  8. Re:Cue the cable company bashing in 3...2...1.... by bondsbw · · Score: 3, Insightful

    Since when is exclusivity a free market solution?

    --
    All my liberal friends think I'm a conservative, all my conservative friends think I'm a liberal.
  9. ADVERTISED FEATURE of Time Warner and Comcast by drtsystems · · Score: 4, Informative

    This is said (although almost in passing) in the article. But I will repeat it because i know how few of us RTFA. Time Warner advertises its PowerBoost feature (and Comcast has something similar) where you get like double your usual bandwidth limit for "burst" downloads and then you get throttled back to your limit after the burst is complete. This is a FEATURE they advertise, not something bad. It allows you to (for example) get 15mbit when download a web page or small file on your 7mbit plan. Notice its a 7 mbit plan, they are not throttling you below your plan's rated speed. They are giving you faster downloads for a quick burst. There is plenty wrong with Time Warner, but this isn't one of the the problems.

    1. Re:ADVERTISED FEATURE of Time Warner and Comcast by Anonymous Coward · · Score: 2, Informative

      My neighbors add a couple more tiers to PowerBoost. Weekends the cablemodem borders on useless for any low latency or sustained streaming service (for example hulu, netflix). Weeknights interactive games are tolerable with hit & miss reliability for sustained streaming. Weekday mornings flawless service.

  10. Re:Cue the cable company bashing in 3...2...1.... by Anonymous Coward · · Score: 2, Informative

    Are you some kind of retard? Deregulation doesn't lead to any sort of fair balance, it leads to excessive and unnecessary price increases across the board as monopoly situations are reached. Check out the New Zealand power market if you don't believe this happens.

  11. Re:Cue the cable company bashing in 3...2...1.... by WindBourne · · Score: 3, Interesting

    Well, of course, there will be cable bashing. I use comcast at home, and I KNOW that they hit netflix HARD. I run a router at home and have studied the data coming through. When it comes from a work site, I can download 4 GB no problem. However, when I grab netflix movies that are about 2 gb streams, then I get issues. Interestingly, Crackle and Hulu do not have the same issues, though crackle uses more bandwidth.

    Personally, I agree that they own the network and should be allowed to do what they want. HOWEVER, I also think that we should pass laws FORBIDDING a monopoly into the home. At the least, we should change the monopoly to be from the home to the greenbox and any company can then sign up for a deal with providing service to the greenboxes, AT THE SAME RATES. IOW, if comcast wants to own the greenbox-home monopoly, not a problem. However, they charge other providers the same price that they charge the rest of comcast.

    Basically, it is time to limit the monopoly's size.

    --
    I prefer the "u" in honour as it seems to be missing these days.
  12. Re:Question by ProfessionalCookie · · Score: 2, Funny

    Pretty sure the internet is made of tubes, not pipes.

  13. Re:Cue the cable company bashing in 3...2...1.... by maxwell+demon · · Score: 2

    There already is regulation (granted exclusivity) favouring the provider. Either that regulation is to be removed, or there has to be another regulation favouring the customer to counter it.

    You need regulation whenever you have monopolies, or the danger of monopolies. That regulation has to either prevent forming of monolies, or if that is not possible, regulate those monopolies so that they cannot do too much harm. Remember, as soon as there's a monopoly (either granted or enforced through market power), there's no free market any more to regulate things. And the free market cannot prevent monopolies when network effects are in play.

    --
    The Tao of math: The numbers you can count are not the real numbers.
  14. Re:Cue the cable company bashing in 3...2...1.... by cheeseandham · · Score: 3, Informative

    HOWEVER, I also think that we should pass laws FORBIDDING a monopoly into the home. At the least, we should change the monopoly to be from the home to the greenbox and any company can then sign up for a deal with providing service to the greenboxes, AT THE SAME RATES. IOW, if comcast wants to own the greenbox-home monopoly, not a problem. However, they charge other providers the same price that they charge the rest of comcast.

    That is kind of how it works in the UK (See how British Telecom has been split up).

    BT Openreach was created to "Ensure that all rival operators have equality of access to BT's own local network" and it works pretty well, I have a BT line and BT Wholesale broadband, but provided by a different company with their own service levels, prices etc. And there are a lot of ISP's like this.

    If an ISP doesn't want to use BT's infrastructure in the exchange, they can even install their own whilst still taking advantage of that piece of cable going from the exchange to the home, laid down by public money.

    How is it not obvious to the US politicians that this is a sensible move? More to the point, how the hell did something sensible happen in a UK Parliament?

  15. Re:Cue the cable company bashing in 3...2...1.... by Runaway1956 · · Score: 2

    Want free market? Stop taking subsidies. Stop filing suits against all the small local governments that try to install any system in areas that don't have service. Free market? Just stop being giant fucking douches, determined to milk every dime possible out of the system. Stop abusing the legal system, stop abusing the legislative system, stop ripping the people off. We don't HAVE free trade, you big dummy - not as long as a telco of cable company is permitted to file suit against a community that is trying to provide a service where none exists.

    --
    "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
  16. Re:Question by AmiMoJo · · Score: 2

    Not quite right.

    Throttling is where they simply slow down your entire internet connection. Typically when you download more than a certain amount of data the ISP punishes you with throttling. Kind of like going to McDonald's and ordering a Supersize Big Mac meal, then a member of staff comes over and punches you in the gut as punishment for overeating.

    Shaping is where the ISP tries to slow down certain traffic to give priority to others. The typical use is to slow down P2P and large downloads so that web browsing and Skype get priority. There is a bit of an arms race between software developers who try to encrypt or obfuscate traffic and ISPs who look for ways to defeat them. Going back to the McDonald's analogy it is a bit like them giving you a 1mm diameter straw to drink your coke through, while allowing you to eat the Big Mac and fries as you like.

    Of course in practice ISPs both punch you in the gut and give you the 1mm diameter straw, and then spit in your burger for daring to ask for the meal you paid for.

    --
    const int one = 65536; (Silvermoon, Texture.cs)
    SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
  17. Re:Question by L4t3r4lu5 · · Score: 3, Insightful

    It's car analogy time!

    Shaping is like putting a bus lane / car pool lane on the motorway / freeway; Buses and car pool drivers can move through quicker, at the expense of car traffic having one less lane on the motorway, much like VOIP would be given priority over BitTorrent, or somesuch, but the cars and buses are all capable of going at maximum speed (should traffic allow).

    Throttling is like variable speed limits. In the interest of keeping traffic moving freely across the whole motorway, the speed of heavily trafficked areas is slowed down so it doesn't cause congestion. 70MPH becomes 50MPH in the same way that 10Mbit becomes 2Mbit.

    Data caps are like a bastard child of toll roads; You've driven a certain distance on this road which is covered by vehicle excise or fuel tax, now you have pay a toll. To travel further on this road, you pay more tolls. You can drive only so far each month on the toll roads for free.

    HTH.

    --
    Finally had enough. Come see us over at https://soylentnews.org/
  18. Re:Cue the cable company bashing in 3...2...1.... by muridae · · Score: 3, Insightful

    If the cable company wants to operate without oversight, then they can buy back the copper that the government paid them to install, and pay me and other land owners for the right to run their service across my property line. Until such time that they do so, and as long as they continue taking advantage of government assistance, they can and should answer to government regulation.

  19. Re:Question by DJRumpy · · Score: 2

    According to TFA, the shaping being done isn't protocol specific (although it can be). They equate it to a bucket of tokens, where you start your transfer with a full bucket, and at specific intervals, you must use a token to transfer X amount of data. What that means is that when you start your transfer, you would have full bandwidth until you empty your bucket, after which you have to wait for the bucket to be filled again to continue your transfer. This has the effect of giving you a full pipe during your initial transfer (burst), but that will quickly turn into a more metered throughput according to whatever policy they have applied to your account.

    From TFA:

    Traffic shaping hardware generally relies on the concept of a “token bucket.” Traffic management hardware will generate a digital token for each Internet user at a predetermined rate. These tokens fill a virtual token bucket; transmitting packets over the Internet removes tokens from the bucket. If the bucket empties, no more data can be transmitted until a new token is deposited.

    The practical result is that the user sits down to her computer with a full token bucket and can immediately blast data through her connection as fast as the connection can go. But after some interval of time, usually measured in seconds, this sort of full-throttle data transmission empties the token bucket and the user is now limited to transmitting at the token generation rate.