Slashdot Mirror
Hunting the Mythical "Bandwidth Hog"
eldavojohn writes "Benoit Felten, an analyst in Paris, has heard enough of the elusive creature known as the bandwidth hog. Like its cousin the Boogie Man, the 'bandwidth hog' is a tale that ISPs tell their frightened users to keep them in check or to cut off whoever they want to cut off from service. And Felten's calling them out because he's certain that bandwidth hogs don't exist. What's actually happening is the ISPs are selecting the top 5% of users, by volume of bits that move on their wire, and revoking their service, even if they aren't negatively impacting other users. Which means that they are targeting 'heavy users' simply for being 'heavy users.' Felten has thrown down the gauntlet asking for a standardized data set from any telco that he can do statistical analysis on that will allow him to find any evidence of a single outlier ruining the experience for everyone else. Unlikely any telco will take him up on that offer but his point still stands." Felten's challenge is paired with a more technical look at how networks operate, which claims that TCP/IP by its design eliminates the possibility of hogging bandwidth. But Wes Felter corrects that mis-impression in a post to a network neutrality mailing list.
Because the operators pay for the bandwidth. The high bandwidth users are less profitable than the other ones.
They don't negatively impact operations in the sense of taking up a scarce resource that degrades other customers' performance. However, they do still use above-average amounts of bandwidth, which costs ISPs money. When offering a flat-rate, unlimited-use service, your economics come out ahead if you can find some way to skew your customers towards those who don't actually take advantage of your claimed "unlimited use".
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
This could just be a problem with your router. Maybe it struggles to handle all of the torrent connections.
I guess it's cheaper to sacrifice 5% of revenue than to have to undertake a network upgrade.
This mentality is part of why the U.S. lags so much in broadband.
"If this doesn't scale, logically, up to the network at a whole, I'm not sure why."
Plenty of reasons why that won't scale up to the network as a whole. First and foremost, your ISP's network topology is a lot more effective for many users than the simple "star" topology most home router/switch combos give you. Beyond just the topology, the ISP uses better equipment that can cap bandwidth usage and dynamically shift priorities to maintain a minimum level of service for all users even in the presence of a very heavy user. The ISP also has much higher capacity links than what you have at home, and certainly more than the link they give you, and so even if there were a very poor topology and no switch level bandwidth management, it would be very difficult for a single user to severely diminish service for others.
I do not have any sympathy for ISPs when it comes to this issue. If they sell me broadband service and expect me to not use it, then they are supremely stupid, and retaliating against those users who actually make use of the bandwidth they are sold is just insulting. They oversold the bandwidth and they should suffer for it; blaming the users is just misguided.
Palm trees and 8
Modern Torrent clients that support DHT (most of them) generally default to UDP. Since the Torrent protocol already includes block checksumming there's no reason to also use TCP for that, congestion control generally isn't an issue with Torrent traffic either, you just push the pipe till it's full. For video unless you have significant buffering there's little reason to have error checking or congestion control because if you can't get the bits in fast enough without retransmits then the video's not going to be watchable. I'm not sure how much video is done using UDP vs TCP, Flash is TCP and Netflix appears to use a design where they adaptively send different encoding levels of the same content across a HTTP 1.1 stream as they see bandwidth starvation.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Or stop overselling your bandwidth and then you wouldn't have to worry about how much bandwidth your users use.
I'm not saying he's wrong... quite possibly he's right, but seriously - how does someone's blog entry that doesn't provide one single data point to back up the claim make it to the front page?
The important thing that he's doing is trying to shift the burden of proof back onto the ISPs and telcos. They just declared that some people are bandwidth hogs and terminated their connection. They didn't give the public any proof that they were ruining the internet experience for anyone else ... nor did anyone come forward after the purge and say, "Gee, my internet sure is fast now that the bandwidth hogs are disconnected!"
So he calls for proof since he hasn't seen any. He has to say that there are no bandwidth hogs in order to get a response from the telcos. Saying someone might be wrong is not the same impact as calling someone a liar. Yes, he's basing this on an assumption but it's just the same that everyone assumed there were individuals out there ruining the experience. All of us just let the telcos terminate the service of whoever they wanted to and then we moved on with our lives.
I welcome his opposing viewpoint and challenge to "because we said so." They can release anonymous usage data without harming anyone so why not open it up to a request?
My work here is dung.
You pay for an 'up to' 70Mbps connection. 'Up to' means exactly what is sounds like - you are never going to go above that rate. It says absolutely nothing about the minimum or average rate. Since they make no claims at all about minimum or average rate, there is no false advertising. Every consumer is well familiar with what 'up to' means. How many times do you see an ad that says 'Sale! Save up to 50%'. Does that imply that you are in fact going to save 50% on everything you buy? No, it implies that somewhere in the store is at least one item that is 50% off - every other item may be full price, or more likely, discounted at a rate less than 50%.
That's just bad configuration, not bandwidth hogging. Prioritize ACK packets and you can run torrents all day without affecting other uses of the network.
One user can bottleneck the pipes, but since their stuff isn't fast any more either, we're all good?
If the "bandwidth hog" isn't fast anymore, he's no longer a hog.
Give me Classic Slashdot or give me death!
Over selling isn't wrong, it is necessary for services like this. The fact is, all service providers oversell their capacity. The trick is to manage the overselling to a ratio that on average, within a specific scope, doesn't cause bandwidth jams for a prescribed statistical level.
Having run an ISP, the oversell ratio is about 10:1 - 15:1 depending on who your subscribers are, and their usage patterns. This means you can get 10-12 people on a data circuit that is really designed to handle 1 fully loaded client. This statistical usage only works at large scales, and actually as the scale increases, may raise the over subscription to 20 or 25 to one.
I guarantee you that if everyone wanted to Torrent all the time, at full speed, the internet could not handle the traffic. It wasn't designed to. It has been over sold.
Bittorrent is not normal traffic pattern. A Torrent is a congestion point on the internet, at a place where one is not expected. Most people don't need 80 gigabytes of streaming data, day in and day out. If this were DVD movies, you'd be downloading more movies than you could watch.
I don't have any complaints for ISPs that throttle Torrents and take other measures against "high usage" users, who are file sharing. I'm not against filesharing, I'm against idiots who cause congestion because they don't know how to configure Bittorrent client to be "reasonable".
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
First of all, I am, and always will be, a bandwidth hog. Why? Because I'm better at using the internet than everyone around me. That means I find more things, and bigger things, to download. If they someone banned P2P, I'd still have more streamed video than anyone I know. If they banned that, too, I'd still download more images. If they banned that, i'd still have more web traffic, email, IM, etc etc etc. I will always be a 'hog' in any environment. I was even told that I was "#1 abuser" of the "Unlimited" service when I was on dial-up in a small town and they tried to charge me an extra $300 that month. As someone else had just come into town, I switched, obviously.
I don't pay for the top tier of residential service to just let it sit idle. I'm going to -use- it.
I have absolutely no sympathy for people that sell me something and then get upset when I actually use it within the original limitations. I have only a small amount of sympathy for people that sell me something and I use it beyond their arbitrary limitations, even if I agreed to them.
Why?
America has -crap- for internet compared to other developed countries. We are quickly falling behind the rest of the world in terms of internet bandwidth. This is purely from greed and laziness on the part of the ISP. They refuse to upgrade and try to prevent competition at the same time. Sprint even has the nerve to advertise Pure and claim that it's faster than Cable internet, despite being 1/10th of the speed.
"If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
I also go through my client list and drop those that consume more of my time and resources in favour of the easier clients who ultimately improve my business at a lesser cost. What's wrong with that? My company, my rules. "We reserve the right to refuse service to anyone" -- it's in every restaurant. Why would you expect a business to serve you? Why would you consider it a right?
Let's say you sell widgets.
You have 5 people come to you, each one wants to buy 1 widget. And another guy shows up and wants to buy 5 widgets.
You only have 5 widgets in stock, you need 10, but you really want their money. So you sell each of those people a coupon for their widgets, and tell them to pick it up at your warehouse. You figure they won't all run over there right now, and you'll probably have time to get a couple more widgets in stock before anybody notices.
Of course you don't tell your customers this. You don't tell them "I only have 5 right now, you'll have to wait 'til the next shipment" You just take their money and leave them with the impression that the widget is there, waiting for them, available for pickup whenever they want.
So all of them show up at the warehouse about 5 minutes later. All of them want their widgets now. But you don't have enough widgets to go around. So you call the guy who bought 5 widgets a "widget hog", cancel his order, and throw up a hastily-made sign that says "limit 1 per customer."
Legal? Yeah, I guess... Assuming you refund his money.
Right? Not so much. You should have clearly explained that you only have 5 widgets in stock, or that the coupon couldn't be redeemed for a week, or that there was a limit of 1 per customer, or something. You mis-represented what you were selling to your customers.
Likely to leave a good impression on your customers? Nope.
"Work is the curse of the drinking classes." -Oscar Wilde
They oversold the bandwidth and they should suffer for it
I agree.
I disagree. overselling is fine, the problem comes when they squeeze too much overselling out of what they've got.
For example, ISP A had 100gb of bandwidth and 1000 customers. They sell each customer 0.1gb, everyone's fine but no customer will use that much bandwidth so most of the network cap is wasted, and when the upstream ISP sells it to you a quite a large sum, you'll find you have no customers as the price you have to charge them is prohibitive.
So, you oversell a bit, and you have 2000 customers, each one getting 0.05gb of bandwidth, most of the time no-one will ever use more that, but occasionally 1 will, when they do a big download. Still, you've made the network more price-competitive but its still probably more than most people will pay for.
So, you go further, and you have 10,000 customers so each could use 0.01gb, which is fine if all they do is surfing and email with the odd bit of streaming and downloading, and the price per customer is great. But then someone decides to up/download 365/24/7... and the model of shared usage falls apart.
The trouble is that most ISPs can look at how much bandwidth they have per customer, and how much actually gets used and make appropriate allowances for that, but most customers will use a lot of bandwidth during 4pm to midnight, and practically none the rest of the day. The better ISPs will tell you this, and will cap you only during that peak time, but the 24/7 downloaders don't care about the other users, so the ISP gets a bit stroppy with them and kicks them off the network, or caps their speeds to persuade them to go or to minimise their impact of the network.
OK, some ISPs oversell to the point where a minimal number of users downloading make an impact, but generally downloads aren't an issue during off-peak times.
The answer is nearly always simple - go with a provider that allows unlimited transfer during off-peak and don't be selfish during the peak times. Its like (car analogy time!) rush hour, drive as fast as you like at 10am, but don't expect to go faster than 5mph at 8am.
The system wasn't designed, nor sold, with torrents in mind. End points are supposed to be content consumers, not content providers.
This is incorrect. TCP is designed so that every computer on the network is a peer. There is no fundamental difference between my computer at home and slashdot.org. The great promise of the internet is that everyone can be a content provider. The ISPs seek to destroy this notion in favor of simply creating a content distribution mechanism that they control. That is far, far worse than any "bandwidth hog" could ever be.
Give me Classic Slashdot or give me death!
Companies overselling is a very popular and acceptable thing too (for them). Airlines, hotels, and movie theaters often do this expecting no-shows and cancels. But i expect the percentage oversold is based on historical facts for that particular day the previous year. ISPs might have been able to oversell so much in the past but as more content moves from tv/phone/radio to the internet, the typical usage might be outstripping the previous years usage numbers. Just my thoughts..
http://soylentnews.org/~tibman
Then wouldn't it be handy for the ISP to state the normal bandwidth vs. the burst rate? If they told me that up front, rather than just the full bandwidth of the pipe, I wouldn't have a problem.
I don't have any complaints for ISPs that throttle Torrents and take other measures against "high usage" users, who are file sharing.
My only issue with throttling is that there are better ways to manage your network than arbitrarily lowering someones bandwidth. My office is located out in the sticks and we can only get a T-1. I have to share 1.5mbits with 60 employees, including time critical services such as VOIP and VPNs. I set up a priority list that looks like this:
Each tier gets a promised amount of bandwidth. When they need more they borrow unused bandwidth from the other tiers in order of priority, VOIP gets first dibs, then the priority tier, then TCP acks, etc. etc. I don't arbitrarily limit the bandwidth available to a certain protocol or end user. If the line is free then even bittorrent is allowed to use 100%. If the line is pegged then traffic is allowed through according to the above priority list. Bittorrent doesn't care if it winds up in the queue -- VOIP does.
I did this with an old Linux box and the HTB packet scheduler. It cost nothing but time to setup. If I can manage to do this then doesn't it stand to reason that a company with the resources of Comcast could pull it off? Why do they need to impose bandwidth limits and the like when they could simply prioritize interactive traffic ahead of the torrent kiddies?
I want peace on earth and goodwill toward man.
We are the United States Government! We don't do that sort of thing.
It isn't a lie, it is accepted business practices. You don't want what you're really asking for. You want 10mb for $24.95, not the dedicated 10mb for $1500.
If you REALLY wanted dedicated 10mb connection, you'd ask for it, with QoS and SLA agreements, in short, you're asking for BUSINESS CLASS SERVICE not the Consumer Class versions.
Yes, most service providers offer this service, it is just that most people really wants to pay for it.
Warning, Car Analogy Ahead:
You're like the guy who buys a F150 truck with the smallest engine, and then complains that they can't tow their big-ass boat up the hill very well, even though the "Tow Rating" says it should.
And you burn up your engine / transmission early because you do it every weekend. Should FORD be sued because they advertised it wrong? Or are you an just an idiot using consumer toys for commercial duty?
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
If they tell you the real rates, the other lying companies would look alot better than, wouldn't they.
Since everyone is lying, no one is going to stop lying.
A more accurate way to put it is this: the telcos want you to pay for a 1Mbps line, but let you run it at 70Mbps if resources are available.
> it takes more computing resources to implement a source based priority queue than to implement a simple fifo queue.
Similarly it takes more computing resources to do what ISPs are already doing (throttling, disconnects based on XYZ) than to implement a simple fifo queue.
> to be of any use such prioritisation needs to happen at the pinch point (that is at the place there is actually a queue built up), unless you deliberately pinch off bandwidth
I'm sure they can find the pinch points - they're the spots where they keep having congestion (which should show up on one of their monitoring screens).
I remember hunting for an ISP back in the mid 90's. All ISPs priced there service as if bandwidth was going to be 100% utilized. A cheap rate was roughly $200/month...
Overselling bandwidth is a good deal for both the provider and the consumer. Without it the net as we know it would have been stillborn. Yes there are abuses, but the alternative is far worse.
In some more perfect world, an ISP could be counted on to clearly explain all the tradeoffs, but in the world I live in, marketeers speak to rubes, and ISPs differentiate themselves via specious and irrelevant shiny talk of "7MBS bandwidth"
The "harm" you experience when the ISP can not fully deliver in return for the artificially low amount you are spending doesn't really hold much weight. If you need the bandwidth, there are many who will honestly sell it to you. It's just that the real premium for that is 2X to 10X the shared rate.
I understand all that. And I don't have a problem with that.
The problem I have is with an ISP selling something called "unlimited" when they know perfectly well that they have neither the ability nor the intention of delivering anything vaguely resembling unlimited service.
And while I can assume that they don't actually mean unlimited literally, I don't generally have any way to know what they do mean by unlimited. Most of the times the limits or caps are not documented or are not predictably enforced or are not made available to customers.
I have absolutely no problem paying for the level of quality that I want.
I do have a problem paying for the level of quality that I want, and then finding out that the ISP has a different definition of "quality."
"Work is the curse of the drinking classes." -Oscar Wilde
That depends on how you man bandwidth. I would say that, in the case you describe, he is NOT getting the bandwidth promised. It is the responsibility of the ISP to A) enforce their caps fairly (and as advertised) and B) make sure they have enough "pipe" to handle ALL of the demands that are made within the allowed amounts.
So if I subscribed for 100 units of bandwith (it doesn't matter what the units are), I should have that available to me, regardless of what any other network user is taking up. If they can provide that while oversubscribing (because most people seldom use even a fraction of their allotment), then more power to them.
If that means they need a LOT more pipe, to deliever what they sold, then maybe they shouldn't have oversold so much pipe. Oversubscription is always a bit of a gamble.
Sometimes you gamble and lose, sometimes you can't prevent your commitments to one customer from causing issues with the commitments to another. When that happens to a company, they need to find a way to deal and to adjust either their capabilities or their offerings.
It is regrettable to offer a service, have genuine problems, and have to modify agreements or spend more on expansion. It is downright dishonest to continue to advertise a service that you already know that you can't provide as advertised, and continue to take your customers money while blaming them for being why you can't provide the service that they are paying you for.
-Steve
"I opened my eyes, and everything went dark again"
Since everyone is lying, no one is going to stop lying.
Until the FCC busts out real ISP regulations that keep them honest.
"a max realistic bandwidth of 38 mbps ... a small customer count, 100 customers... cable companies usually offer 10-20mbps of dowload speed on docsis 2.0 plant"
In other words, you're selling something you don't have and then blaming the customers for wanting to use what they bought.