ISPs Fight Against Encrypted BitTorrent Downloads

oglsmm writes to mention an Ars Technica article about a new product intended to detect and throttle encrypted BitTorrent traffic. When torrents first saw common use ISPs would throttle the bandwidth available to them, in order to ensure connectivity for everyone. Some clients began encrypting their data to get around this, and the company Allot Communications is now claiming their NetEnforcer product will return the advantage to the ISPs. From the article: "Certainly, increasing BitTorrent traffic is a concern for ISPs. In early 2004, torrents accounted for 35 percent of all traffic on the Internet. By the end of that year, this figure had almost doubled, and some estimate that in certain markets, such as Asia, torrent traffic uses as much as 80 percent of all bandwidth. However, BitTorrent is an extremely important tool that has many uses other than what everyone assumes it is good for, namely movie piracy."

Comments

lol, moustrap, mouse

[(fagging beta)]

If you build a better mousetrap someone will fling a couger at you.

Re:lol, moustrap, mouse

[iminplaya]

Is that an OSX joke?

Re:lol, moustrap, mouse

[letxa2000]

You know it's time to sell stock in a company when you see the company in a technical arms raise against the customer to deny the customer service. Great thinking, ISPs!

Re:lol, moustrap, mouse

[shmlco]

"However, BitTorrent is an extremely important tool that has many uses other than what everyone assumes it is good for, namely movie piracy."

Yeah. Too bad those "important" uses only account for 5% of the total traffic.

Okay, now a while back when ISPs first started throttling traffic the big workaround was encryption. Now it seems that encryption isn't a silver bullet either. Other sources have indicated that pattern analysis would catch attempts to emulate other protocols, such as secure VPN connections.

So is the war over? Or is everyone going to focus on other ways to outsmart the system?

Re:lol, moustrap, mouse

[Millenniumman]

No, no, no! You can't do that! You have to support regulation of ISPs, not use the free market and switch! You are a powerless peon, and only the government can protect you from the evil corporations. You are preaching a philosophy of hate, you evil fascist!

Re:lol, moustrap, mouse

[rawg]

You don't know how much bandwidth costs do you. $500 a month for 1.5Mbps. Normally, that can work find for about 100 customers. But when you have someone using P2P, connecting to 100's of other clients, using every bit of bandwidth there is, then there is a problem. Now if 80 of your 100 customers are doing it, expecting that they should all get 1.5Mbps... You would have to buy 80 T1's at $500 each, per month. Then your customers are only paying $20 per month. So do the math.

Re:lol, moustrap, mouse

[bky1701]

I don't really care, yanno? If they say "3 MB/s!!!!" then as far as I care, blocking anything form having that is nothing other then false advertising. If they don't want you to use your full advertised speed, then they need to stop saying they are providing it.

Re:lol, moustrap, mouse

[Anonymous Coward]

OSX is the joke.

well, it only makes sense

[bunions]

You don't want your customers actually using the stuff they're paying you for, after all.

Re:well, it only makes sense

[iPodUser]

I agree. We pay up to $60 per month to have this great thing called broadband, and what do we get? Carriers wanting to restrict VOIP use, throttling Bittorrent traffic, refusing to guarantee any particular level of service, etc. A question for the service providers: Why do you think users sign up for the service? To check email? to browse a few websites? We could do that with cheap or free dial-up. These applications you are so quick to restrict are the reason that people signup in the first place! Instead of putting the effort and expense into creating hurdles for the users, spend the time and money on upgrading the infrastructure to support the increased demand.

Re:well, it only makes sense

[secolactico]

You don't want your customers actually using the stuff they're paying you for, after all.

Of course not! How else am I going to re-sell it to some other sap.

What we need is more truth in ads. Make sure your customers know that you are not guaranteeing a given bandwith unless they pay for a clear channel or some such.

Re:well, it only makes sense

[arivanov]

Exactly.

The price is formulated on the basis that you do not use it.

I agree with you - this is fraud and there is only one way to fix this.

The problem will go away immediately if ISPs turn off flat pricing and users start to pay for bandwidth used. Even better - if they start charging a differential/tiered pricing depending on the type of traffic. There is no rocket science here. The gear currently on the market is supposed to be able to do it (does it do it is a different matter).

The business models is well known and this is the way the Internet used to operate all the way up to the end of the 1990-es (especially in the slower peripheral parts). This was abandoned when the incumbent telcos entered the access market in the end of the 1990-es. They went after scale and port densities which resulted in bandwidth accounting features being abandoned across most of the equipment. Cisco broke all of its accounting by introducing CEF, other vendors were not any different.

Over the last 5-6 years most of the features crept back due to demand by business users so technologically the gear is in the same (or better) shape as before the telcos entered the market as far as accounting is concerned. In addition to that new gear from Ellacoya, P-cube and such can do things the old systems were not capable of.

All it will take to get this working now will be people who know how to formulate a viable product and tie this up all the way into billing, CRM and relevant backend systems. Unfortunately there are not that many people left capable of doing it in most ISPs so they prefer the BIG STICK(tm) or the "magic vendor silver bullet". It is easier. It does not require investment. It does not require thinking. It does not require competence. Sad, but true - this reflects the state of the industry.

It is rotten, it sucks and it hates its customers.

Re:well, it only makes sense

[man_of_mr_e]

Well, actually, yes.

In the eyes of the ISP, they're selling you a 3Mb pipe for burst traffic, so your email or web page loads really fast, not so that you can saturate your pipe 24/7. I'm not saying I agree with that, but that's what the ISP has priced things at. The average person uses nowhere near the bandwidth of his connection, and that allows them to charge cheaper rates by overselling.

To put this another way, if everyone saturated their pipe, they would have to charge upwards of 10x for your cable or DSL connection as they currently do.

Re:well, it only makes sense

[Fatal67]

We pay up to $60 per month to have this great thing called broadband, and what do we get?

A best effort burstable connection.

Carriers wanting to restrict VOIP use, throttling Bittorrent traffic, refusing to guarantee any particular level of service, etc

I have only heard of one company blocking competitor's voip, and that was dealt with by the FCC. As far as QoS goes, the service providers would like to offer higher QoS for a fee, which I have no issue with. Unfortunately, some folks are of the opinion that offering a QoS service is a Net Neutrality issue and would like to see legislation passed to prevent service providers from offering such a service.

Re:well, it only makes sense

[Anonymous Coward]

On the other hand, they don't want to charge per GB because they make $40/month from grandmas who only email or browse.

Re:well, it only makes sense

[giorgiofr]

I don't know about your case, but my contract has "NO guarantees whatsoever" written all over it. They insist that no minimum speed is guaranteed, heck they even claim that they could be down 90% of the time and you still would have no case against them.
In other words, I know full well my provider could start throttling and it would be OK because that's what I agreed with. I also know that my provider is not throttle happy.

Re:well, it only makes sense

[Desert Raven]

I wouldn't go so far as to call it fraud, but I would say that the ISPs have to be a LOT more clear about what you're actually paying for.

I lease a rack with a colocation service, I've got a 100base-T connection from my rack, and no doubt I can saturate it 24/7 if I had a mind to. However, my contract clearly states that I am allowed an average usage of X Gb/month, which is significantly lower than 100Mb/sec * 1 month. This is standard for such contracts.

The problem on the consumer side is that to enforce that, every single customer would have to be managed for bandwidth use, and billed accordingly. Believe it or not, that's not so simple. It requires a significant amount of additional hardware, software and manpower to deal with that, plus the additional cost of dealing with customer service in the billing department. For the colo, not so big a deal for a few hundred customers, for a cable company, it would be more of an issue dealing with several tens of thousands of customers. Truth is, I'd bet that a lot of colos don't even bother to monitor bandwidth on individual customers unless they suspect the customer is really hitting the bandwidth hard. I'm willing to bet if I called my provider, they wouldn't be able to tell me what my usage was, because my traffic is low enough as to not be worth monitoring. Collecting those kind of stats on the router definitely hits performance.

I can't say for sure what the best answer is, but ala carte pricing for bandwidth use is not likely to be practical from the ISP's perpective. All I can say is that they should be very up-front about what you are and are not paying for.

Re:well, it only makes sense

[evilviper]

In the eyes of the ISP, they're selling you a 3Mb pipe for burst traffic,

It's a shame their ads and the terms in the contract THEY wrote-up doesn't have any mention of this inconvenient little fact...

The average person uses nowhere near the bandwidth of his connection, and that allows them to charge cheaper rates by overselling.

It also allows them to charge MORE EXPENSIVE rates, as the people using almost no bandwidth are being charged far in excess of what they need. If ISPs would just offer cheaper, lower-speed packages (perhaps with high-speed burst), there would be NO PROBLEM.

When your business model is a problem, you don't start violating your contracts to maintain that model.

Re:well, it only makes sense

[J. T. MacLeod]

Bandwidth accounting isn't necessary.

I work for an ISP. Yes, we oversubscribe. It's the way the business works. We only see problems when many people use their bandwidth *at the same time*.

Moving more data total does not cost any more many than for the electricity to move it. What costs more money is having more available bandwidth so that more can be moved at one time.

We get our bandwith from first-tier providers. They do not charge us by the amount we transfer, but they charge us for the speed of the port. They don't care how much we transfer in total, they only care how much they use at once. We do likewise for our customers, with the exception that we oversubscribe.

Oversubscribing doesn't cause problems as long as there's enough available bandwidth out and the hardware to handle it. Some people expect dedicated bandwidth, and for them there are the options of lower speeds or more money.

I want to see oversubscription come to an end, but I don't see it happening. The dropping price of bandwidth and network equipment is primarily driven by increasing customer demand for higher speeds rather than by an increased number of customers. Unless prices drop as customer demand for higher speed remains static (or at least grows slower than the prices drop), dedicated bandwidth at today's consumer-appropriate speeds and prices isn't going to happen.

Re:well, it only makes sense

[walt-sjc]

I'm not convinced that the only way to handle it is tiered pricing.

Classification of traffic with QoS allows bandwidth utilization to be maximized without degrading interactive / non-bulk traffic. The number of ISP's that actually IMPLEMENT QoS (especially on peering links) is near zero at the moment which would need to change. Now that torrent and other bulk traffic is as high as it is, they need to make these changes. Hell - savvy users have been asking for QoS for YEARS already!

Re:well, it only makes sense

[computational super]

if everyone saturated their pipe, they would have to charge upwards of 10x

I've heard this before, and I'm not sure I buy it. Let's say 3 Mb/s costs $60/month. I see that Cisco's 12000 series router go from 2.5 Gbps to 10 Gbps. Assuming that Cisco is being honest about their bandwidth capabilities (e.g. not lying through their teeth like a broadband service provider), that means that a single low-end Cisco 12000 series router can service about 800 customers (assuming that each one actually saturates the pipe 24 hours a day, 7 days a week), each paying $60/month, which equates to $50,000/month in revenue. Now, Cisco doesn't tell you how much these things cost (or even hint at how much), but lets say one router costs a (ridiculous) million dollars. In well under two years, the provider will have recouped the cost of the router itself. Even if the router lasts only a measly year after that, the provider clears an additional $800,000 on their initial investment to cover paying the admin staff (over three years, probably $600,000), power bills, rent, etc. That's pretty close to break even, if the router cost $1,000,000 and only lasted three years (somebody around here has to know what they cost and how long they last - I'll bet it's a rosier picture than I've painted). So I figure $60/month must cover the actual costs they'd incur if we all used the bandwidth we pay for (which would be almost impossible, even for a die-hard torrent user) - I find it impossible to beleive that they'd need to charge $600/month to turn a profit.

Re:well, it only makes sense

[suitepotato]

I want to see oversubscription come to an end, but I don't see it happening. The dropping price of bandwidth and network equipment is primarily driven by increasing customer demand for higher speeds rather than by an increased number of customers. Unless prices drop as customer demand for higher speed remains static (or at least grows slower than the prices drop), dedicated bandwidth at today's consumer-appropriate speeds and prices isn't going to happen.

Oversubscription is a fact of life. Buy a plane ticket and you have a chance of being bumped because the flight was oversold. Buy a movie ticket and you have a chance of being barred because they hit capacity. Many businesses oversell because they cannot guarantee every sale will actually be used. If they didn't oversell, planes might fly much less than full, movie theaters might play to almost emoty houses, and while that isn't the rule, it happens more often without overselling and that is seen as a loss.

DSL is oversold as well. Most providers have far less than a single OC3 backhaul (usually a single DS3) feeding their DSLAM farm and aggregate bandwidth usage potential far in excess of that. They gamble that not everyone will be on at all hours. I've seen extra DS3 circuits laid in when some customers insisted like schmucks that they should have the right to utilize their pipes to maximum around the clock but it is rare. More often, the company has to obey the laws of economics and cannot lay in another $10K a month connection just because one or two people are hogs. More over, the contract fine print doesn't allow for that kind of usage.

There is such a thing as being a good neighbor and not being a pr*ck. THROTTLE YOURSELF. Set the limits on your P2P clients well below your max, ESPECIALLY UPSTREAM. Don't be a fool and bring your downstream to 98% utilization and then complain to your ISP that mail is timing out. Don't be a childish tool and insist that you are supposed to get unlimited bandwidth. You aren't and the fine print says so. It IS supposed to be burstable. Furthermore, they CANNOT guarantee EVER reaching that maximum speed beyond the first IP hop after you and in the case of DSL there may be a dozen Frame Relay or ATM links underlying it.

Me, I throttle my P2P, I don't run it 24/7/365 but only when I need to get something, and by being good my ISP doesn't whack me for overutilization. I'm paying for a 15Mbpsx2Mbps line and with multipart downloads have many times kicked my aggregate downstream usage to 16.5Mbps and average 14.6Mbps. But I don't do it every waking second. Looking at my firewall graph, my usage is just the bursty sort the average target user's should be.

Re:well, it only makes sense

[Tacvek]

Oversubscription is a fact of life. Buy a plane ticket and you have a chance of being bumped because the flight was oversold. Buy a movie ticket and you have a chance of being barred because they hit capacity. Many businesses oversell because they cannot guarantee every sale will actually be used. If they didn't oversell, planes might fly much less than full, movie theaters might play to almost emoty houses, and while that isn't the rule, it happens more often without overselling and that is seen as a loss.

The prblem with that is that it is unfair and absurd. Airlines should never oversell tickets because if they sell all tickets they have made a full plane's profit. If they really want to avoid flying with empty seats, they can sell standby tickets. The same goes with a movie theatre. Oversubscribing (instead of standby) is unreasonable.

Re:well, it only makes sense

[secolactico]

According to a friend, who used to work at an Airline's checkin counter (or whatever they are called) they also "oversell" luggage capacity. Apparently, airliners are designed assuming that not all passengers are going to be carrying luggage (or maybe only a carry-on). So the incidents of "lost luggages" are usually suitcases and the like that were left behind and sent afterwards.

Of course, she might have been pulling my leg. Can anyone confirm or deny this?

Re:well, it only makes sense

[jmilne]

> lets say one router costs a (ridiculous) million dollars

It's not that ridiculous. In fact, I'd say you're low-balling the cost by quite a bit. And if you want to have redundancy (no one likes having their service disrupted for days while you're waiting for a replacement card), you can start doubling that automatically. Not only that, but you're not accounting for the cost of doing anything with those connections. A local ISP has to buy service from one or more of the Tier 1/2s. Oddly enough, purchasing an OC-192 (that's that 10 Gbps pipe) isn't exactly cheap. Considering most of the world's backbones consist of OC-48s and OC-192s, and considering that the backbone providers don't want to oversaturate their own lines, they charge the local guys a heck of a lot for that OC-192. No local ISP could ever afford to purchase an OC-192 just for 800 users, and no backbone provider could ever support it as well.

The pricing worked rather well when people were only downloading relatively small files periodically. As long as traffic is bursty, that is. It's when people start downloading large files (like movies) constantly where everything goes awry. If you honestly expect to use that cable providers 5 Mbps down, 1 Mbps up service at $60/mo, when they in turn have to purchase 4 T1 circuits at ~$500/mo to support you, you deserve the crappy service you get. If you want to push that much traffic constantly, buy the T1s yourself.

Re:well, it only makes sense

[laffer1]

Comcast is very clear that you are not guaranteed any speed. I've seen similar wording with charter and SBC when I had DSL. Comcast also has busness packages which are designed for full use. I pay $160 a month to have the ability to host servers and do almost anything with my connection. I get 6 ips including the router and 8Mb/1Mb speeds. (in reality i don't get full upstream and about 6Mb/s down)

Even hosting websites, email, an ET server, and dns I still don't max out my connection even half the time.

Many other uses

[Wind_Walker]

However, BitTorrent is an extremely important tool that has many uses other than what everyone assumes it is good for, namely movie piracy.

I agree wholeheartedly. There's pornography, music piracy, video game piracy, and pornography.

Re:Many other uses

[fistfullast33l]

You forgot pornography.

Re:Many other uses

[Anonymous Coward]

Are you saying you don't use BitTorrent to download ISO images of Linux

Hell no. I'm a BSD user, you know.

Many uses other than Movie Piracy

[neonprimetime]

many uses other than what everyone assumes it is good for, namely movie piracy.

- Game Demos
- Software updates / upgrades
- Free / Legal Videos

Re:Many uses other than Movie Piracy

[Guysmiley777]

WOW patches... god damn sucks that my ISP tries to hamstring torrent traffic. I get 10-15 kB/s on a 3 megabit cable modem when patching. I usually wait until someone hosts the patch, then download it via HTTP.

Re:Many uses other than Movie Piracy

[jimmypw]

In response - I was able to contact my ISP and mentioned this problem. They then put me on a service that had no blocked or throttled ports but also made me agree to accept any civil proceedings brought against my IP address.

Re:Many uses other than Movie Piracy

[Anonymous Coward]

Dear Sir,

Your IP address has slandered me, insulted me and publically humiliated me. Additionally your IP address has repeatedly "stolen" PDF versions of 'Goat Shaving Monthly' and 'Creamed Midgets Weekly' from my shared download folder. Are you as you claim, prepared to take full legal responsibility for the unfortunate actions undertaken by your IP address?

Sincerely,

AC

Connections

[Doc Ruby]

"in order to ensure connectivity for everyone"

No, that's in order to continue selling people bandwidth they couldn't deliver, known to ISPs as "statistical oversubscription". Then when we want to get what we paid for, they take it away entirely. Unless you're watching the telco's own IPTV, which somehow has as much bandwidth as they need to sell it to you, for an additional charge.

Blocking competitive services to support ripoff monopoly business models is the reason telcos and other big ISPs hate Net Neutrality.

Re:Connections

[Dr Caleb]

"If you buy a business package, you actually GET the bandwidth sold to you. "

Not from Shaw Cable in Western Canada. I had their 'Business' package and still had unencrypted torrent traffic throttled, negating the speed increase. Although they denied throtteling it, my speed went from 80k/s max on *every* stream, to 500k/s one some streams (encrypted)

Re:Connections

[Doc Ruby]

It's much more likely to enforce "common carriage" rules on networks than to force them to sell off networks to competitors. The people are never going to get ownership returns on our investment in WAN infrastructure. The best, and minimum, we can expect is that the networks we paid to build will be run "the American Way": equal access to opportunity for everyone, without anticompetitive monopoly abuse by the "gatekeepers".

ATT is doing the same

[Anonymous Coward]

with teh Telephone System, returning the advantage to the communication providers
by filtering the words Cocaine , Heroin, Ganja, LSD, Skunk, PCP, Speed, Crystal Meth
as they are used by people using the telephone system to conduct illegal conversations

filter my torrents and i will sue you for NOT filtering childporn
if you want to give up common carrier thats fine, but be aware YOU WILL be held to account for anything illegal i find on YOUR network

But I thought SPAM was 80% of traffic?

[NotQuiteReal]

Spam + Torrent = %160, plus whatever "real" traffic the net has...

Wow, stunning efficiency, or bad statistics.

Re:But I thought SPAM was 80% of traffic?

[bunions]

I think spam goes in either a different pipe or a truck, I'm not 100% sure of how it works though.

Re:But I thought SPAM was 80% of traffic?

[Anonymous Coward]

No No, its not a pipe you see it a series of tubes...and if someone sends you an internet it might get stuck in a tube behind a different internet....

Or at least thats how I have heard it works, not positive.

Re:But I thought SPAM was 80% of traffic?

[Watson Ladd]

80% of email traffic. I hate newspaper reporters who drop crucial adjectives. Of course, email is the internet according to Ted Stevens. And since he's the commite head, he knows what he is talking about.

Re:But I thought SPAM was 80% of traffic?

[niceone]

I think spam goes in either a different pipe or a truck, I'm not 100% sure of how it works though

I'm 80% sure of how it works, how sure are you? Between us we're probably completely certain.

Re:Question

[Xemu]

Easy. All traffic is slowed down by default. If the traffic is digitally signed by a Microsoft trusted computing device then it's allowed to travel faster through the pipes. All other traffic is slow pr0n.

Not quite...

[Poromenos1]

Well, to their defense, if they didn't oversell their prices would be quite higher.

Re:Not quite...

[Anonymous Coward]

Well, to their defense, if they didn't oversell their prices would be quite higher.

Obviously true, but would you actually accept that for any other commodity?

"Hey, this 'pint' of milk only has half a pint in it"

"Yeah, well you get the theoretical pint capacity but if you actually got the milk, our prices would be quite higher!"

Seems like a straightforwards case of fraud.

Re:Not quite...

[interiot]

Or they can just be nicer about their bandwidth caps... don't advertise "unlimited bandwidth", and if a customer gets near their monthly cap, then slow them down to 64kbps down or something like that. If a customer only uses BitTorrent twice a month, why does the ISP have to go to the trouble of trying to detect an encrypted connection and slowing it down?

Re:Not quite...

[interiot]

But there are bandwidth caps, you just don't often find out about them until the ISP cuts off your account or asks you to upgrade. And while an honor system might be an optimal solution (if it was practical), there are users out there who don't necessarily know how much bandwidth they'll eat up if they leave a BitTorrent application running the whole month.

Why sell the bandwidth then?

[Rearden82]

They shouldn't be allowed to advertise (and charge a premium for) 3-5+ mbps service if they're going to actively prevent their customers from using it.

If car manufacturers operated like ISPs, they would sell 300 horsepower cars with shoddy transmissions, then limit them to 150hp so they wouldn't have to deal with the warranty repairs.

"war"?

[NineNine]

"However, those who feel this all amounts to an imminent war between the users and the ISPs over BitTorrent... "

A war? You gotta be crazy. If my ISP doesn't provide me what I'm paying for then I'm either dumping them or suing. It's that simple. There's not going to be an "war" over my ISP usage at my home or my business. I'm going to get what I pay for, or they can speak with my attorney (and yes, I do use my attorney for little stuff like this).

To the people who just have a home ISP and may not have much choice, I say: don't worry about it. Somebody will come in to provide the service eventually. Competition ensures that it'll happen. With wireless getting a little bit more useful every day, I think that we'll soon have some competition amongst ISP's again, soon.

Re:"war"?

[Bassman59]

To the people who just have a home ISP and may not have much choice, I say: don't worry about it. Somebody will come in to provide the service eventually. Competition ensures that it'll happen. With wireless getting a little bit more useful every day, I think that we'll soon have some competition amongst ISP's again, soon.

Competition? Surely, you jest. Unless, of course, you mean "Competition between two subsidized monopolies," namely the local cable company and the local telco. Some choice.

As Lily Tomlin's telephone operator character liked to say, "We're the telephone company. We don't care. We don't have to."

Bittorrent will fight back.

[Skynet]

I suppose the only way they can really do this is by analyzing the high level protocol transactions and by keeping tabs on particular IPs and their behaviors. Pretty flimsy.

All of this could probably be pretty easily foiled by having Bittorrent mask what it's doing by sending noise once in a while to throw these tools off.

Re:Bittorrent will fight back.

[CodeBuster]

All of this could probably be pretty easily foiled by having Bittorrent mask what it's doing by sending noise once in a while to throw these tools off.

This is actually a common feature in many cryptosystems which serves to prevent a successful cryptanalysis via "cribs" or short passages of known plaintext within the cipher text, especially at known location such as the start of the message (the Germans made this mistake with their Enigma traffic during WWII for example with standard message headers on their daily weather reports to the U-Boat flotillas). If the protocol were modified to introduce random segments of padding (i.e. junk) into the packets then cryptanalysis via cribbing would most probably be rendered impractical.

Illegal?

[BloodyIron]

Isnt it illegal to read any part of encrypted data accross the internet? (with certain exceptions, ie: NSA actions/warrants, etc)

Re:Illegal?

[99BottlesOfBeerInMyF]

Isnt it illegal to read any part of encrypted data accross the internet?

Probably not, but they aren't "reading" data in any case. They're just looking at the encrypted streams and figuring out, based upon the way the traffic flows, the ports, etc. that it is bittorrent traffic. Of course engineers can just make bittorrent traffic mimic other, legitimate traffic more closely to make it impossible to distinguish between them.

Ever notice that whole lot of crap runs on port 80 these days? The reason is that ISPs and maintainers of firewalls have turned off the rest of the internet under the assumption that it will stop the traffic they don't like. Really it just squished everything into one place and made it harder to properly administer.

Um, mirror?

[tomstdenis]

Why don't ISPs that worry about their net usage outside their network just mirror shit?

Would it be really hard to throw together a 1TB file store with the latest patches, demos, ISOs and the like?

That way the customers can get stuff inside the network and the ISP doesn't have to worry about upstream net usage.

OMG it's like I'm smart and all.

Tom

Stunned...but not

[svunt]

This is funny...last month, I downloaded one linux distro via torrent, it was a dvd iso, can't remember the file size, let's say 4.5GB for argument. The other squillion terabytes I grabbed all came from my ISP's own news server, about a zillion hours of not-so-legal content, all provided at full speed by the guys who'd like to throttle my legal torrent traffic? If ISPs were that concerned about traffic, they'd close some of the zombie hosts on their own networks sending out billions of spam emails a day.

Re:Stunned...but not

[Sloppy]

The other squillion terabytes I grabbed all came from my ISP's own news server, about a zillion hours of not-so-legal content, all provided at full speed by the guys who'd like to throttle my legal torrent traffic?

Waitaminute, you're saying this like it's a bad thing or ironic or something. This is the way it should be. If the ISPs would store things on their own network (whether it's a Squid cache or a Usenet spool doesn't matter) that would be fucking awesome. Then they only pay once to move it over their expensive pipes (and likewise, only slow down the backbones once), and it doesn't matter how many of their customers download it, because that part is essentially "free." The internet as a whole would benefit.

Every ISP should have a news server and web cache, and encourage their users to use them. It's not like disk space is expensive. It doesn't need to be reliable, either. Use a bunch of the cheapest Maxtors you can find, and RAID0 'em. Or just make it all swap and serve out of a 64-bit-address-space tmpfs. ;-)

Likewise, it would be neat if bittorrent or Gnutella or something like it, would peer with users on earby networks first, instead of just anyone. If a bunch of an ISP's users are all passing lots of packets to each other the the ISP has little reason to throttle. I'm kind of surprised that I haven't heard of ISPs working on this, because they (moreso than the users) are the ones who would most benefit from it.

Has to be done

[realmolo]

Look, I use Bittorrent and it's great. But I also run an ISP.

The thing is, bandwidth isn't cheap. People bitch that ISPs "oversubscribe", and that we can't really deliver our advertised bandwidth to everyone all of the time. This is true, but how do you think we manage to sell people 5Mb connections for $40/month? Do you know how much 5Mb of bandwidth costs and ISP? It's a lot more than $40. In the market I'm in, we pay THOUSANDS of dollars for that much bandwidth.

The real problem is that bandwidth is too expensive in this country, thanks to the likes of AT&T and MCI and all the other big players. They've got tons of unused fiber lying around, and it costs them next-to-nothing to use it, but it still costs the end-user (in this case, the ISP) a hell of a lot of cash.

Re:Has to be done

[Pantero Blanco]

The thing is, bandwidth isn't cheap. People bitch that ISPs "oversubscribe", and that we can't really deliver our advertised bandwidth to everyone all of the time. This is true, but how do you think we manage to sell people 5Mb connections for $40/month? Do you know how much 5Mb of bandwidth costs and ISP? It's a lot more than $40. In the market I'm in, we pay THOUSANDS of dollars for that much bandwidth."

No, it doesn't "have to be done". You could just advertise what you can actually deliver, and anything a customer happens to get above that is gravy. Right now, you "manage to sell" people 5Mb connections for $40 a month in the same way that the guy at the corner "manages to sell" Rolex watches for ten dollars a shot.

Re:Has to be done

[Anonymous Coward]

People bitch that ISPs "oversubscribe", and that we can't really deliver our advertised bandwidth to everyone all of the time. This is true, but how do you think we manage to sell people 5Mb connections for $40/month?

You don't. Like you just said, you lie about how much bandwidth you have available. You don't thikn people will be interested in paying for what you can really provide them with for $40, so you pretend you can provide more than you can afford to. You're a conman. A fraudster. A common crook. HTH.

Re:Has to be done

[AnotherBlackHat]

Do you know how much 5Mb of bandwidth costs and ISP? It's a lot more than $40. In the market I'm in, we pay THOUSANDS of dollars for that much bandwidth.

I suggest you shop around then, 'cause I can buy 100Mbps of transit for just under $3000 a month.

Look at the complaints here on Slashdot.
Most of them are complaining about ISPs lying about the service they sell.
If you can't accommodate bit torrent that's OK, just sell an honest service plan that doesn't appeal to people using bit torrent, but does appeal to people who just surf.
For example, 5Mbps for 100 hours a month (burst), plus 128Kbps continuous.

-- Should you believe authority without question?

Re:Has to be done

[silas_moeckel]

Funny I install big networks for a living, 3 megs a sec is 90 bucks a month from cogent (yea I know they have issues and yes thats ISP rate not end user ($30)) now granted you have to be looking for at least 100bt if not a gigabit ethernet over fiber handoff. At the low end a DS3 can be hand with bandwith for 5k thats a little over $110 per megabit and froma major carrier (I have done those with MCI and AT&T) Bandwith gets cheaper and cheaper as you buy more and more, getting into overly long contracts and buying incrementaly rather than with a strategic plan gets ISP's into bad agreements and pricing plans. Realy bittorrent should be a boon to larger ISP's as it will allow the ratio's needed to get into statement free peering relationships.

The levels of oversubscription on some ISP's are just insane my previous cable company had a 512kbs cap per user (90 homes per channel not over subscribed) and had problem providing that to there head end at peak times. ISP's are going to 100x ratios and investing mroe in help desk and fixes than just getting more bandwith.

Re:Has to be done

[evilviper]

People bitch that ISPs "oversubscribe", and that we can't really deliver our advertised bandwidth to everyone all of the time. This is true, but how do you think we manage to sell people 5Mb connections for $40/month? Do you know how much 5Mb of bandwidth costs and ISP?

I know it sounds insenitive, but it really needs to be said: "It's not my job to make sure your business model turns a profit."

Your the one in control. You write-up the contract any way you wish, and the customers' only choice is to accept or refuse. If you aren't able to provide 5Mbit connections, then clearly make it a point in your contract that you're limiting them to a maximum ammount of throughput, or something similar.

Honor your contracts, don't complain that you can't. Making contracts "on the margin," so to speak, gets lots of people thrown in prison all the time, when things don't go their way.

What's more... singling out bittorrent, or P2P in general, is insane. The same things can be done with http, ftp, etc. If you're going to restrict traffic, at least do it in a sane way, which applies to ALL the bits, and doesn't unfairly penalize one protocol/technology over another.

Re:Has to be done

[Hope Thelps]

don't blame the ISP here.

Why not? They're the ones who are lying about the level of bandwidth they can provide.

Make people pay for their actual usage and it will all be fair.

How do you suggest that I make people do that? If you mean that a solution would be for the ISPs to be honest with people and specify a price based on usage and then apply that rate then I agree; that would be fair. But that brings us back to the problem. The ISPs aren't doing that. They're pretending to have more bandwidth available than they really have and then they're complaining when customers who have paid for it try to use it.

They think that people won't be willing to pay usage rates, so instead the ISPs lie about what they can afford to offer.

Re:First

[Cpt_Kirks]

Not "first".

"Fist".

Up you ass.

About elbow deep.

(let's see, will this one be modded: Troll? Flamebait? Off-Topic? The suspense is KILLING ME!)

What new ATT SBC does

[shawn443]

I have noticed that once the upload stats get to about 10 gig or so my dynamic ip expires about every 2 hours. Before I started using btdownloadcurses my ip would change about once every two weeks. Remote access in terms of my dynamic ip address was rarely a problem. Granted this is only an observation, yet I still assume categories of customers are made by upload stats. This caused me to script ipshow. ATT, go screw yourself and your "sticky ips", I am not running ebay here, I just want access to my computers.

Espically for Linux

[Sycraft-fu]

I remember when Knoppix 5 came out. The official mirrors weren't carrying it yet, it was offloaded to other sites to try and get the feeding frenzy over with. So I downloaded it at the request of my boss and then left my computer to seed for the weekend. I served out 1.2TB in 48 hours. Would have been higher too, but I was capping my upstream. And I was only one of hundreds of seeders (though in fairness I was the top seeder).

I just don't see how else a not-for-profit group is going to get fast distribution of something that big for cheap. If you look at web hosting you find that bandwidth of that order is not at all cheap. However, BT let us all share the load a little.

I'm sure people do sue it for illegal purposes but I tell ya what, it has made getting free legal software so much easier. Gone are the days of waiting around on a slow ass FTP that seems like it's being run out of some guy's broom closet (which is probably where it is being run). I find on most Linux torrents I can get 30+mbits/sec no problem.

Two Choices

[Nom du Keyboard]

1: Shift to new encryption method.

2: Sue them under the DMCA for reverse-engineering and breaking the technological protection method used to protect your content.

Use either, or both, as appropriate.

Re:Two Choices

[Mr. Protocol]

Sadly, I doubt that changing to a new encryption method will work: it is highly unlikely that this product is decrypting encrypted BitTorrent packets. It is almost certain that it detects BitTorrent packets by size, number and frequency. If you have 600 open TCP connections, which are constantly shifting, and all of your inbound packets are the same size, then these are BitTorrent packets.

And, in the end, the ISP doesn't care if they're BitTorrent packets or not. If you're filling your inbound pipe for days on end, then throttling whatever it is that you're doing is a good thing, from their perspective.

Re:Two Choices

[Wildclaw]

"And, in the end, the ISP doesn't care if they're BitTorrent packets or not. If you're filling your inbound pipe for days on end, then throttling whatever it is that you're doing is a good thing, from their perspective."

And that is the right kind of throttling. ISPs should not care what is in the packets. If I have been using more than my share of bandwidth, throttle me. However, don't peek at my packets and decide that some of my traffic is worth less than the other. The ISP may give me x bandwidth for bulk traffic and y bandwidth for priority traffic, but it isn't the job of ISP to decide what traffic is bulk and what is priority.

That is what net neutrality is really about.

While they are largely at fault

[Sycraft-fu]

You also have to consider that consumers want things real cheap, often cheaper than is affordable. Big lines (like OC lines) cost a lot of money. So you need to have a good number of subscribers per line to make it work, if you are to charge those people a low amount. That means that bandwidth can be scarce.

One option people have is to just get better service. I personally went with Speakeasy. They don't block or throttle your connection in any way (they claim they don't, and I haven't detected any). You can host servers, whatever you like. However, it's more pricey than lower grade service. I drop about $130/month to get 6m/768k DSL with 8 static IPs. But, I've never had it fail to work at the highest speeds, and they are true to their word, I do a TON of upstream with those servers and I've never heard a peep out of them or seen my connection throttled at all.

Net access is just another area where you get what you pay for. Sure, I could offer people 100mbit net access for $20/month and just lay ethernet to their houses (we are assuming I had the permits here). However at that price, I couldn't guarantee 100mbits of upstream for each subscriber. Hell I'd be lucky to get 10mbits of upstream for all subscribers.

Re:Bittorrent is shit and you too are shit Zonk

[Nom du Keyboard]

next time you're faced with a decision to use either DSL or Cable, buy the one that doesn't give your neighbors the ability to waste all the bandwidth.

You are assuming that the DSL ISPs aren't throttling traffic at the higher level of their network. You are wrong in that assumption. DSL is no panacea to cable oversubscription and traffic-shaping.

Let me sum up the counter arguments

[Travoltus]

Customers getting what they paid for? Are you nuts?! That's communism! You pay for 6mbps per second, you should be happy with 768kbps. People having the freedom to use bit torrent and the privacy of encryption, what kind of collectivi-er, confiscationli-er, what are you, one of those SWARTHY PEOPLE?!!!

compare to land

[dragonsomnolent]

I saw other posts saying the same thing about ISP's hoping that you don't use all of your available bandwidth, and overselling the service, and my question is this:

If I buy say 25 acres of land, and I sell 1 acre parcels of this land, normally valued at 1,000 dollars per acre to 50 people at 750 dollars per acre (to give a good deal and sell my land), in the hope that they don't use it all, how long do I have before I go to jail, and how much of a jackass am I for counting on the fact that no one will try to use thier full acre?

Re:compare to land

[IDontAgreeWithYou]

uh... how about we ban analogies completely from /. Who's with me?!!

In the meantime, I will point out that the flaw with this particular analogy is comparing a service (broadband) to a physical object (an acre of land). You can oversell a service, but it doesn't work with physical objects. People tend to want to get their hands on a physical object and it becomes apparent very quickly that it's been oversold. Most of the time, users will be surfing the web or checking email. They won't be using their full bandwidth. When they do occasionally use their full bandwidth, most likely it will be available.

...seriously, who's with me?!!

Re:compare to land

[kthejoker]

It may be abstract and not quite as apt, but clearly the pipes and the elctrons being served are discrete units that can be measured for each user. So yes, there is a physical object here - it's just not as easy to see as an acre of land.

A better analogy (and a car-related one at that) is an actual highway.

You build a 4 lane private tollway between two specific points. You promise high speeds for toll access. Then you oversell access.

The thing in question here is sentence 2: "promise high speeds." What does that mean? Clearly we can quantify that.

And guess what? In our ISP service contracts, we've quantified it, too. It's fairly simple; either

a)charge me bit-for-bit and quit throttling
b) up everyone's price until you're not overselling any more because of lower demand
c) offer tiered pricing for higher bandwidth users. That's great for me; I don't mind slower speeds, so I can save me some dough.

Re:compare to land

[Maxo-Texas]

That's a pretty atrocious analogy.

Hmm. Can I find one.

It's about sharing something.

Okay.. here we go. I'll use a riding lawnmower analogy.

The ISP's leases use of a riding lawnmower for a year $1,000. The leasing company agrees that up to 5 days a year, they can use two lawnmowers without extra charge and three lawnmowers for $10 per day.

They sell 50 people the right to use a lawnmower for $25 each (pocketing a nice $250 profit).

They reasonably expect that people are going to mow about 2 hours a day once every 2 weeks- some will mow for 2 hours a day every week and some will mow for 2 hours a day once a month. Given 365 days a year, there should rarely be a line for the lawnmower. And when there is they have a bit of extra capacity.

Now- someone figures out that the lawnmower can be used to drive to work with while their car is in the shop.
Someone else figures out that they can run a small busines mowing people's lawns with it.
Another person borrows it and then loans it out to his 5 best friends to mow their lawns with too.

The business model that was going to work doesn't work any more. Because every day people are using 7 lawnmowers all day. The ISP is now paying $60 per day over what they thought they would pay. The rules have changed.

---

Don't get me wrong- I torrent things too. I know at some point they are going to charge per megabyte (or gigabyte) downloaded. This is a very temporary window where they did not know how people would use their services. Server accounts have always taken bandwidth into account.

I expect in the next few years that we will see things like "4 gigs a month and then $1.00 per 10 gigs a month" and then the ISP's will compete on price. These will get higher as bandwidth grows (USA is pathetic at 9mbps when Japan/korea/etc. have something like 100mpbs).

Smarter caching could prevent a lot of this.

Re:compare to land

[dragonsomnolent]

My fair share is what I paid for. I don't seed torrents, or abuse my pipes. I am the customer that my cable company loves, I don't use all my bandwidth 24/7. We have it so my wife can play neopets, and I can research things without waiting long for pages to load. Most of my traffic is web/e-mail. If I go to a speed test, I regularly post well, but it takes days (yes days) to download a single cd iso for a distro from a torrent. Why? I am paying for 1.5 mb/s. It is in my agreement that I had when signing up for service. It is a gauranteed speed. If they can't meet that, then either charge me more, or don't gaurantee it. Yes land is product, but would my analogy hold any less true if I had used a service? How does that change the fact that they are over selling the service? And isn't a service just a product without a physical medium? If I were a temp worker, I would be the temp agency's product, no?

Re:compare to land

[IgnoramusMaximus]

In the meantime, I will point out that the flaw with this particular analogy is comparing a service (broadband) to a physical object (an acre of land)

That does not alter the validity of parent's analogy. Consider a car mechanic, who being a similar jackass, sells you a coupon for "tire change in 10 minutes - guaranteed!" (clearly a "service"), obviously hoping that all of his customers ... err ... marks, will not show up at the same time. But if they do, he is in the same boat the ISP is: he sold something he could not deliver, i.e. he lied, cheated, and ripped the consumer off.

Sometimes analogies do work, because Internet is not some new magical, never before experienced thing from the perspective of mercantile trade. It simply fits into the ages old criterion of "service", rules of which have been long established as are all the different ways thieves and con-men have tried to abuse those rules.

Re:compare to land

[Alan Hicks]

Perhaps a better analogy can be found in the airline industry (also a service). Historically airlines have routinely oversold seating because more often than not it works out for them. Some people will cancel the fight, some people won't show up for the flight, and sometimes they won't be able to sell all the seats in first class and can bump overbooked coach passengers to first class. In the event that they can't put you on your purchased flight, they will put you on the next one, or refund your ticket. Either way generally sucks for you, but you're at their mercy. So there is at least one industry that has been overselling a service for a very long time.

Re:compare to land

[IgnoramusMaximus]

It may be abstract and not quite as apt, but clearly the pipes and the elctrons being served are discrete units that can be measured for each user. So yes, there is a physical object here - it's just not as easy to see as an acre of land.

This made me realize that there is even a better way of visualisng the problem: think traditional telephone companies. They also provide, for a fixed monthly fee, unlimited access to the telephone network. If they operated on the same principle as the ISPs, you would get nothing but busy signals if more then 0.1% of people decided to call each other. Furthermore, if their response to the problem was like that of the ISPs, you would see people's calls being monitored and those made by teenagers would be terminated prematurely, because they make the system too busy for Grandma to call her grandkids. In other words: total nonsense. Instead the telcos of old did the only sane thing: expanded the switching capability until the odds of the system reaching its capacity were so small as not to impede its normal use.

ISPs simply believe that no sane rules apply to them because they operate in this magical, fantastic, cosmic, new wonder medium of Internet. Its time someone hit them with a sizeable clue bat and made their noses contact the firm ground of common sense, violently.

Re:compare to land

[IgnoramusMaximus]

Some people will cancel the fight, some people won't show up for the flight, and sometimes they won't be able to sell all the seats in first class and can bump overbooked coach passengers to first class. In the event that they can't put you on your purchased flight, they will put you on the next one, or refund your ticket.

The key word being refund. Also, airlines have many other reasons for bumping flights, such as weather and what not. In other words, while they can be sleazy, the level of their machinations is insignificant to what the crooks, otherwise known as the ISPs, are up to.

Re:compare to land

[Anonymous Coward]

The Small cable ISP I work for does not have this problem. We have burstable bulk internet lines that give us as much bandwidth as we need. We pay for bandwidth by the terabyte. We Prioritise traffic at our packet shaper like so:

1. VOIP traffic
2. game traffic
3. HTTP + e-mail
4. everything else.

Every customre has a contract that clearly states: you have access to X amount of GB's per month. Download what you want in accordance to local laws. If you go over X amount of GB's per month you will be charged overage per additional GB's.

That way we dont need to throttle traffic and the customer knows exactly what they are paying for. So little johney goes to town on bit torrent, everyone else is still fast.

Re:compare to land

[HiThere]

Your solution is a fair one, so I don't expect it to be chosen. (I've become a bit cynical about corporate ethics...as well as the ethics of many individual people.)

Re:compare to land

[masterzora]

You're right... it was missing a car.

Re:compare to land

[CopaceticOpus]

The main problem with this second analogy is that the service is not advertised this way. It's more like they rent you the lawnmower for a year. You take it home and start cutting your lawn, which happens to be a rather large lawn. An hour later, Guido shows up to take your lawnmower and bring it to another customer, because they figure an hour should be plenty of time to cut your lawn. But you didn't sign up for an hour a week, you signed up for a lawnmower for a year!

Re:compare to land

[Yaotzin]

I don't really understand what the problem is. Here in Sweden you can get a 24 Mbit/s (down) for about $40 USD a month and that is without bandwidth throttling.

Re:compare to land

[Kadin2048]

I had the same idea a while back when I was reading about how bad the folks in Australia get hit for broadband. If you think the situation sucks here in the U.S., they really get screwed -- it's almost impossile to get an uncapped (transfer) account there at all. In a situation like that, it seems to me like it would make sense to have two distinct tiers of traffic: local traffic that wasn't going to leave the country (and thus wouldn't have to go through expensive undersea cables and be subject to peering agreements), and international traffic. The latter is what's expensive, the former ought to be free or close to free.

Rather than fighting bittorrent, an ISP like Comcast could just put a cap on the traffic that you could send through to other networks (and publish what the limits are, in terms of burst versus constant throughput, etc.), and then give you your full unthrottled connection to other Comcast subscribers, because this really doesn't cost them anything. Their network ought to be capable of letting someone basically saturate their connection from one node to another node on the same subnet, and with some intelligent caching, they could keep a lot of the BT traffic here.

If they set up the incentive structure correctly, they could probably reduce the load at critical points on their network due to BT traffic, while giving end-users (both heavy downloaders and "burst" users) a better overall experience. They would also eliminate the incentive to obfuscute BT traffic and end the cat-and-mouse game that seems inevitable under the current system.

Re:compare to land

[IgnoramusMaximus]

In a situation like that, it seems to me like it would make sense to have two distinct tiers of traffic: local traffic that wasn't going to leave the country (and thus wouldn't have to go through expensive undersea cables and be subject to peering agreements), and international traffic. The latter is what's expensive, the former ought to be free or close to free

Trouble is, as I explained to the other poster above, that in case of the Internet, unlike the telephone network, the bulk of normal usage is non-local. A huge number of sites being accessed by Australians is overseas, in USA primarily. Google, Yahoo, MSN, Blogger, MySpace, etc and so on. While some of those companies can have local "beachheads" with some sort of caching and data replication capabilities, and while the ISPs can cache some contents, this is only a very partial remedy because of the highly dynamic nature of the content of those sites. Also ISPs can get in legal hot water for caching, running afoul various wacko laws, such as anti-"piracy" or anti-"hate"-speech ones.

Again, the only way ouy of this jam is for ISPs to do what they were supposed to do in the first place: invest in the backbone infrastructure until it is capable of handling the bandwith they sold to their customers.

Re:compare to land

[HeroreV]

If I buy 25 cars and charge 50 people for the ability to come by and use a car at any time, how long do I have before I go to jail? And how much of a jackass am I for counting on the fact that there will never be more than 25 people wanting to use a car at any one time?

Re:compare to land

[99BottlesOfBeerInMyF]

They sell 50 people the right to use a lawnmower for $25 each (pocketing a nice $250 profit)... The business model that was going to work doesn't work any more. Because every day people are using 7 lawnmowers all day. The ISP is now paying $60 per day over what they thought they would pay. The rules have changed.

You're forgetting one thing. The government bought the company the lawnmowers in the first place with people's tax dollars, after the company promised to supply enough lawn mowers for everyone, which they failed to do and pocketed the cash. The US government paid hundreds of billions of dollars in subsidies to ISPs who promised reliable, cheap, high-speed access to the home, just like they have in Europe and Asia.

Re:compare to land

[Big_Al_B]

think traditional telephone companies.

Okay...

They also provide, for a fixed monthly fee, unlimited access to the telephone network.

Hardly. They offer very limited access to the telephone network--you can make and receive phone calls with a limited finite set of optional features such as caller-id and voicemail. They offer unlimited use of that application within, well, limits, including geographical toll boundries and pay-per-use products such as directory assistance and three-way calling.

IP networks offer an ever-expanding variety of access, limited only by the contractual terms of service that each customer agrees to at the time of purchase. In practice, those terms are most often loosely enforced, if at all, and usually only in response to some operational problem caused by a violation. New network applications are developed and widely adopted as time passes.

If they operated on the same principle as the ISPs, you would get nothing but busy signals if more then 0.1% of people decided to call each other.

Actually, telephony capacity is engineered to some threshold of dropped calls per 100 at the network's "busy hour". This threshold is either dictated by regulatory bodies or is left to the telco. Either way, few--if any--telcos build to "zero drops per 100 at busy hour".

Telephony networks are a smidge easier to engineer from a capacity perspective because there's fewer variables to address. A PSTN/TDM phone call takes a discrete unit of bandwidth per call, either 56K or 64K depending on the underlying transport technology. The only variables are start time and duration. Erlang modeling, based on queue theory, addresses this quite well; it isolates start time by normalizing duration to 3600 seconds/call and provides useful, realistic measurements.

IP networks, though, have difficult to model traffic flows with packets of varying size, varying latency from node to node--and from packet to packet within the flow all transmitted at different start times and with different durations. This is only exacerbated by the variety of applications on the network. Variables are nearly impossible to isolate (practically) and capacity planning is more reliant on utilization trend analysis rather than proactive modeling.

As an example, the network I help operate sells ISP service over DSL lines provided by a local carrier. We have a meager 300 or so customers that have DSL products that range from 384K down to 3M down. Let's normalize all of them to 1M to make the math straightforward. We pay our upstream providers about $30/M each month for connectivity. So you would have me pay ($30/M x 300M)==$9000 per month to support those customers. That's more than I currently charge agreggated across the whole group for DSL service.

Now, in reality, what is the actual average utilization for those 300 customers? Three megabits per second on average for the whole group. And that's just the amount on the direct circuit from the local carrier--not the amount from those customers that use my upstreams. Around 14% of their traffic goes to other customers on my network, so only 2.6M or so actually goes upstream. That's around $75/M monthly on average for upstream. Now I can afford to charge what I do, and still provide email, personal webpages, news, DNS, etc, plus staff 3 tiers of support. BTW, peak utilization for these customers doesn't exceed 5M 99.999% of the time.

Also in reality, I also have thousands of T1 (1.5 Mbps), dozens of DS3 (45 Mbps), six OC3 (155 Mbps) customers and 14 GigE (1000 Mbps) customers. My peak daily upstream utilization is around 800 Mbps for all customers combined. It's never spiked above 924 Mbps, including DoS attacks.

I price and operate my services according to that reality, not magic or fantasy. If you feel that means I lack common sense, then I submit that common sense...isn't.

Re:compare to land

[Blkdeath]

a)charge me bit-for-bit and quit throttling
b) up everyone's price until you're not overselling any more because of lower demand
c) offer tiered pricing for higher bandwidth users. That's great for me; I don't mind slower speeds, so I can save me some dough.

Oh no no no! ISPs tried that. The people got up in arms because they're paying a premium rate (ha!) for their "broadband" connection, then when they find out they have to pay when they monopolize the service (3% of users using 80% of the bandwidth, for example) they cried about how unfair it was!

Broadband Internet access made surfing the web and streaming music and medium-quality video easier because you didn't have to wait forever for things to queue. It was offered initially as a burstable service, much like dial-up was (eg. 100 28.8k modems sharing a single 1.5MBit T1), however with today's modern multiple-GB downloads (mostly pirate software, movies and porn I'd wager), high quality streaming full-screen video, VOIP services, etc. it's a lot more noticeable.

How would consumers react if they were charged market-value (even wholesale) for the bandwidth they consumed?

Cut the ISPs some slack here. They entered into a free-market, improved their services, then the free market changed so drastically they couldn't adapt without serious backlash.

If you use your 3.0MBit connection at 90% capacity for 100% of the month, you should expect to have to pay more than the casual websurfer, or you should lease a line from your telco and see how THAT bill feels.

Re:compare to land

[IgnoramusMaximus]

Hardly. They offer very limited access to the telephone network--you can make and receive phone calls with a limited finite set of optional features such as caller-id and voicemail. They offer unlimited use of that application within, well, limits, including geographical toll boundries and pay-per-use products such as directory assistance and three-way calling.

The unlimited access is for the bulk, most common, practical part of the service. All of the other features are optional and non-essential to the basic function of the telephone network. The "limits" are such that they do not interfere in any conceivable practical use of the system, even going as far as including many 24/7 dialup connections to Internet.

The geographical area restrictions are for the far less common usage, and historically originate from the fact that various telephone companies used to be restricted only to the sets of wires within their corresponding geographical areas, thus nessecitating peering agreements and fees/contracts associated with those. At least that was the original excuse.

IP networks offer an ever-expanding variety of access, limited only by the contractual terms of service that each customer agrees to at the time of purchase. In practice, those terms are most often loosely enforced, if at all, and usually only in response to some operational problem caused by a violation. New network applications are developed and widely adopted as time passes.

And this mumbo-jumbo has any bearing on the topic of discussion how precisely?

Actually, telephony capacity is engineered to some threshold of dropped calls per 100 at the network's "busy hour". This threshold is either dictated by regulatory bodies or is left to the telco. Either way, few--if any--telcos build to "zero drops per 100 at busy hour".

And despite of all these mighty efforts at obfuscation, you still did not manage to hide the fact that the telephone networks are required to sustain a reasonable level of service, even at a peak hour, sufficent to allow a vast majority of calls to be serviced, and the remainder merely with a small delay. And all of that without the need of snooping on conversations and terminating those deemed "unfairly" using the system.

Telephony networks are a smidge easier to engineer from a capacity perspective because there's fewer variables to address.

Total hogwash. They are both packet switching networks. The only unit of capacity that has any bearing on both is full-size data packets switched per second. Additionally, PSTN systems suffer from added complexities of having to sample, encode, and decode analog voice data, which pure data networks do not have to deal with.

A PSTN/TDM phone call takes a discrete unit of bandwidth per call, either 56K or 64K depending on the underlying transport technology.

Right, and a broadband connection takes a discrete unit of bandwith per connection, either 1mb/s, 5mb/s or 100mb/s depending on the underlying transport technology. The fact that it can take less is as relevant to the discussion as the fact that the PSTN connection can take less then 56k when silence is being transmitted. According to your genius reasoning, the PSTN network should be designed to handle mostly silence and croak when all of the people start "unfairly" chit-chatting at the same time.

The only variables are start time and duration. Erlang modeling, based on queue theory, addresses this quite well; it isolates start time by normalizing duration to 3600 seconds/call and provides useful, realistic measurements.

All of which meant dick when people use faxes, dial-up connections, and what not. Face it, the only analysis feasible is practical masurement of the network usage and expanding it to meet capacity. Demanding that people start calling

Re:compare to land

[Wolfbaine]

Such a thing is (or at least was) commonly done in Australia. Many ISPs hold membership to their regional internet association, which provides low cost traffic for local transit (through exchanges such as WAIX, PIPE and the academic AARNET). Traditionally ISPs have passed on unmetered access to these networks (not contributing to the established quotas) however this has become uncommon with many ISPs pocketing the savings and counting all the traffic.

Indeed many local pirates were using the networks for file trading under an assumption of protection from prosecution (saving the ISPs from the usual traffic bills) until crack downs shut down the more popular sites.

Token Bucket

[Anonymous Coward]

Back in my networking class we learned about the Token Bucket algorithm for traffic shaping.
I don't get why ISP don't apply this to their customers, it would be perfect, or am I missing something?

ISPs oversell bandwidth to consumers: If they sell you 1 MB/s then they might have 1 MB/s for every 50 customers they serve. Now with a token bucket that fills at a rate of 10 to 30 KB/s, depending on demand, and has a capacity of perhaps 1 GB normal users would generally have full speed almost all the time, while heavy users would be limited to the bucket fill rate, unless they save up some tokens.

Furthermore it's a standaard traffic shaping algorithm, so I would guess the ISP's equipment could easily handle this.

What am I missing?

They're not filtering content

[rsilvergun]

they're filtering a service. There's still no distinction in what you send, just how you send it. This is like saying ISPs can't filter spam without giving up common carrier. You want to send one or two unsolicited emails, ok then. Send 1 million? Then we've got a problem.

Reality check

[Anonymous Brave Guy]

What we need is more truth in ads.

OK, here you go:

Dear customer/potential customer,

At present, you pay a flat rate for your broadband, but the costs we incur in supplying your service increase with usage. If you are up/downloading 10x as much as most customers because of your heavy broadband use, then you are costing us more than those others. With a flat pricing model, that cost is being passed on to all of our customers equally. We don't believe this is fair to the vast majority of our customers, most of whom don't make such heavy use and simply want an always-on connection with a reasonable download speed.

In recognition of this, we are giving our customers the option to decide between two alternative pricing schemes. One of these will be introduced within the next six months, at which point we will stop offering our existing flat-rate service.

For option (a), we have a tiered approach. Light users can have a max 512Kb/s connection and a monthly bandwidth cap of 1GB, for $5/month. This package is suitable for most people who use the Internet primarily for e-mail, web browsing/e-shopping, and Usenet newsgroups. Medium users can have a max 2MB/s connection and a monthly bandwidth cap of 4GB, for $15/month. This package is suitable for most people who make somewhat heavier use, such as on-line gamers or those who download occasional multimedia content. Heavy users can have a max 8MB/s connection and no monthly bandwidth cap, for $200/month. This is the only appropriate standard home user package suitable for those who run continuous, high-traffic services such as peer-to-peer file sharing or web servers linked from Slashdot articles.

For option (b), we will simply charge a fixed fee per megabyte up/downloaded, keeping the total income we receive across our entire customer base constant. We expect this to result in a cost reduction for light users of up to 90%, little change for medium users, and a tenfold increase in charges to heavy users.

Please select the option you prefer and we will go with the majority vote. For those who require guaranteed download speeds and no bandwidth cap, the same leased line services we offer to businesses are also available to private customers, with prices starting at only $1,000/month (installation charges apply).

Kind regards,
Your ISP

Torrents are identifyable

[davidwr]

It's easy for a router to distinguish traffic that "looks like" web traffic from traffic that "looks like" typical torrent traffic.

It's not practical yet to distinguish child porn, drug sites, and hate mail except on a whack-a-mole basis.

The technology is coming. Someday, they will be able to identify an unencrypted image as "likely child, likely porn" and flag it for human review to send to the police, or simply drop it. Ditto hate mail and drug sites where those are illegal.

Re:Steganography?

[davidwr]

Use s teganography [wikipedia.org]. Basically you could send images with extra e ncrypted data tacked on the end; c an the product detect that??? And if some unlucky admin type looks at the image, they get to see goatse in all his glory, but don't see the enc r ypted data hidden in the image.
--
Is my hatr e d of this box of wires sensible? Or am I a Linux geek trying to make XP work as I slowly go batshi t insane?

I found your secret! I found your secret!

But wait, was it the correct secret????

3rd World

[alexgieg]

Here in Brazil many broadband ISPs guarantee you a MAXIMUM of 10% of the bandwidth you contracted. Meaning: you get something that's announced as 2 Mbps connection and that usually works at that speed, but which could drop to 200 kbps (in peak hours, for example, or for whatever reason the provider thinks is deserved), and the ISP wouldn't be required to improve the situation at all. Nice guys, eh?

You're REALLY gonna be up shit's creek

[Travoltus]

when IPTV becomes more ubiqutous, or when more people start watching news, music videos, etc. online. Yahoo! music streaming, anyone? iTunes, anyone?

IPTV is set to evolve soon, too, to where a Comcast user in Los Angeles can subscribe to a provider streaming from Texas. Episodes of TV shows can be seen online now, and whole libraries are going to be coming online.

Bittorrent's just a big shark in a really really big ocean of bandwidth problems about to hit you like a tsunami.

The backbone will get stronger or new markets won't emerge. Apparently in this day and age, the market has right of way.

Depends upon the ISP

[element-o.p.]

At the ISP where I used to work, we really didn't give a rip what you did with your pipe so long as it didn't cause us problems :)

So, if you were using Bit Torrent (or KaZaA or gnutella or....), we didn't care so long as MPAA/RIAA/BSA/**AA didn't send us a notification of infringing content. If we received such a notification, we would send a warning to the infringing customer. If said customer continued pirating software/movies/whatever, we would continue to send warnings to the customer until either the customer learned how to not get caught or the *AA's would send a subpoena request. While our AUP's stated that we *could* terminate a user's account for copyright infringement, I can't think of a single case where we actually exercised the option.

The bottom line is that dictating how a customer uses the pipe is a waste of time and resources. For me (as an ISP) to tell you (as a customer) how to use your connection just involves me in a never ending arms race and annoys you. So why bother?

Basically, as has already been mentioned, the biggest reason that ISP's get upset with file sharing is because it taxes networks that weren't designed for 24/7 usage from so many customers. Rather than trying to restrict what *protocols* are used on networks, I suspect that, some time in the future, ISP's will begin charging the same way almost every other utility does: charging per unit of bandwidth consumed, possibly with a flat fee until some cap is reached, and then a price per unit of bandwidth consumed after that.

MOD PARENT UP

[MarkusQ]

If I buy say 25 acres of land, and I sell 1 acre parcels of this land, ...to 50 people...how long do I have before I go to jail...?

Exactly. And of course the ISP apologists chime in with the "bad analogy--you can get in trouble for overselling goods, but not services" nonsense. Of course, what they are overlooking is that it isn't somehow "less of a crime" to oversell a service, it's just harder to get caught.

The hollowed principle that the ISPs were relying on was the ancient "but I didn't think I'd get caught" defense.

If somebody takes money from people for X, be it a good or a service, and then blocks them from getting what they paid for in order to resell it to others, they are committing fraud. Period.

--MarkusQ

Need more info

[Panaphonix]

Sorry but this doesn't sound credible. How would the ISP be off the hook after giving you an unrestricted connection? Also, when people have been sued by **AA in the past, there was never an understanding between the customer and the ISP that the provider would block certain kinds of traffic and that would legally shield the users (I'm inferring this based on the discussion you describe in your post). Which ISP do you have and who did you contact there?

Q. - How can you tell if this is happening?

[Brit_in_the_USA]

I would like to know how I can tell if such throttling is happening.

I use the latest Bitcommet Client and no matter what I set for upload/download rates I never break about ~ 80kb (or is it KB) download speed. No matter if I am downloading ~10 files simultaneously or just 1 or 2 popular files. "Health" is always >1500%

I have tried several web speed tests and my cable speed is indeed the ~ 5Mbit/s DL / 0.5Mbit/s UL that I am purchasing.

I am using XP, XP firewall and a recent model linksys router. I have configured port forwarding on the router and Bitcomments reports that it is happy and not struggling behind a firewall.

I would appreciate any thoughts or suggestions. - I was wondering if there is any reliable Bitcomment speed test that can be performed?

Re:Uh

[Elminst]

UH yourself..
He used a little "k"
768 kilobits != 6 megabit

I wouldn't have a problem with that, if...

[Solandri]

If an ISP wants to sell a 3 Mbps service but wants to oversubscribe it by 10x, that's fine. But then they should advertise it as 3 Mbps at 10% saturation. Instead they advertise and sell it as 3 Mbps, then use secret criteria to determine who they try to kick off their service for "overusing" it. Lately they've started adding (very, very) fine print stating you're not supposed to use all that bandwidth 24/7. But the whole thing would sit better with the public if they were just up-front about it.

Re:What fight?

[Cheeze]

So.....you are limiting an unlimited service?

Cost is in buying transit

[Kadin2048]

I think you're looking in the wrong place for the cost.

The cost of you saturating your pipe would really not be in the upgrades necessary to the local infrastructure (at the subnet level), it would be in the additional cost to their ISP (assumedly one of the Tier-1 providers).

The switches at the head-ends of the local cable "exchanges" (whatever they call exchanges in cable parlance) are probably more than capable of pushing 5-6 Mb/s per customer, continuously. Where it gets problematic is as you start aggregating that kind of traffic through the network. If an exchange serves 500 customers, and each of them want to pull 5 Mb continuously, then you're talking about a 2.5 Gb backhaul; if you have an actual connection to the Internet for every 5 local exchanges, that means you'd need to buy a 12.5 Gb/s x 24/7 pipe from the Tier 1 provider. To do that, you're talking Real Money.

What I suspect the problem is, and why you can't use all the bandwidth that Comcast advertises to you, is because Comcast only itself buys a fraction of the connection to the global net that it would need, in order to provide that level of service.

The bottleneck probably isn't down at the local level, it's up where Comcast's network meets the rest of the Internet; they're not peering, they have to buy transit from another provider, thus they have an incentive to try and discourage people from using too much traffic.

The best solution to this, IMO, would be to have two separate limits on traffic, one limit (say, 128kb/s continuously, or an equivalent amount of burst traffic, maxed at 6Mb/s) for packets that actually need to transit to the global net, and another limit for packets that never leave Comcast's copper (6Mb/s continuously). This could be expressed either as a distinct amount of transfer per month, or as rates.

Using those figures just for an example (128kb/s continuous global, 6Mb/s continuous local), you'd be able to push approximately 40GB per month onto the public net, and 1.9TB per month on the Comcast network. (I think I did my math correctly...)

Re:Cost is in buying transit

[Randseed]

1. The ISP goes to torrentLeech, the Pirate Bay, etc., and downloads all the torrents, caching them.
2. the ISP provides them on their local network.
3. The ISP doesn't have to pay their tier-1 provider for the bandwidth, because it's all on their local net.
4. PROFIT!

Problem isn't the price, it's advertising

[Kadin2048]

I think this is a big part of the problem. Buying transit is expensive, when you're talking about significant amounts of continuous traffic (non-burst rates); you think that a 10Mb connection is relatively slow, but buying a pipe that would let you use that 10Mb connection all the time, saturating it, and give you a decent QoS is not cheap -- thousands of dollars a month, probably. I think most people would be stunned to figure out how much a "real" internet connection actually costs.

Whether the backbone providers are "ripping off" the tier 2 and 3 providers, is arguable. They're the ones with the massive overhead expenses to cover, but on the other hand they seem to be making a lot of money...but who can blame them, when they own the lines? The cost isn't in the routing, it's in the lines and the associated maintainance (backhoe fade, anyone?)...it takes a huge amount of infrastructure to get your packets from NYC to LA in 100ms.

I guess if you don't like their pricing, see if you can get a few billion dollars of capital and run your own long lines, and try to compete. It's not as though there's only one backbone provider, either -- there is some competition in that market, at least.

The real problem that I have is not the service being provided by Comcast/et al to end users. For $40 a month, you get what you pay for, and it's not that much. I just dislike the way they advertise it. The average person is not that smart, but he's not entirely stupid either; if you're advertising burst speeds, then say they're burst speeds.

When you buy one of those $500, 1Mb connections, they don't advertise it as being "1Gb internet!!" just because that happens to be the maximum burst speed, they advertise it at the continuous-throughput level, or they state both: 1Mb continuous, 1Gb burst. By refusing to advertise and price their home plans this way, Comcast and the rest of the home-broadband providers have only themselves to blame when people get upset.

Does this mean...

[MikeTheMan]

Does this imply that if you had not used this unthrottled service, then they would act as a barrier and protect you from civil proceedings? Somehow, I doubt it...

Re:lol, broadband, junkie

[nettdata]

Bullshit. Read the terms of the contract. EVERY SINGLE ONE has a clause that says that they can terminate it without cause, unless you get into higher grade Commercial contracts for longer terms for much, much more money.

You'll also find that most of them have clauses in there to deal with "abuse" of the service or network, and "abuse" is how THEY define it, not you.

To them, "abuse" could be you running Torrents 24x7 and saturating their network.

Don't kid yourself, the contract you entered into is written TOTALLY in their favour, not yours. For that matter, I bet 99.9% of the subscribers have never even read the thing, never mind understand what it says.