Slashdot Mirror
Bandwidth Demand at American Universities
Robert Rwebangira writes: "There is an article in The New York Times (free reg required), discussing college students 'insatiable demand for bandwidth.' Of particular interest is the continuing prominence of file-sharing (inspite of the demise of Napster) and the amount of bandwidth consumed in even 'legitimate' activities. It seems students demand for bandwidth just keeps growing."
When the demand for bandwidth has usurped the demand for beer. What's wrong with children today?
Some universitys are now capping people at around adsl speeds to try and limit the charges
Without specific proof, I'd be very willing to say that it's not just students. As the internet grows, and we get faster computers, and more visually intense websites, its only obvious that bandwidth demands for EVERYONE is going to grow. The size of applications and games has also risen, and even downloading legal demos and share/freeware games is bandwidth intensive, this is not even to mention 'warez' and the fact that nobody seems to be happy with porn 'pics' anymore, they want vids. So, as download sizes grow, its only obvious that bandwidth demands will also grow.
Don't Tread on Me
... is that people get used to high bandwidth as customers. Even though they may technically be customers who are supposed to be buying an 'education,' the fact is that (typical, 4-year, residential) colleges / universities seem to provide professors and classrooms only to supplement their provision of high-speed, on-site-service, always-on, relatively unrestricted network access. This is one reason I regret not living in the dorms at Univ. of Texas, which it turns out grew some good-at-the-time ethernet ports while I was in school, and I bet are still good.
... I'd pay $300/mo for the always-on mediumband available in rural Montana etc), I want there to be an increasing supply of college grads used to insane, insanely cheap bandwidth to help drive the market :)
As someone who wants to be a customer for better internet access of all sorts (true all-continent roaming access for N. America at least would good
timothy
jrnl: http://tinyurl.com/c2l8yr / foes: http://tinyurl.com/ckjno5
I am a student at a university in South-Africa. From campus you are only allowed internet if you have an account for which you pay, and once you have this you are only allowed HTTP. I had to use tunneling to get on IRC for instance. Another thing is our whole university probably has about the same amount of bandwith in total as about 10 computers in the US use. This is not the universities fault, but rather out countries weak communications infrastructure and the fact that we have one telco with no compitition.
All you students and other people in the US should stop complaining, you have _loads_ of bandwith.
At my big ten University bandwidth use by the residence halls has been enough of a problem to cause our keycard access system to become DOS'd. You need keycards to buy food, enter buildings etc...
As of Monday 1.5 GB a week upload and 1.5 GB a week download restrictions go into place. You get two warnings if you exceed these limits and then your residence hall connection is yanked for a semester.
The university I work at has a huge pipe (1Gb I think) shared with two other local universities. Generally we use the least amount of the bandwidth, but at one point our usage had hit like 500Mb/s Needless to say teh other schools were freaking - they were losing packets due to teh pipe being so full. Well, our dorms are on their own network. Sure enough, thats where most of the bandwidth was going. Blocking Kazza/Morpheus and co is tough since it'll switch and seek out other ports. So the only solution was to limit the total bandwidth for the dorms to 25Mb/s Sure enough, once that block went in place our usage overall dropped to like 90Mb/s. 300-400Mb/s of bandwidth just for the dorms????
The students were upset since their pipe was now slam full and they had trouble getting out, but the response basically was - stop running servers and stuff for music that suck up bandwidth and you'll be able to get on the Net to do the stuff you need to do. Its not perfect, but for now it works and keeps us from totally saturating our pipe.
Top Most Bizarre/Disturbing Error Messages
There's no way to stop filesharing except at the endpoints of communication. Unless the users stop wanting to use filesharing, there will always be workarounds for all the filtering and blocking you can think of. The next step is encrypted connections below tcp level, aka ad hoc virtual private networks. Since there's a heap of good reasons why one would want all traffic (even the "non-shady") encrypted anyway, universities will most likely refrain from blocking the necessary protocols. Once the traffic becomes opaque to the transport, there goes the ability to filter based on contents or protocols.
What can be done is this: Restrict bandwith or volume of data. That however will limit certain promising aspects of network development like freenet and other decentralized protocols. That's why especially universities which are supposed to be interested in innovation should think twice before crippling network access.
I work in the Academic Computing Services department of one of the UC Schools...
The majority of our problems come from about 10% of the population on campus who are online 24/7 downloading their pr0n, thus giving most of the other students probably polled in this study a reason to start asking for more bandwidth. I should also add that these are the same 10% who are hogging internal bandwidth playing counterstrike, etc.
I think that the term "Insatiable Demand" is definitely a misnomer. Although the "Prominence of file sharing" does apply to quite a few people in our dorms, 90% of the people are utilizing the network for, at most 10-20 megs a day. In fact, we have a 2Mbit cap on the routers coming out of the dorms, and most users find that they can surf the web and get their 3 or 4 files a day with no problems, and are pleased that, at 4AM, they can get an insanely high throughput. The reason that the students complain about the network being slow is because of the caps (which most don't know about) at peak times, because, again, the 10% that actually do have an unquenchable thirst for data would take full advantage of the situation.
I should add also that we block Morpheus, thereby removing those oh-so-lovely TCP standards hacks it implements, so YMMV
At an unnamed university in California for whom I work, we have available at any given time ~ 40 Mb/s, with around 800 students living on campus. Normally our network situation isn't bad, but this last fall semester it got completely out of hand.
Of the 40 Mb/s, on average one-half of it would be in use directly by students in the dorms. At times, individual ports would be using 7-8 Mb/s, for as long as ten hours at a time. Eventually, it was decided that the impact on the university's bandwidth was affecting the educational functions of the campus network and all users were reminded by mail of the campus AUP for the network.
Students, being students, ignored it largely. The offenders who chose to ignore it and flaunt the fact they were ignoring it (anything above 2 Mb/s for over a few hours) were warned by mail individually, and after that, had their ports shut off and the MAC address of their computers banned from the DHCP pool, so no matter where they went (i.e., plugging it into their roommate's port), they were locked out. To receive service again, they needed to contact the student judicial affairs, which involved only signing an agreement not to be naughty again, with the threat of being kicked out of the dorms.
Long story short, a few people got their ports shut off and had to go through all the rigamarole. Most of they had no idea what they did was wrong, and didn't understand that leaving Kazaa, Morpheus and all their other file trading utilities on all day long was not only illegal, but the reason they received the notices in the first place.
It boggles my mind to think that these kids got into a university and don't understand that downloading the new N'sync album before it's on store shelves is illegal. Theft is theft, no matter who you're screwing over, but luckily, most will figure it out pretty quickly when the university tells them they were disconnected because Sony contacted the university about their particular computer, and yes, both the university and Sony would be more than happy to have them kicked off campus rather than deal legally with a pirate.
Here are just a few router stat graphics from my university. As you can see, Kazaa/Morpheus is 85% of the outbound traffic!! Inbound isn't quite as bad, only 63% or so.
OK, first off, I'm very serious. And I'm ignoring such things as Morpheus, Gnutella, etc. Those should be blocked.
But honestly, is it so unreasonable for bandwidth demand to go up? The medium is getting richer. Websites are taking advantage of media like Flash, movies, and sound more and more. More information abounds. People want stuff in more than just plain marked-up text. Maybe the increase is disproportional, but there are people (like my parents) that still believe that a 28.8kbps modem is sufficient. Not true.
Yes, as new services (including gnutella and napster) come about, there is a natural demand for more access. Deal with it.
More, quicker, better. It's the way things will go.
"To err is human, to forgive is simply not my policy." --root
I attend a small state school of about 7000 undergrads(well, maybe not super small) and across the past year we have experienced precisely this problem. We have ethernet jacks in all the dorm rooms and everybody was running all the usual file sharing apps and it got to the point that you could not surf the web. The browser would time out because it went to long between packets! So the beginning of this year, A QoS package(I beleive we're running packeteer) was set up on our firewall to hunt down and block the packets for these programs as well as streamline some other network traffic and things have really cleared up. It's not blazing fast but surfing happens at a reliable 20k/second which is pretty snappy for browsing. Linux ISO's still take a while though ;-)
Things to think about in the current suggested solutions:
1. Limiting bandwidth to dorms just hurts students who don't run these programs! Yes there are some of us out here. The majority of students even in a small school can not be organized to stop running this type of software. They just bitch about the slow connection and keep right on downloading mp3's. At least that's what happened when our college tried it.
2. Blocking ports isn't effective. One of the earlier posts mentioned about how Morpheous and others seek out new ports. This makes normal port blocking on a router or firewall useless and may arbitrarily block other software that is not a problem but happens to use the same port. You have to have software smart enough to look at packet type/content to be effective.
3. QoS software works if you get the right package. I work for the computer center at my college and I know we went through a number of packages before we settled on one. But it really makes a difference and it's suprising how many people don't even know this sort of software exists.
Of course by then I'll be demanding real-time, life-size holographic video of a "phone-call" to a friend in Asia @ 3 million DPI.
Then finally, matter transport. I wonder how many bytes it'd take to decribe each atom and all its subatomic particles. How many atoms to a human body? Let's do it Star Trek style, and do it in about 5 seconds.
Fast forward a million years, and let's say we haven't blown each other up yet. We'd probably be at the equivalent of God by then.
"Hey Jeff, fancy creating a solar system today?"
"Why not Bob?"
"Well fancy that. OK." *click* "What do you think of that for a Sun?"
"Pretty impressive. Hey let's transport Dave's planet from quadrant four over here. That bastard is always gloating. It'll take him a few seconds to find it."
"OK." *click* "Hey it sort of looks nice doesn't it?"
I did a security assessment at a large university late last year, and found something astonishing. The number one expenditure of time for the computer security staff was dealing with cases of "copyright infringement" from the representatives of record companies. And I mean, it was something like 80% of the manpower. What was also infuriating was that a lot of these cases involved MP3s that had been posted by the band to their own website (that week that I was onsite, most of the warnings given to the university had to do with a song by Incubus, if I remember correctly, that had been downloaded from the official Incubus website.)
For your security, this post has been encrypted with ROT-13, twice.
At my school (Lehigh University), we address bandwidth problems this way...
Each student (each MAC address, really) is allowed to transfer one gigabyte within a 12-hour period. If you go over the limit, you get put in a "penalty box" (basically sub-58k speeds) for a while until your transfer total for the past 12 hours is under a gig. Uploads and downloads are counted separately, and transfers that don't go off-campus don't count at all. One of the university's servers holds a list of what addresses are in the penalty box, and what their transfer totals are.
This is quite effective - it gives each student a reasonable amount of bandwidth, and it only punishes those who actually use too much of it. And our 45mbit internet connection is rarely maxed out.
Oh,dear boy, you had it easy, why back in my day we used beads carved from rock with holes chipped inside so they could be fashioned on spun strings. We had slaves to slide the beads according to set rules. In fact, it was a dangerous job due to the overwhelming number of beads required for basic computational analysis. To calculate prime numbers, MMMCX tons of beads were required and a failure of the supporting members meant an avalanche of rolling beads upon the camp. Begin worker's rights and other heretic movements that impeded technology.
But I digress. You young whippersnappers think you have it so good with silicon, you ought to try pushing carts of beads uphill by the bucket uphill both ways with no round wheels.
I've said this before, but its a good point. Even though its been pointed out to me that some of the file sharing software supports this, people don't primiarly share locally. They abuse the upstream connection for all of their sharing, when chances are good on a large university campus, there will be numerous others sharing similiar things, and the local bandwidth is cheap and plentiful.
The clients used for this purpose need to prioritize on local networks. Even if there is a limit on the number and speed of the connections, give immediate unrestricted access to anyone thats on the local net. This will encourage people to look first from within and only search the rest of the internet if it can't be found locally. If other large universities did the same thing, then the incoming requests would also be significantly minimized.
Remember, if the upstream connection is used or a local one is used, the local bandwidth is spent anyways. Might as well quit wasting one of them.
-Restil
Play with my webcams and lights here
Yep, that kicked in around the very late 90s (until which point JANet had been charging flat rate, or the University Computing Service had been absorbing the variation, but either way the individual colleges and departments didn't take the hit).
Perhaps slightly surprisingly, it wasn't the zillions of us playing Quake II over the 'net that did it. We were generally responsible enough to avoid doing so at peak times and keep it to the evenings and weekends, and many college computer officers had an informal policy of allowing such use as long as it was fair and didn't disrupt legitimate academic things.
Also perhaps surprisingly, this all predates things like Napster. Mass music interchange wasn't going on then on the scale it was until a few months ago.
What did it was the Warez servers blatantly running on university networks. They knew where they were, of course, and for legal reasons closed them down every now and then. But a certain type of hax0r dudez just kept abusing the system. So, now small groups or individuals get charged, caps are in place, traffic is presumably monitored, yada yada.
Sadly, and as all-too-usual, the irresponsible and downright illegal behaviour of a few has now impacted the facilities available to the rest.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
I suppose you don't remember the day this fall when packet shaping was turned off and absolutely nothing worked. The "poor performance" mentioned in that announcement is quite an understatement -- traceroute probes came back in times on the order of seconds or not at all. I also remember the days just before the traffic shaping was put in place and I was getting over 5-second ping times. People like games, but when email and the web stop working, people quickly start thinking about realistic priorities.
You also might ask yourself: did your games work when no traffic shaping was in place and ping times were measured in seconds and packet loss was rampant? I doubt it. How, then, can you blame the shaping for your problems if your problems didn't go away when the shaping did?
First of all, because of retransmits, dropping packets can lead to high "ping" times, depending on the protocol/application and what it considers a "ping." Second, the software may be trying to "smooth" out the traffic to fit under some limit -- queueing packets from burst periods to be transmitted in lower-traffic periods.
That's a nice idea in theory, but the problem in practice is this: tracking all those individual TCP flows would require immense amounts of computation by the router. AFAIK, we're talking orders of maginitude greater than what is currently available. From what I hear, the really expensive Cisco router at the border is already extremely busy doing just the simple packet shaping, which just limits the aggregate high-port traffic. Breaking that one giant flow into millions of little flows is non-trivial and probably impossible.
Now I'm no networking expert, but it seems to me that doing traffic limitation on a per-user or per-flow basis would probably require some sort of distributed model that did the limiting closer to the user. This might mean not only replacing all the switches with more expensive models, but also hiring new staff (non-trivial) to install and integrate the new hardware into the existing network and to maintain the configuration on all those switches or to write some fancy new automated system to do the maintenance. Of course, all that is just another idea, and there may be some other pratical considerations that make it even less feasible. It also sounds like a lot of work, and considering why you would be asking them to do it, I can imagine that it would end up a lower-priority item than other things that the FAS network people have to do.
I don't know either, but the file-sharing folks have shown themselves to be pretty adaptable. If they would play nice and limit themselves to certain ports and protocols, then everything would be easy. And, of course, who knows whether this would just require too much CPU time, as well.
Anyway, some background/historical info:
The undergraduate dorms get their connections through the FAS (Faculty of Arts and Sciences) network, which in turn gets its connection through UIS (University Information Services), which provides networking for all of Harvard. Back in the good old days of 2000, before the file-sharing people went crazy, UIS had just upgraded its internet connection (to a 155MB/s OC-3, IIRC) -- oh what heady and naieve days those were. Now, however, the situation is this: file-sharing programs seem to act as a gas that consumes all available bandwidth. That first started happening, IIRC, the weekend when Napster was going to be shut down. Suddenly, the undergraduates doing file-sharing shut down the connection for the entire university (which is much larger than the undergrads).
This is a classic "tragedy of the commons" scenario. The file-sharing folks abused a shared resource and ruined it for everyone else. What the traffic shaping essentially does, since it limits the student network to some portion of the FAS-UIS feed, is allow the file-sharing programs to ruin only the undergraduate dorm commons. Now, it's easy to blame the shaping for the bad performance, but the real truth of the matter is that if the file-sharing programs weren't trying to consume essentially infinite traffic, your games wouldn't have a problem. The router doesn't slow things down just to mess with you. Gamers are getting "scrood" by the other undergraduates doing file sharing, not the traffic shaping. Incidentally, the shaping is similar to the sometimes-raised suggestion of giving the students their own internet connection and fighting it out amongst themselves, except that traffic shaping makes it easier for people to complain about "the Man" and that a separate feed would be stepping on UIS's toes a bit. Also, a student-only connection would have to be much bigger than an OC-3, because it has already been demonstrated that an OC-3 can't handle the file-sharing traffic.
What most people want is for the file-sharing people to be moved out of their commons and into someone else's. That's what you're suggesting when you want TCP but not UDP to be limited. As an off-campus user, the file-sharing people are already out of my commons, so I'm happy that I can access Harvard websites and mail and login servers again. Most of the users of Harvard's network aren't on-campus undergraduates, either. Perhaps you can understand, then, why I'm defending the shaping. I like the network to be actually usable instead of the packet-dropping mess that it is when shaping isn't there.
There's also the option of getting rid of the commons, which is the shaping-per-user suggestion, but that has some disadvantages, too. Even if the undergrad dorms get one third of that university-wide OC-3, that's only 7.5kb/s per undergrad, which is not too great. Up the per-user bandwidth to something reasonable and now a certain number of file-sharing people can take everything over again.
Then there's the get-a-bigger-commons option. There are several problems with this. First, it's not clear that there exists a pipe fat enough for the number of file-sharing users among the undergraduates plus the other uses from the university at large. Second, of course, is what someone else has already mentioned -- try to imagine the FAS network admins justifying to UIS the need for the university to get a bigger feed, and UIS in turn having to justify that budget item, just for undergraduate file-sharing.
Everything I have said is based on what limited stuff I have seen and heard, but it seems to me that it's all really complicated, and if there were an easy solution, I'm sure it would already have been adopted.