Slashdot Mirror
Limiting Bandiwidth in a Shared DSL Environment?
stylee asks: "We have a DSL connection that runs from a Cisco 675 DSL modem to a 24 port hub. Cat 5 cable has been run to the utility closet of each unit. The condo assoc. pays for the DSL from the monthly condo fees collected. The internet connection has been terribly slow the last few days, so I did a little snooping with ethereal and found that there is an individual who is using eDonkey 2000 to download and share movies. This user is eating up all the bandwidth. I want to set up a good router that can do load balancing so that an individual can't take up all the bandwidth and I was wondering what Slashdot would recommend. I would have to do it on the condo assoc. dime so it would have to be done on the cheap. Any suggestions?"
I always look on freshmeat.net for these solutions...here's a tip...
Linux Bandwidth Arbitrator looks like it was designed for this sort of thing...
What you need is a managed switch. They will allow you to limit bandwidth or completely disconnect a specific port. HP's switches are supposed to be particularly good
Be warned... a managed switch WILL cost several times more than a normal switch.
But apart from that, your only other choice is to use some sort of arbitrary setup to limit bandwidth to certain IP addresses and force each user to have one static IP (virtually impossible to enforce with your setup)
-- If you try to fail and succeed, which have you done? - Uli's moose
At my house we have four guys and we all download pretty heavily (bittorrent, edonkey, gnutella, etc.) Unlimited this just chokes up the whole connection (a fragile cable modem that gets confused if it gets too many packets)
So I just run "tc qdisc add dev eth1 root tbf rate 250kbit latency 20ms burst 2kb". This keeps the network running at full speed with all the downloads going.
Checkout the Bandwidth Limiting HOWTO on tldp.org
I set up a DSL traffic shaper on Linux a bit ago. It's a bit of a pain in the ass to figure out the right things to do, and I don't have the script handy, but here are some pointers (given that this is from memory, some of this will probably be wrong).
Get a Linux box. Get two NICs (c'mon, NICs are cheap these days, and the DSL modem only needs a 10Mbit one).
Set up bridging on the Linux box.
ifconfig eth0 0.0.0.0
ifconfig eth1 0.0.0.0
brctl addbr br0
brctl addif br0 eth0
brctl addif br0 eth1
If your boxes use DHCP, you might want to give your shaper an outside IP address (so that it can run ntp and the like, if nothing else). Use br0 as the interface -- this tripped me up at first.
dhclient br0
Add per-host rate limiting. There are two *excellent* solutions to do this automatically under Linux -- esfq and wrr. Both automatically detect IP addresses on one side and spread bandwidth out evenly. Neither is apparently actively maintained, unfortunately, so if you're using a 2.6 kernel, you're out of luck. Your best bet is probably HTB (which *is* included in 2.6 and I believe current 2.4 kernels). HTB requires you to manually create a child of the main HTB qdisc for each IP address, and filter based on source IP address (or source MAC address, which is probably more appropriate if you have a single Ethernet segment and dynamically assigned IP addresses) but lets you filter traffic differently for each host. For a small network, this may be feasible. I'd hang another qdisc off of the HTB that reduces the priority of P2P *within* each host's account, so that someone can use spare bandwidth for eDonkey or whatever, but still retains reasonably snappy SSH, even on their own box.
You must set the maximum flow of the HTB just below the DSL modem's data transfer rate, or else the modem's buffer will fill up when outbound traffic fills up its (big) buffer, making interactive use impossible. Keep reducing the limit and then ping flooding (ping -f) the outside world from an inside box. Keep a regular ping running in another terminal, and monitor it. When your system is working right *ping times should not climb above 150 or 200ms or so on a box*. No 1000ms latency. You should simply start seeing packet loss.
I must say that setting something like this up was a huge pain in the ass, and that if I had the script handy at the moment, I'd post it. The Linux networking/filtering/routing system is not as well documented as it should be, and is *not* always the most intuitive thing in the world. It is, apparently, pretty powerful, based on what I've read from folks that have used other systems, though. [shrug]
Speaking of which, I can't figure out why sfq is in mainstream Linux but esfq is not. SFQ is, to my mind, almost useless for most people. Who on earth wants to balance all their TCP flows evenly? Even per-host bandwidth allocation is a *far* more common problem, and one that vanilla Linux (and any 2.6 kernel) cannot handle well.
I did not find it necessary to use ebtables or ipchains to produce an effective traffic shaper. YMMV.
May we never see th
IPCop v1.3 w/ Wondershaper or wait a couple more weeks for 1.4 which will have bandwidth shaping built in. It's a linux distro just for firewall/routers, runs on anything from a 486 up.
Monowall (www.m0n0.ch/wall) is a greaqt application for this. Can run from CDROM, CF or on a Soekris board - can do per IP bandwidth limiting/shaping, and totally free, based on BSD. It was trivial for me to set it up here.
Or FreeBSD, we use our firewall box where I work, and use the traffic shaping portion of ipfw2 (man pages, ipfw at www.FreeBSD.org) to limit bandwidth to certain hosts. FreeBSD allows you to add a rule that passes all traffic through either a pipe or queue (pipe is what you want), set the bandwidth, size of the backlog queue, and monitor usage of the pipes. If you set it up as a transparent bridge (see the advanced network topics in the FreeBSD Handbook at www.FreeBSD.org), you won't even have to change host settings. This way, you can limit traffic on an individual (or group) basis, monitor usage, and just drop the box between the main switch and the dsl router, turn it on, and pretty much forget it (especially if you don't allow remote access to the firewall, except maybe ssh or a VPN).
The same can of course be done with Linux, but in my (though somewhat limited to my place of work) experience, FreeBSD's traffic shaper is a bit more reliable, and much easier to set up (it's all in the handbook). In our case, that box is a transparent bridge, accessible only via ssh or from the inside interfaces, with three NICs, one for the outside router, one for the inside public systems, and a third with private addresses, where natd (man natd, also integrated with ipfw via FreeBSD's divert sockets) translates the private addresses as they go out of one of the other two interfaces. We also run nagios (network monitor), etherape (looks cool when you see the traffic real-time on a GUI), and poptop (MSCHAPv2 capable VPN server), along with IDS logging via ipfw and tcpdump/ethereal, all on an old Duron we had laying around collecting dust.
In all, our Firewall/VPN/IDS/Traffic Shaper/Network monitor cost us about $250 in hardware, and two day's labor. I saw a similar product (though in a nice 1U rackmount case) listed for $6000 at CDW, so whatever you do, you can't go wrong with Linux or FreeBSD on cheaper hardware, unless your time is worth a few thousand dollars an hour.
--That's the point of being root, you can do anything you want, even if it's stupid.
As another aside, some distros bundle pump as the DHCP client, rather than dhclient.
/etc/sysconfig/network-scripts/ifup, and search for a line that looks like the following:
/var/lib/dhcp/dhclient-${DEVICE}.lea /var/run/dhclient-${DEVICE}.pid -cf /etc/dhclient-${DEVICE}.conf"
Oh, speaking of DHCP, big tip for Red Hat/Fedora users. Absolutely do not use the vanilla ifup scripts that Red Hat provides. They *suck*. If you are on any kind of a consumer DSL connection, every now and then (perhaps rare, perhaps common) you will lose your connection, for whatever reason. For some reason, Red Hat sets up their copy of dhclient to *give up* if it fails to get a dhcp lease, which means I frequently endured having a power outage at my house when I was away (killing the line) and then being unable to reach my computer remotely because it never acquired a DHCP lease.
Look in
DHCLIENTARGS="${DHCLIENTARGS} -1 -q -lf
ses -pf
Change the -1 (telling dhclient to die if it can't get a lease immediately) to a -w (telling it to keep trying) in that line, and you won't have to endure your Linux box randomly becoming unreachable and losing the IP address on its interface.
May we never see th
One other thing, if you don't want to limit on a host-by-host basis, you could do it by type of service. Say you allocate 80% of your available bandwidth to common web, instant messaging, mail, and DNS traffic, and the remaining 20% for everything else. Just watch your tcpdump/ethereal/etc. logs for about a week to see the normal behavior (and the abuses). This way, the normal, non-abusive services are quick, while the unknown/abusive services are limited, which has a side benefit of discouraging improper use. Hell, if you can lock down the most abused ports, set the pipe they go through to 2400bps, and see how many people still use them in a week.
--That's the point of being root, you can do anything you want, even if it's stupid.
I used to use CarraFix, but Throttled whips the shit out of it.
I had to play around with the startup file for a few hours to get it working right. Here's my relevant modifications, if anyone's interested:
Don't forget to add a default class to the HTB to match any MAC that all your previous matching work didn't match. That way, anyone that you *haven't* added a MAC entry for (adding a child to the HTB tree) will at least go into a general class and get connectivity...they just have to share it with all the other people in the "general" class.
You may want to toy with the idea of having a perl script or something look at unmatched packets or maybe scrape the ARP cache (arp -a) to automatically add new entries to the HTB tree.
You will want to be sure that this box is set to autorestart on power failure.
You will want to include instructions (probably on the face of the box, as well as on file with whoever owns the property) on how to remove the box from the loop. That hard drive will fail someday.
For some reason, when I insert my shaper in between my DSL modem and the local network and start using it, I seem to see a delay of a minute or two before requests from clients on the inside start hitting the ouside). This confuses me immensely, since a major benefit of using a bridge over a pseudo-bridge is that the ARP entries, the MAC-IP mappings, stay the same. No idea what the cause is.
May we never see th
Netlimiter is good for running on an individual machine (I run it myself to prevent my mailserver and HTTPD from eating all my upstream), however there are better windows solutions for gateways.
http://bandwidthcontroller.com/
Is a fairly decent gateway traffic shaper - not quite as configurable as linux solutions, but fairly easy to set up and you can limit by a number of options, port, protocol, etc.
Free trial version to so you can see if it works for you. $50 to buy.
N.
"Nothing strengthens authority so much as silence." - Charles de Gaulle