Domain: lartc.org
Stories and comments across the archive that link to lartc.org.
Comments · 135
-
iproute2/tc, not iptables. http://www.lartc.org/
not iptables so much as iproute2/tc.
http://www.lartc.org/Wondershaper can be a wonder, tho I have found that it can be improved somewhat by rewriting the concept in perl (Python should work too), rather than bash. Makes it more flexible. Use your language of choice of course, so that you understand it.
IPtables cannot do shaping, although you can use it alongside iproute2/tc with MARK.
-
Re:Vested interests...
You can get around this problem by running the traffic conditioner script from the Routing-Howto cookbook section. Even with sacrificing a bit of download speed I've been using it with great success for the last year or so (upstream bandwidth never fails to max out even if I'm downloading at 350KBytes/sec.)
The Ultimate Traffic Conditioner: Low Latency, Fast Up & Downloads
or use the WonderShaper script, probably a lot easier than wading through the docs. Although I haven't used this... -
Re:Traffic shaping
In Linux you don't (need)/use iptables. However, for resources, http://www.lartc.org/
Their Wondershaper is a good start, but can be improved upon, or extended. -
Bandwidth shaping with Linux
Actually, it is 100% possible for you to set up traffic bandwidth shaping so that any particular IP is only allowed a certain amount of bandwidth, for example.
Use a UNIX-like machine as a router/firewall for your network, and you suddenly have amazingly detailed networking possibilities within your reach. I strongly suggest reading the Linux Network Administrator's Guide. Even though it's getting a little outdated it has some downright cool-ass information within.
Of course, few users are technically adept enough to actually set up a router like this, but I'm sure it has been used a lot for people who want to keep their wifi access "open", but safely limited.
On a related note there are pre-built linux firewall packages out there which will surprisingly easily allow you to do what I was just talking about.
Also, here is the Linux Advanced Routing & Traffic Control HOWTO ... It's a bit technical but a useful resource nonetheless. -
Re:The report you are looking for should be called
You need a smart gateway. Your E1's border router, or a gateway immediately behind it, needs traffic shaping and queueing. Pretty much any circuit anywhere needs traffic queueing. Either side of your E1 could probably benefit from a compressed virtual circuit such as maybe a VPN. Compress all traffic that way. If you locally host your web servers, you can use a reverse proxy that includes mod_gzip and other stuff to strip whitespace from their content. You can also control your users' behaviors with caching proxies like squid and with a layer 7 packet filter. The layer 7 filter will protect against p2p and such. If you think the network is being abused but you want to encourage self-censorship, make the squid logs public.
:) -
Re:Traffic Policing
Linux or BSD boxes are free on the curb. Either is fine if you dont dick with it constantly, put it in a closet and go. Gigabit is $100 for the switch alone, I havent found any in the trash yet. Radio stations often operate on shoestring budgets. We have the largest record collection on the east coast and the station still operates on pennies.
Besides, there's no reason you need gigabit for such a piddling task. If someone's going to be saturating a 100bt such that even a 1.4mbps stream is underbuffering, there's a good chance of under run on gigabit too.
QoS is the only acceptable solution for a radio station. If you have to find two boxes on the street for absolute redunancy, go for it. Get redunant $5 switches too. While you are at it, the most likely thing to fail will be the hard drive; if you need bullet proof do netboot to hard drive less routers or use one of the distro's on floppy or distro on cd's. There are some good distro's made explicitly for routing for this very purpose.
Really, the big problem with this is that if you are on a switch all packets now have to flow through the single box. In effect, its much like running a full duplex hub from there since all communication is flowing through the single point.
That aside, I'd like to thank you for at least providing a valid argument against my post. The other two people who replied were a bit more challeneged. I like to think I at least made some sense. Aside from DHCP relays I dont think anything I mentioned wasnt straight out of LARTC. And you do have an excellent point that should definately be taken into consideration. -
Linux can do this too - wondershaper
Requires CBQ or HTB. I personally think it works better with HTB. See http://lartc.org/wondershaper/
-
Linux Alternatives
Wondershaper http://lartc.org/wondershaper/
Got known of it when it was included in the development branch of floppyfw http://www.zelow.no/floppyfw/ - A Linux-based router software package that fits on a floppy disk (yes, those 3.5" diskettes).
And to answer someone above questioning the performance of using a PC to route packets instead of dedicated hardware, it depends on what dedicated hardware you're talking about. For once I believe many of the $50 boxes out there are indeed quite poor in performance when compared to a reasonablty antique (say 486DX2-66) PC running a *nix router package.
But if you mean the $5000 layer 2 router, then I don't know. -
Re:Linksys?
I was basicaly doing this on a 486 33Mhz pc using shorewall and a tc script, untill i replaced it with a linksys wrt54G running openwrt, shorewall and wondershaper.
the openwrt handle everything i throw at it. on my 5/5Mbit link, with low cpu consumption (10-30% depending on load). and the pingtimes are lovely with wondershaper.
the linksys isn't a powerhouse exactly, and a shorewall restart akes about 90 seconds. but with iptables save/restore, this is a nonissue. Boottimes are quite acceptable compared to all semiadvancved routers out there. Not that you ever reboot the thing...
I Admit that it do takes quite a beating to saturate my 5/5 in the first place, but it happened frequently enoughf to be worth the 30 minutes it took to setup shorewall and wondershaper on the router.
the wrt54g+opwnwrt have lower power consumption then a full pc, and very low noise compared to a pc. But still remain a full linux with the ipkg package management, allowing you the usualy freedom you experience in linux. Something you dont get from all the custom firmwares out there.
And It's dead easy to install for even the least technical inclined gamer out there. But it do require the use of ssh and reading skills, so it's a notch harder then custom firmware's that use the webinterface only.
sepski -
The Real Issue..
The real issue with these kinds of routers is the fact that the cable/dsl modems themselves are not interactive once their data queue becomes filled. Sure, traffic shapers are execellent and I've read http://lartc.org/howto/ which has great information for linux. Cable/DSL connections are asymetrical, and when you send data from your pc to the actual cable modem, you send it at 10/100megbit (whatever speed the nic in your pc and cable modem agree on) Your ISP will limit you to 512kBit upload for example. The modem cannot send data to your ISP as fast as you can send it to your modem thus the data queue fills very fast and your modem has trouble keeping up. These shapers can simply slow down the rate at which your PC sends data to the modem and thus stopping the filling of the data queue in the modem which will allow it to be more interactive. That is the biggest problem you'll have with cable/dsl connections for a few users. Sure, more detailed protocol based shaping can and should be used to reserve bandwidth on a larger scale.
-
Most name servers obeyed DNS TTL for meI just recently switched ISPs and hence IP addresses. I found that most name servers obeyed the 3 day DNS TTL that I have been using for a very long time.
I used the Linux Advanced Routing & Traffic Control utilities to set up the split access stuff. This allowed me to send all packets from the old ISP back through their link, while packets from the new ISP went on the new link. I changed the DNS entries and then I monitored the traffic going through the old link. After one TTL period, almost all of my traffic was using the new link. The main exception was NTP clients, which run for a very long time and only do their DNS lookups on startup.
I run a (non-tech) website that is used by many people, and also a the authoratiatve name server for a domain that gets a couple million lookups per day from tens of thousands of caching name servers. If there were widespread problems, I think I would have noticed it.
I'm not saying that there aren't a lot of really broken name servers out there, just that they don't appear to be rampant.
-
Re:Dupe?
-
Re:central machine
Set up one machine with some sort of filesharing & VNC & big HDDs and make that everyone's central torrent location. Have it set with reasonable bandwith limitations & go from there.
torrentflux is a web based bittorrent interface that supports multiple users. Install that rather than vnc and you have a very convenient central bittorrent machine.
It doesn't provide particulary good control over bandwidth allocation for a group of torrents, but something like wondershaper (as mentioned elsewhere here) should help with that. -
Wondershaper!
Stuff at lartc.org, notable the wondershaper may serve as a basis for what you want.
-
Wondershaper!
Stuff at lartc.org, notable the wondershaper may serve as a basis for what you want.
-
Re:Build your own?
I've been using a linux server as my router for roughly 5 years now. I don't think I could ever stand being limited to a generic hardware router. Advanced Routing & Traffic Control certainly isn't that simple to understand and use in advanced ways, but for those who want to you can do some rather nice things.
I use my linux router to split my upload bandwidth 50/50 between myself and a roommate. I've also setup various bandwidth guarantees to certain software, making voice-chat always work well, despite running many bittorrents. While I believe some custom firmware for one of these routers might do everything I'm doing currently, they never seem as reliable or customizable. -
Re:Overpriced
While there are plenty of good reasons to have an all in one little box that does this. I like my current linux box setup for flexibility. Like Running a dynamic dns client on the router or a script to do dshield reports. Anyway, you can do all the qos stuff pretty easily event if you are fairly new to linux. Just install your favorite linux distro, use the shorewall firewall, grab the wondershaper, and follow these directions to adjust the shaper to your needs. Like lowest priority bittorrent and ftp and highest priority ssh, http, and your games. Its probably free if you have an old box laying around too.
-
Re:BitTorrent's usefulness?
I find it to be the case more often than not
Yes, thanks to the TCP/IP protocol, this is the case more often than not.
Because the TCP/IP protocol requires you to ACK everything you download, if you cram your upstream pipe full of junk, your ACK packets are going to be delayed a nice long time, causing your download to stop while the other end wonders what happened to you.
Fixing this is as simple as limiting your upload rate. Or if you want to discover the internet as it was really meant to be on broadband, implement a Quality of Service setup that prioritizes ACK packets and watch in amazement as everything seems to go faster when under load. -
Re:Bartering?
Traffic shaping really helps with this. For starters, have a look at wondershaper: http://lartc.org/wondershaper/
-
Fix: Traffic shapingWhen excessive uploads interfere with other traffic, you need a traffic shaper. Linux users are in luck: It's built into the Linux kernel. You just have to enable it. Download the Wondershaper script, set a couple of variables at the top describing your connection, and run it to install the settings in the kernel. Once you have it set the way you like, run it from your boot script to automatically configure your kernel on restart.
Those using a consumer router based on Linux, such as the Linksys WRT54G, may find a way to run the Wondershaper on it. For instance, you can get replacement firmware for the WRT54G from Sveasoft that incorporates the Wondershaper. (Just turn on the QoS feature.)
I use the Sveasoft firmware and add a couple of iptables commands to put my UDP game traffic in the high-priority queue, so P2P uploads don't disturb my gaming. See the Sveasoft forums (registration and $20 required) for details. You'll also want to do this if you use UDP-based VOIP.
-
Re:sounds like a cool idea but
Worse yet, due to the assyemtry, if you let BitTorrent use that full 384Kbps upstream, all other Internet use will be abysmally slow. So you're best off capping it at half that, or so.
You can get around that, at least on Linux, using LARTC. I have set up my box so "miscellaneous" packets (p2p, email, etc) are only sent if there are NO ssh or web browsing packets ready to go (script). There may be a few remnants of wondershaper in there, but I think mine is better :)It does work. With this in place the effect of running BitTorrent (or whatever) in the background is tiny.
-
Re:sounds like a cool idea but
Worse yet, due to the assyemtry, if you let BitTorrent use that full 384Kbps upstream, all other Internet use will be abysmally slow. So you're best off capping it at half that, or so.
You can get around that, at least on Linux, using LARTC. I have set up my box so "miscellaneous" packets (p2p, email, etc) are only sent if there are NO ssh or web browsing packets ready to go (script). There may be a few remnants of wondershaper in there, but I think mine is better :)It does work. With this in place the effect of running BitTorrent (or whatever) in the background is tiny.
-
Re:Firewall ?
That's not entirely accurate. NetFilter can do it. It's really ugly though. You have to tell it where the ACK flag is in the headers because it doesn't know.
From the site I linked:
tc filter add dev ppp14 parent 1:0 protocol ip prio 10 u32 \
match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u16 0x0000 0xffc0 at 2 \
match u8 0x10 0xff at 33 \
flowid 1:3
That will (apparently) prioritize ACK packets with no payload. Then there's the PF way. I can actually write one of these rules myself without copy & pasting some magic out of some HOWTO.
pass out on $external_nic queue( out_standard, out_fast )
You'd probably want to add a "keep state" to that, and maybe other stuff... but that's the basic idea. -
Re:Got plenty of time? eDonkey may rock.You don't actually even need to run it setgid anything, use --pid-owner and some scripting that adapts your firewall rules when you launch mldonkey, or even easier, --cmd-owner (in combination with --uid-owner to prevent other users (if any) programs from matching).
You'll probably want to run it as an entirely different user and chrooted just for security purposes though.
Also, to answer the grandparent post's question some further, have a look at the Linux Advanced Routing and Traffic Control howto.
(this all assumes you're using linux ofcourse, but that should have been clear by now) -
Linksys Linux firewall
There are several Linux options for a WRT54G. Check out LinksysInfo.org. Some include the WonderShaper for prioritizing traffic like VOIP and game packets.
-
Re:FreeBSD Dummynet
FreeBSD's Dummynet is crude in comparison to the Linux kernels QoS faclilities. To mention a few- CBQ, HTB, RED etc. Funny that I don't see much mention of these here though.
Take a look at the Linux advanced routing and traffic control howto.
-
Re:hmmm-They went, all those ways.
Check out LARTC.
-
cheapest onsite redundancy?
-
Re:NAT forwarding
For your query, see the Linux Advanced Routing and Traffic Control page on NAT solution with QoS.
That's overly complex for basic traffic shaping. See Wonder Shaper instead. -
Re:NAT forwarding
I doubt it - they are writing software to run on top of the OS (like zebra/quagga does).
The Linux Router Project created a distribution (for routing) around Zebra on Linux and other things. IMHO there are big advantages to being able to specify everything in a single file/config (a la vendor-C, vendor-J, etc) and create an easy-to-install routing platform using something like the old LRP.
For your query, see the Linux Advanced Routing and Traffic Control page on NAT solution with QoS.
- Ivan -
Re:NAT forwarding
I doubt it - they are writing software to run on top of the OS (like zebra/quagga does).
The Linux Router Project created a distribution (for routing) around Zebra on Linux and other things. IMHO there are big advantages to being able to specify everything in a single file/config (a la vendor-C, vendor-J, etc) and create an easy-to-install routing platform using something like the old LRP.
For your query, see the Linux Advanced Routing and Traffic Control page on NAT solution with QoS.
- Ivan -
Re:Why in New Mexico!!??This is a dead thread, but I'll answer anyway.
This is going to happen. Your "can't do" attitude indicates you're not going to be the one to do it. That is all.
The commodity hardware is available. The software is available. It has been done before.
This might not be true in your area, but there are enough wide-open broadband + 802.11g access points in my area to anchor a freewan mesh of any size without even paying for internet access at all. Please note that many people (myself included) run their access points wide open deliberately.
Investment cost to host a freewan cell (802.11g) is about $100 upfront and $0 for ongoing costs. To anchor it to an ISP with acceptable TOS (or one that's known to turn a blind eye) is a minimal monthly many of us are already paying. Contrast that with the corpnet million dollar towers and the municipal $50 million dollar 802.11b networks and you begin to understand why the little guy has the advantage here.
When the mesh grows to the point where it's got consumers in the six figures, you can bet somebody is going to want to connect to it badly enough to pay the freight, and then the monthlies go away.
Apart from a solid municipal commitment to fiber-to-the-door, (and perhaps even then because of the side benefits) I don't see this not happening all over the country in the near future.
Of course, YMMV.
-
Re:No thanks
You might want to look at wondershaper(a linux-based traffic-optimisation script) for your ping-times. Even if you aren't running linux, the descriptions, in plain english, of why it's a very bad idea to try (on most dsl/cable networks available today) to do a response-based task(online game) and a large, block-based transfer (bit-torrent, p2p, large downloads, windows update, etc...) are well worth the visit wondershaper
-
Re:Two words.. Hardware Firewall
See also the LEAF Bering firewall/router mini Linux distro. With some tweaking the uLibc version can even do a bridging firewall with traffic control.
With some carefully crafted traffic control/shaping I can now run multiple P2P apps (legitimate ones, FWIW) on multiple machines and still surf the web, download http & ftp, ssh and serve web pages with low latency and high bandwidth while the p2p traffic fills in the gaps. Traffic control is to bandwidth what nice is to CPU utilization.
LEAF Project
LARTC, good info on traffic control. Ignore the crap about giving ACKs priority, though. I quickly found out most p2p traffic packets have ACKs and choke the connection if you give them priority. -
Re:Slashdot and Bittorrent
The Wondershaper can throttle all upstream bandwith with one script, and prioritize ssh traffic too.
-
Throttle your upstream , and use Wondershaper
kyhwana is right - you need to throttle your BT upload speed so that there's room for the acknowledgement packets you are sending to people you are downloading from. On my ADSL connection I have 1.5Mbps down and 256kbps up, so my uplink is theoretically able to handle 256/8 = 32 kbytes/sec, ignoring protocol overhead. I am using the wondershaper script on my firewall, so that gets eroded to 220kbps or about 27 kbytes/sec max throughput.
On a fast torrent I hit my max download speed (120 to 140 kbytes/sec) which uses about 6-8 kbytes/sec of my upload bandwidth for ACKs. So I can run up to 19 kbytes/sec uploading before I see congestion and slowdowns. I usually throttle it back to 15 kbytes/sec to maximize my downloading while allowing me to do other things online while BT is active. BT is set to allow a 10:1 ratio for your download speeds from other leechers. So, if you are uploading at 15 kbytes/sec, you should be able to download at 150 kbytes/sec. If you are downloading from a seed, then the ratio doesn't matter :-).
If I am only seeding, I can just let BT use the maximum uplink speed of my connection, since there isn't anything else downloading. The wondershaper script on my firewall is set up to give low priority to BT packets. Activity on my other machines will get placed at the front of the queue, if there is a queue. The price I pay for low latency for ssh, web browsing, email, and gmaing is the loss of about 15% of my max bandwidth. It's a worthwhile tradeoff. -
Re:You don't need gigabit
Sorry to reply to myself, but I got excited reading about Wondershaper... it's a wonder this isn't more popular! By slightly limiting upload/download bandwidth the ISP's or modem's upload/download buffers never fill and *my* Linux router can control how much to buffer. I'll be able to send my packets right at my SSH servers with much less perceptable lag, even while downloading. Should also give gamers a boost. Nice!
Example (from the README):
Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms
Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms
Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms
With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms
With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms
Very, very trick! I'm going to have to play with this... probably mirror it too...
Thanks man! -
Re:You don't need gigabit
It is a growing project. The traffic management code in the Linux kernel is immensely powerful, but very poorly documented.
Start here. The wondershaper is the kernel of what I did to get that kind of performance. I strongly recommend using the HTB queueing discipline. The strength of the HTB code is enough to get you 60% of the way there. The rest is fine-tuning and careful, scientific measurements of your traffic patterns. -
Linux Advanced Routing and Traffic Control
Time to let your fingers do the walking...
Linux Advanced Routing and Traffic Control
I know this stuff is dense, but I happen to think it's stuff that any serious Linux admin should know about eventually, so I spread the word. If you want some pointers on where to start, send me an IM. I'll be at work all day today more-or-less. -
Simple solutionI had a similar issue with my DSL connection at home. Using SSH was really painful, when someone was downloading...
A lot of people got down to the nitty gritty technical details, but as I understand you want something simple that just works. Well, I use a Linux Firewall distro to do the routing in combination with a small script to configure the QoS.
Try Clarkconnect in combination with Wondershaper. Wondershaper uses some basic input parameter to configure the kernel to traffic prioritization. I found it very easy to define my available bandwidth, what services require a higher or lower priority.
-
my rc.iprules script
See:
http://hibernia.jakma.org/~paul/rc.iprules
For a script that does something similar to what you want, policy routing to route based on source IP. It should be easy enough to add an additional 'firewall mark' field to the table and policy route based on that (i'm on holiday, otherwise i might have done that for you). The listed "intranets" will use the main table.
Basically, all you need is:
1. create a table for each policy (edit /etc/iproute2/rt_realms)
2. use iptables to add arbitrary 'fwmarks' to incoming packets based on whatever criteria you have
3. use the 'ip rule' command to direct routing for packets with specific fwmarks to specific routing tables.
4. direct other traffic to the default 'main' table.
Finally, see the Linux Advanced Routing & Traffic Control site for further information. -
IPTables and QoSA Linux box with 3 network cards and some IPTables and QoS should do what you're looking for. Take a look at the Linux Advanced Routing and Traffic Control HOWTO for the nitty-gritty QoS details, and here for the routing parts
.Basically, you'd be looking at doing the following things. Multiple outbound providers, which will need another routing table built for the second link. Then you'll need to dive into QoS to split up your traffic into your definitions of bulk (HTTP, FTP), priority (Gaming), and drop (P2P). I notice that you have no default set up, but I leave that up to you. Finally, you can use iptables to mark and NAT your traffic out the right interface.
Under Windows, you would need some advanced routing software I think. ISA may do it, but I doubt your budget allows it. By default, Windows does have the ability to enforce QoS terms, but you'd need something to apply those QoS marks (I doubt that games commonly mark their packets with ToS)...which means a bridge in front of the Windows router. Might as well use a Linux router instead.
If anybody knows of a way to get a Windows box to route based on ports, I'd love to hear it.
Oh, and a simple solution for the exact problem you describe (which I don't think is what you really want) would be a proxy for the HTTP and FTP link, and a router for the other link. All HTTP and FTP requests would be sent out the proxy, everything else would go the default route (to the router) which could be configured to drop P2P and route everything else. Optionally, you could do QoS on the router to prioritize certain traffic. If you go that route, I'm fond of AnalogX Proxy (for Windows) because it's free and simple. Of course, that does require client configuration....unless you use Transparent Proxying.
-
IPTables and QoSA Linux box with 3 network cards and some IPTables and QoS should do what you're looking for. Take a look at the Linux Advanced Routing and Traffic Control HOWTO for the nitty-gritty QoS details, and here for the routing parts
.Basically, you'd be looking at doing the following things. Multiple outbound providers, which will need another routing table built for the second link. Then you'll need to dive into QoS to split up your traffic into your definitions of bulk (HTTP, FTP), priority (Gaming), and drop (P2P). I notice that you have no default set up, but I leave that up to you. Finally, you can use iptables to mark and NAT your traffic out the right interface.
Under Windows, you would need some advanced routing software I think. ISA may do it, but I doubt your budget allows it. By default, Windows does have the ability to enforce QoS terms, but you'd need something to apply those QoS marks (I doubt that games commonly mark their packets with ToS)...which means a bridge in front of the Windows router. Might as well use a Linux router instead.
If anybody knows of a way to get a Windows box to route based on ports, I'd love to hear it.
Oh, and a simple solution for the exact problem you describe (which I don't think is what you really want) would be a proxy for the HTTP and FTP link, and a router for the other link. All HTTP and FTP requests would be sent out the proxy, everything else would go the default route (to the router) which could be configured to drop P2P and route everything else. Optionally, you could do QoS on the router to prioritize certain traffic. If you go that route, I'm fond of AnalogX Proxy (for Windows) because it's free and simple. Of course, that does require client configuration....unless you use Transparent Proxying.
-
Re:Linux Advanced Routing and Traffic Control.
I know what you mean, but I also wanted to let everyone know with my original post that ingress policing under Linux solves this problem.
http://lartc.org/howto/lartc.adv-filter.policing.h tml
(If that isn't easy to understand, keep in mind it's section 12 of a long HOWTO with lots of conceptual material. If you start reading from the beginning, and skip sections that don't involve your problem, everything should start making sense.)
For example:
tc filter add dev $DEV parent ffff: \
protocol ip prio 20 \
police rate 640kbit buffer 10k drop \
flowid :1
This adds a simple filter rule that limits inbound traffic to 640 kbit/sec and drops matching *outbound* traffic to slow down inbound traffic. You don't have to do the whole line at once: this is class-based and the above example assumes only one class, so you could add several classes, one for each user, and make them borrow from each other when others' classes aren't maxed out. (Just don't make them 'bounded' or 'isolated' and you get this borrowing for free.)
What would the end result of this be? If you set up seven queues (using u32 filters on each rule to match each individual user) all underneath one parent queue representing the entire downstream pipe then you get some interesting and fair behavior:
Suppose you have a max line speed of around 70 KB/sec, and seven active users. Six are well-behaved, doing one HTTP file download each, and the seventh is running as many filesharing clients as he possibly can. Each user would notice about 10 KB/sec download speed on their download. (If one user's download wasn't capable of going any faster, then maybe the other guys would borrow his spare KB/sec and go a bit faster, until he downloaded something else that used all of his allocation.) The badly-behaved filesharing user, though, might be attempting 30 downloads at once. He'd still be getting 10 KB/sec across all of the connections though. The other endpoints of his connections may be trying to send him faster than 10 KB/sec, but traffic policing will notice this is over his limit and will cut off his acks to match. So this one user may notice an 80% frame loss rate and almost useless web browsing, but everyone else will have pretty much normal service.
And the best part is: when things aren't busy any more, this "partitioning" of bandwidth just neatly gets out of your way and shares any unused classes with other users. So at 4 AM this filesharing user might get 70 KB/sec across all of his downloads. But if someone pops on to check their mail, this user's downloads will get pushed down to make room for the new user.
Another thing to keep in mind: for these filters to work well, you need to give them some overhead. If your actual linespeed is 640 kbit/sec, set the filters to a max of 620 kbit/sec. This way it can detect and act upon overlimit conditions before inbound and outbound queues start filling up. If you set the ingress filter to a max which is the same as your line speed, you won't be able to detect when people are sending packets faster than you can receive them: your ISP will be helpfully buffering your packets in an inbound queue and adding tons of latency.
So to recap: this ingress policing will work for you too. You'll have to learn the weird way these filters work -- but they're very powerful. As with most learning in Linux, half of the documentation-reading work is bringing you up to speed with the universal concepts needed to understand what you want to do, and the other half is understanding how those concepts translate into this specific bit of software.
So if the person who posed the original question is more familiar with Linux than with your OS, it sounds like they can accomplish the same thing with Linux. No need to force someone to switch just yet. :-)
--Michael Spencer -
Linux Advanced Routing and Traffic Control.
http://lartc.org
It's difficult to understand, much less set up, but essentially the stuff from this site can solve your problem by tightly controlling outbound traffic (since it is possible to have perfect control over what packets you release to the network) and by loosely attempting to control inbound traffic (since it isn't really possible to perfectly control what packets other people send you).
For example, my home setup has four priority classes:
Class 0:10 is for high priority traffic: ping replies, TCP ACK packets, and online gaming.
Class 0:20 is for everything not otherwise classified.
Class 0:30 is for BitTorrent traffic -- lower than normal, but higher than all the other p2p stuff. I do this because BitTorrent traffic is very likely to be directly related to a file I'm personally interested in.
Class 0:40 is for lowprio.mspencer.net and other misc filesharing programs. If the rest of the Internet connection is busy, class 0:40 ends up with around 24 kbit/sec out of my total 640 kbit/sec upstream.
I guarantee you can adapt this as needed, so each user has a fair slice of upstream available, but if someone's not using their slice then everybody else can split it. (So at 4 AM one user can still get the whole line speed, but at peak usage everybody gets the same bandwidth.)
The other side of the coin is ingress policing. I don't have a lot of experience with this, but you'll almost definitely need it. Basically the policing module tries to slow inbound packets by throttling the outbound acknowledgements. It's not perfect but it can help.
Some filesharing programs incorrectly state they are "firewalled" when you use a setup like this. Instruct the user to just tell his client to retest so it can confirm he's not firewalled.
My final paper for my 4000/8000 level networking class was regarding my traffic shaper. Maybe it'll help.
http://mspencer.net/traffic-shaper.doc in Word 2000 format.
http://mspencer.net/traffic-shaper.txt in plain text.
--Michael Spencer -
Re:Ouch. This is going to hurt.
Well, basically I just followed the directions on these sites:
http://lartc.org/howto/lartc.ipsec.html
http://www.ipsec-howto.org/t1.html
Get yourself a late model 2.4 kernel and follow the directions for 2.6. Everything works the same. If you use Debian 'testing' or 'unstable' the other packages you'll need are ipsec-tools and then racoon (KAME) or isakmpd.
It's actually pretty easy if you just follow the examples. -
Wondershaper...
For those of you that don't know, and are interested, Wondershaper can be found HERE.
It is AMAZING.
Sample config:
DOWNLINK=6000
UPLINK=200
DEV=eth0
# low priority source ports
NOPRIOPORTSRC="6881 6882 6883 6884 6885 6886 6887 6888 6889 80"
Sets those ports to only use up 200k of my 256k upstream leaving me the rest for SSH etc. I never have any problems w/my remote connection speeds this way. It's fantastic.
I have only had a single problem, recently, with Debian unstable... It removed my libatm for some reason. I reinstalled that and all was well.
Highly recommended for everyone, not just users of this "hackable" router. -
Time to get smart about your bandwidth...... and set up a shaper on your ISP link that slows down your outbound BitTorrent traffic. Me, I use a SmoothWall box with a regular old Wondershaper script. Keeps my DMZ traffic in line (so it doesn't choke my isp link) and works well enough for a system that you don't have to twiddle the knobs on too much.
(Yes, I read the docs for tc, and I'd love to have an HTB shaper instead of the standard qdisc one I use, but I'm too busy to spend that much time for the small advantages a truly custom firewall box would offer.) -
Re:Some thoughts here
I've never ever figured out traffic-shaping in linux--and I doubt many have.
RTF HOWTO =)
-
Traffic shaping
I believe Linux already has some pretty sophisticated "traffic shaping" features, where you can set priorities for different streams based on source, destination, and a handful of other properties of the TCP stream. See the Linux Advanced Routing and Traffic Control website.