Slashdot Mirror
Apache Bandwidth Limiting?
IOOOOOI asks: "I work at a high traffic web hosting company and we're trying to find a simple effective way to limit bandwidth hogs, some of whom we've clocked pulling over 4Gb/hr off our servers. We've tried mod_throttle and have looked into QoS/fair queuing as well as a couple of custom solutions in-house. None of these quite did the trick. Has anyone found an effective way to do this, one that can handle individual connection streams?"
But the solution seems easy to me. Simply charge your customers for their bandwidth.
This rectifies the disparity between flat rate pricing and incremental bandwidth costs.
When I went to find a solution for my web appliction, I chose to put in a DSL line and host it myself (Because of the complexity of the app, this is cheaper than colocating my computers there).. but I chose a DSL provider that doesn't give "all you can eat" - but instead charges for bandwidth.
The reason I chose this is the theory that the bandwidth hogs would go elsewhere and the latency at this ISP would be much lower. so far this has proven tru, and I've yet to exceed the basic "free" bandwidth level.
If, on the other hand, you're talking bout people who are downloading your customers content at huge rates, then maybe you should charge your customers based on the service they are providing. If they're hosting lots of large files, they should probably be paying more...
Dunno if that's a viable solution-- but smart customers will prefer someone who charges "by the byte"... because the bytes are better quality.
Yeah, and you guys panned the ipod too: http://apple.slashdot.org/article.pl?sid=01/10/23
try altqd. i've only used it on openbsd, but with it you can selectively throttle bandwidth.
Big Daddy, Johnny, Burp, Aunt Zelda, Scott, Slurp, Big Momma
Not an Apache based solution, but check out Packeteer Packetshapers..specifically the ISP models.. lets you set SLA's by protocol, IP, etc, perform rate limiting, and all other kinds of really cool stuff. Not exactly cheap but extremely effective, and simple to manage.
I have used mod_bandwitdth to a certain extent, it may have what your looking for. I would love to hear about other solutions though.
The solution that the (defunct) etoys.com adopted for their site was based on code from one of my Perl columns. My code is based on CPU throttling, but you can quickly change it to bytes sent using the same technology.
I use mod_bandwidth at work to simulate 56k connections to the web server.
It works quite well and will throttle per-connection or per-virtualhost.
I'm not verey experienced with bandwidth limiting.
. html
:-) should be doing perfect.
I did play with mod_throttle, and all it did was actually allow all traffic until the limit was reached, and then deny the next new connections. Hmm, not too great actually.
I'm planning to try out mod_bandwidth, but I dunno if it works different.
Bad link (sorry, I don't feel for html now):
http://www.cohprog.com/v3/bandwidth/doc-en
I tried playing with QoS on linux 2.4.
According to the documentation it's actually quite hard to have that functional, because if you have a 10 Mbit connection, it will shape the traffic elative to that. But 10 Mbit is not always the same. If you have lots of lost packets it will behave different then with a perfect connection.
In my experience I couldn't reliably limit the traffic on a 10 Mbit connection down to 80 kbit (almost 1% of the 10 Mbit). My cable connection of 16 kbyte still could get choked.
Maybe I should just get a card of 1 Mbit and try again, the numbers might be better then.
Or hey, a card of 100 Kbit
Well, don't worry about that. We can get you back before you leave. (Dr. Who)
If anything, this should be modded troll, not flaimbait. It's just a typical FIRST POST, not anybody saying microsoft is good and should be allowed to rule the world. Since when was first post considered flaimbait?
You could look at using a combination of content acceleration and bandwidth pools in squid. I've used these features before and it actually works pretty well for static content. You can tune the caching params to allow for large files, etc.
Derek
Don't Panic...
I am having the same problem, and I think you guys are missing the point. He said 4GB an hour, which means he probably has an OC-3, OC-12, or Gigabit Ethernet connection.
"Blocking" network appliances such as Packeteer can't handle these high rates, and even if they had gigabit interfaces, they would only be able to do 600-800mbps on them.
None of the kernel QoS/queueing options I've seen allow for anything other than classifying traffic or "fair" queueing. None of this seems to help someone that wants to limit all webserver connections to 2mbps - everything here is expecting an IP range, ports, or something to distinguish by. What if I don't want to?
Apache needs real per connection, per user, and per IP rate limiting. mod_throttle and everything else I've seen has to starve connections after they perform too well. How about something that hard limits connections to 2mbps/sec. I will pay for anything that can do that for Apache today...
Forgive me if I have overlooked the obvious...
You know I'd love to help... but I can't check out your server's particulars 'cos you didn't include a URL
Have you considered using FreeBSD Traffic Shaping? ("man ipfw").
here is a story to a problem that sounds identical to yours. A hosting company (using a virtual host) has a customer who uses exessive bandwidth, and they wish to throttle it. After trying mod_throttle, they went with a better solution.
If your not using FreeBSD, i am very suprised. Perhaps you should look into it.
D.
You can tell how powerful someone is by the magnitude of the crime they can commit and be able to get away with.
So, you could say "xyz.dom may only use 200kbps for *.mpg files".
thttpd is especially suited for serving static files; it is not an all purpose machine such as Apache.
You can find out more information at thttpd's homepage.
See http://www.cisco.com/warp/public/105/policevsshape .html
for a good tutorial on Traffic Policing and Traffic Shaping, two ways of doing what you require with Cisco hardware.
Cisco has a great IOS feature called CAR that can do exactly what you're asking for at the router level. You can rate-limit specific physical ports on the router (even using a schedule such as from 8am to 8pm, allow anything, from 8pm to 8am throttle to xxx kbytes/second).
This is assuming that you're not running virtual hosting (multiple domains sharing one IP address), in which case all customers on that IP/physical port would be affected by the CAR limitations you would impose. It is possible with the amount of traffic you're talking about. Just make sure that the puppy has a good processor and plenty of RAM.
It's better to burn out than to fade away
High traffic and Apache is almost an oxymoron. If you are running a high traffic web hosting company, then you need to stop playing games and use Zeus. Apache has its strong points, like being free and open source, but that's about it. If Zeus was free, then it wouldn't just be the best web server for UNIX platforms, it would also be the most popular.
You want Zeus because it is high performance (it doesn't use the toy process-per-connection model). It comes with an easy to use, powerful web based GUI. The GUI doesn't just hold your hand. It lets you set everything, and then will show you the exact lines that are changing in the config files.
It doesn't use extremely complex format for config files that Apache uses. A good comparison is BIND and djbdns. Do you want to try and deal with the incredibly complex BIND zone files, or the simple, one record per line data files that djbdns uses? Zeus config files are one record per line of the form "modules!throttle!enabled yes". It also comes with tools that let you do everything from scripts. But only if you want to. Otherwise, use the GUI.
And speaking of throttling, Zeus does it correctly, unlike any other web server (at least any of the freely available UNIX ones, as that is all I am familar with). It will let you set a limit on the number of users, or set a max number of bytes per second on a virtual server or subserver level. It doesn't serve some people at max speed and then start dropping connections (mod_throttle) or set the throttle speed at the beginning of the request, then start dropping connections (thttpd).
Virtual servers in Zeus actually make sense. There is no master server configuration like in Apache. Instead, you create one or more virtual servers. As such, each virtual server has its own separate configuration. Virtual servers can serve a single website, or any number of websites, via subservers. Subservers all share the configuration of the virtual server (kind of like Apache's mass virtual hosting only much better). No more restarting the server to add a site. Simply create the directory, and it starts serving the site.
There are plenty of other reasons why Zeus is superior to Apache, but the ones I listed should be enough to start considering it. No, I don't work for Zeus or own stock (don't think they have any) or anything like that. I'm just a satisfied customer.
For some things, Apache works just fine. But for anything high traffic, requires throttling or needs a flexible or scripted configuration, Zeus beats Apache hands down. It's worth every penny. Check it out. I doubt you'll be dissapointed.
(subconscious message to Apache developers: stop being lazy and make Apache more like Zeus!)
how much are you going to pay me then?
:-P
Using ipfw and dummynet on freebsd is the way
I have gone in a VERY high traffic hosting
and colo company.
you can not only simultae a link of a certain speed, but can also limit any ip that hits a certain destination to a max speed...
how about a script that goes through the output of netstat every 5 minutes and adds entries to a table. If that table shows its "interesting" traffic, then nail it with something like ipfilter or just set it to a null route. In the case of a dedicated hosted serer, stick in another ethernet card and route all the funny trafic to it and let the switch or router set it to something slow. Its amazing what a perl script, a setuid wrapper for route and a 10mb ethernet card will do.
I would use ipfw to limit bandwith on that port. Everything can be done with the pipe option.