I am having the same problem, and I think you guys are missing the point. He said 4GB an hour, which means he probably has an OC-3, OC-12, or Gigabit Ethernet connection.
That's only 9.1 mbps. T1 = 1.544, T3 = 44.736, OC1 = 51.84. OCx = OC1 * x.
The issue is not about our customer's bandwidth consumption and how much they can/can't use. It's about being able to provide services to all of their users without experiencing slowdowns because of the occasional hog.
Are your slowdowns bandwidth or CPU based? If you are serving lots static content (like porn), then Apache is going to kill you, due to its process-per-connection model, which the developers refuse (read: are too lazy) to fix. Zeus doesn't have this problem. Neither do the open source boa or thttpd (but they unfortunately lack many important features that may stop them from being used for commercial web hosting). Zeus will allow you to max out your network card (100mbit) on a modest machine (P3/500 w/ 1gb RAM).
High traffic and Apache is almost an oxymoron. If you are running a high traffic web hosting company, then you need to stop playing games and use Zeus. Apache has its strong points, like being free and open source, but that's about it. If Zeus was free, then it wouldn't just be the best web server for UNIX platforms, it would also be the most popular.
You want Zeus because it is high performance (it doesn't use the toy process-per-connection model). It comes with an easy to use, powerful web based GUI. The GUI doesn't just hold your hand. It lets you set everything, and then will show you the exact lines that are changing in the config files.
It doesn't use extremely complex format for config files that Apache uses. A good comparison is BIND and djbdns. Do you want to try and deal with the incredibly complex BIND zone files, or the simple, one record per line data files that djbdns uses? Zeus config files are one record per line of the form "modules!throttle!enabled yes". It also comes with tools that let you do everything from scripts. But only if you want to. Otherwise, use the GUI.
And speaking of throttling, Zeus does it correctly, unlike any other web server (at least any of the freely available UNIX ones, as that is all I am familar with). It will let you set a limit on the number of users, or set a max number of bytes per second on a virtual server or subserver level. It doesn't serve some people at max speed and then start dropping connections (mod_throttle) or set the throttle speed at the beginning of the request, then start dropping connections (thttpd).
Virtual servers in Zeus actually make sense. There is no master server configuration like in Apache. Instead, you create one or more virtual servers. As such, each virtual server has its own separate configuration. Virtual servers can serve a single website, or any number of websites, via subservers. Subservers all share the configuration of the virtual server (kind of like Apache's mass virtual hosting only much better). No more restarting the server to add a site. Simply create the directory, and it starts serving the site.
There are plenty of other reasons why Zeus is superior to Apache, but the ones I listed should be enough to start considering it. No, I don't work for Zeus or own stock (don't think they have any) or anything like that. I'm just a satisfied customer.
For some things, Apache works just fine. But for anything high traffic, requires throttling or needs a flexible or scripted configuration, Zeus beats Apache hands down. It's worth every penny. Check it out. I doubt you'll be dissapointed.
(subconscious message to Apache developers: stop being lazy and make Apache more like Zeus!)
I remember a while back, one of my clients needed to move a bunch of dns records from one server to another. Took me ~ 45 minutes to write a php shell script using REGEX to create new bind zone records for over 300 domains, and convert them - records intact, complete, ready to restart named.
Forty five minutes? Wow. Had you been using djbdns, you could have been done in thirty seconds. The BIND zone file format is needlessly complex.
I used to use Register.com, since it had what I thought was a good interface, and free DNS. Then I found out about directNIC, transferred all my domains without problems and have never looked back. directNIC has an awesome interface (great if you have lots of domains), is less than half the price ($15) and a lot of optional extras (free hosting, free email forwarding, free parking and redirection, DNS, POP3 accounts).
This reliability makes it good for things that need zero corruption (file transfers for example), and aren't time criticial.
TCP and corruption is an interesting subject. TCP's error detection is only designed to solve the problem of data not being delivered. It is not designed to stop data corruption. If the transport layer corrupts data, then eventually, TCP could deliver incorrect data.
What happens if data is corrupted so that a corrupt checksum matches corrupt data? Eventually, from a probability standpoint, there will be undetected errors.
Would you mind explaining how Spamcop works? I've been to the site, but I just don't "get it."
What exactly does it do, and how does sending a piece of spam you received to them help *you* get less spam?
It helps you because it automates sending abuse reports. Whether or not sending abuse reports actually helps cut back on the amount of spam is anyone's guess, but it keeps you from having to do it manually.
What indiigo was talking about is different than reporting spam to SpamCop. He purchased a filtered email account from them. For a small price ($30 a year) you can an account that is guaranteed to be virtually spam free. They either give you a new address, or if you can forward mail, then you can filter your mail through them. Not a bad deal, if spam really bothers you.
I always use my ring finger to type P;/ (and the other keys further right), so 'pwd' is easy to type. I also use my ring finger to type Q, and do all shifting with my right pinky. I never really noticed that until reading this article.
Dude. If I had to compile in support for all possible hardware devices, my kernel would make xp look light.
You're missing the point entirely, as did the original poster. The modules don't have to be part of the kernel. You can have modules that only get to be loaded at boot time. This solves the problem of having to being to be loaded and unloaded at run time, and all the various problems that go along with it.
I don't think anyone wants to get rid of modules entirely and put everything into the kernel. They want to eliminate run time loadable and unloadable modules.
Hot swappable hardware is different, as it's designed to be hot swappable. The individual devices can be hot swapped, but the base hardware cannot. You would need to load the module for a USB controller at boot time. The USB devices that use it would be controlled through this driver, and wouldn't need to be part of the kernel.
Re:modules, and why Rusty is wrong:
on
Kernel Summit Wrapup
·
· Score: 2, Insightful
If you swap ethernet cards, then it will exist when you boot the kernel, and you can configure it to be loaded with the kernel, at boot time. Being a module doesn't mean it has to be loadable at run time.
So, I lay in wait and slipped in a bid of $251.55 right at the last minute, which is a good thing because a bid sniper tried to take it but his max bid was $250.
Why wouldn't you just bid the max that you were willing to pay for it? Ebay's proxy bidding system should keep you from worrying about bid sniping.
It gets a lot cheaper when you buy more. We had four OC3's (OC12 equivalent) from AT&T and were paying $175,000 a month (though we also had a CTO with awesome bargaining skills).
Well about the only way you could realisticly disabled chunked encoding off the top of my head is force the server to only support HTTP 1.0 rather than HTTP 1.1 with a directive such as:
That's wrong. The bug involves chunked encoding on requests, not responses.
Each daemon should have it's own UID, with file permissions set accordingly, ie. write access to the pid and log files, and usually nothing else, not even/tmp (if it needs a temp folder, it gets it's own.)
That's why you set the sticky bit on/tmp. Temporary files that should not be read by other users are created without world read permissions. The sticky bit prevents other users from modifying files that they do not own.
How many people using Linux/Unix/X need that professional kerning or font management? I say maybe around 5% of users.. they are better off using Mac OSX of W2k, anyways. I'm happy with anti-aliasing certains fonts and leaving other fonts unaliased. Fonts have looked the same to me since last year in X compared to Windows
One of the reasons I use Windows 2000 on my primary desktop is because the fonts simply look better. If you do a side by side comparison, then you'll see that the smaller fonts (the ones that can't be anti aliased) simply look better.
Applications that take data from untrusted sources, like web browsers, should course make sanity checks. So the error is in Mozilla, not X11.
They should in some, but not all, cases. That's why rlimits exists. Certain classes of applications should not have to check everything for themselves. For example, the qmail SMTP server can be made to allocate an arbitrary amount of memory by feeding it a huge list of recipients. This is not a bug. It is designed to be run with resource limits, usually set using
softlimit. It is bad engineering to include needless checks in every single application, when the OS has this built in.
BTW, if any sympatico users with debian boxes running ipmasq are worried about keeping Sympatico honest about their usage, apt-get install ipac.
Better yet, install MRTG. The mrtg-ip-acct program will read the IP accounting statistics directly from your iptables firewall. There is no need for the depracated ipchains compatibility module. Simply create a config file for it, such as/etc/mrtg.cfg:
WorkDir:/var/www/mrtg/
WriteExpires: Yes
Title[eth0]: Traffic Analysis for tourian
PageTop[eth0]: Traffic Analysis for tourian
Target[eth0]: `/usr/bin/mrtg-ip-acct eth0`
MaxBytes[eth0]: 12000000
You will need to have it run every five minutes, using cron, so add an entry to your crontab:
*/5 * * * * root/usr/bin/mrtg/etc/mrtg.cfg
Because the default page generated is named.html, for some reason, certain browsers (Internet Explorer) want to cache it no matter what you do. So the easiest thing to do is to wrap it in a PHP script, such as index.php:
My cable modem provider limits us to one gigabyte per day. After numerous arguments with my roommate, including how exactly to read and interpret these graphs, and because they do not cover a day exactly (we are supposedly counted from midnight to midnight), I wrote a simple PHP script to modify the MRTG output with nice, easily readable usage statistics:
The numbers at the top of the page are always live, since MRTG graphs are not. Note that the page has a latency of at least one second, because it takes two counter readings, one second apart, to generate the current usage rate. This is not always totally accurate, but should be close enough. Reloading the page a couple of times may give better results. The source to the script is available here:
I agree completely. The PHP developers seem very misguided. They refuse to fix many bugs in the name of "performance". Things like infinite recursion crashing the interpreter. At the very least, they could include a debug version, which would run slower, but help developers find errors. Things like the new private member variables silently erroring is NOT acceptable behavior for a scripting language, especially one that claims to be easy to use.
PHP already has enough bugs that make it hard to use, especially for those not experienced with it. It's really fun when a syntax error causes the parser to indicate that the error is hundreds of lines away from where it really is, or even in a separate file. Adding MORE bugs of this kind in the name of "performance" does not help to make PHP more professional. I really like PHP, and have been using in commercial environments for more than two and a half years, but these types of things are making me seriously consider switching to Python.
They charge ~$50/month, dial-up ISPs charge ~$20/month for unlimited access. Since there are no other benefits to you other than faster speeds and always-on access, you're basically paying $30/month for bandwidth, for using a fat piece of their pipe (versus 5k/sec if you're dialup). So, how much bandwidth does $30/month buy?
I believe that you're forgetting what is probably the biggest benefit to broadband: decreased latency. Usually "faster speeds" means more bandwidth. For me, the 3-4k/sec that dialup provides would be quite usable for 95%+ of the time that I use the computer (which is a lot, considering that I telecommute every day). But the 200-300ms latency makes the connection unusable for me. Sure, web surfing is annoying when the connection is slow, but it's usable. Trying to work over SSH is almost impossible (and forget playing StarCraft).
If you're going to charge me between $40 and $60 a month for a capped service to begin with, I hope you're doing quality control.
Cable ISP's don't do the capping on the ISP end. The capping is done on the customer end, inside the modem. If you read the recent Slashdot article on cable modem uncapping, then you can see more about this.
Have you noticed that when hit your upload limit that the latency goes WAY up (say 30ms -> 1000ms)? This is due to the cable modem having an internal buffer and cap. You can fill up the buffer very quickly, but it takes it longer to empty. If the throttling was done on the ISP's side without buffering, then this wouldn't happen.
Umm, no. Bandwidth at OC-12+ rates is less than $400/month for 1.5mbps (T1) rates. And someone as large as Comcast is probably using significantly more, and thus getting a much better deal. Or they're a tier 1 provider and pay virtually nothing.
If ISP's were smart, they would peer with web hosting providers, and sell their outgoing bandwidth, since the majority of an ISP's usage is incoming.
Slashdot Mirror
I am having the same problem, and I think you guys are missing the point. He said 4GB an hour, which means he probably has an OC-3, OC-12, or Gigabit Ethernet connection.
That's only 9.1 mbps. T1 = 1.544, T3 = 44.736, OC1 = 51.84. OCx = OC1 * x.The issue is not about our customer's bandwidth consumption and how much they can/can't use. It's about being able to provide services to all of their users without experiencing slowdowns because of the occasional hog.
Are your slowdowns bandwidth or CPU based? If you are serving lots static content (like porn), then Apache is going to kill you, due to its process-per-connection model, which the developers refuse (read: are too lazy) to fix. Zeus doesn't have this problem. Neither do the open source boa or thttpd (but they unfortunately lack many important features that may stop them from being used for commercial web hosting). Zeus will allow you to max out your network card (100mbit) on a modest machine (P3/500 w/ 1gb RAM).High traffic and Apache is almost an oxymoron. If you are running a high traffic web hosting company, then you need to stop playing games and use Zeus. Apache has its strong points, like being free and open source, but that's about it. If Zeus was free, then it wouldn't just be the best web server for UNIX platforms, it would also be the most popular.
You want Zeus because it is high performance (it doesn't use the toy process-per-connection model). It comes with an easy to use, powerful web based GUI. The GUI doesn't just hold your hand. It lets you set everything, and then will show you the exact lines that are changing in the config files.
It doesn't use extremely complex format for config files that Apache uses. A good comparison is BIND and djbdns. Do you want to try and deal with the incredibly complex BIND zone files, or the simple, one record per line data files that djbdns uses? Zeus config files are one record per line of the form "modules!throttle!enabled yes". It also comes with tools that let you do everything from scripts. But only if you want to. Otherwise, use the GUI.
And speaking of throttling, Zeus does it correctly, unlike any other web server (at least any of the freely available UNIX ones, as that is all I am familar with). It will let you set a limit on the number of users, or set a max number of bytes per second on a virtual server or subserver level. It doesn't serve some people at max speed and then start dropping connections (mod_throttle) or set the throttle speed at the beginning of the request, then start dropping connections (thttpd).
Virtual servers in Zeus actually make sense. There is no master server configuration like in Apache. Instead, you create one or more virtual servers. As such, each virtual server has its own separate configuration. Virtual servers can serve a single website, or any number of websites, via subservers. Subservers all share the configuration of the virtual server (kind of like Apache's mass virtual hosting only much better). No more restarting the server to add a site. Simply create the directory, and it starts serving the site.
There are plenty of other reasons why Zeus is superior to Apache, but the ones I listed should be enough to start considering it. No, I don't work for Zeus or own stock (don't think they have any) or anything like that. I'm just a satisfied customer.
For some things, Apache works just fine. But for anything high traffic, requires throttling or needs a flexible or scripted configuration, Zeus beats Apache hands down. It's worth every penny. Check it out. I doubt you'll be dissapointed.
(subconscious message to Apache developers: stop being lazy and make Apache more like Zeus!)
I remember a while back, one of my clients needed to move a bunch of dns records from one server to another. Took me ~ 45 minutes to write a php shell script using REGEX to create new bind zone records for over 300 domains, and convert them - records intact, complete, ready to restart named.
Forty five minutes? Wow. Had you been using djbdns, you could have been done in thirty seconds. The BIND zone file format is needlessly complex.I used to use Register.com, since it had what I thought was a good interface, and free DNS. Then I found out about directNIC, transferred all my domains without problems and have never looked back. directNIC has an awesome interface (great if you have lots of domains), is less than half the price ($15) and a lot of optional extras (free hosting, free email forwarding, free parking and redirection, DNS, POP3 accounts).
This reliability makes it good for things that need zero corruption (file transfers for example), and aren't time criticial.
TCP and corruption is an interesting subject. TCP's error detection is only designed to solve the problem of data not being delivered. It is not designed to stop data corruption. If the transport layer corrupts data, then eventually, TCP could deliver incorrect data.
What happens if data is corrupted so that a corrupt checksum matches corrupt data? Eventually, from a probability standpoint, there will be undetected errors.Would you mind explaining how Spamcop works? I've been to the site, but I just don't "get it."
What exactly does it do, and how does sending a piece of spam you received to them help *you* get less spam?
It helps you because it automates sending abuse reports. Whether or not sending abuse reports actually helps cut back on the amount of spam is anyone's guess, but it keeps you from having to do it manually.
What indiigo was talking about is different than reporting spam to SpamCop. He purchased a filtered email account from them. For a small price ($30 a year) you can an account that is guaranteed to be virtually spam free. They either give you a new address, or if you can forward mail, then you can filter your mail through them. Not a bad deal, if spam really bothers you.I always use my ring finger to type P;/ (and the other keys further right), so 'pwd' is easy to type. I also use my ring finger to type Q, and do all shifting with my right pinky. I never really noticed that until reading this article.
Have the customer install Lavasoft's wonderful Ad-aware. This freely available program should remove any spyware that is installed.
Dude. If I had to compile in support for all possible hardware devices, my kernel would make xp look light.
You're missing the point entirely, as did the original poster. The modules don't have to be part of the kernel. You can have modules that only get to be loaded at boot time. This solves the problem of having to being to be loaded and unloaded at run time, and all the various problems that go along with it.
I don't think anyone wants to get rid of modules entirely and put everything into the kernel. They want to eliminate run time loadable and unloadable modules.
Hot swappable hardware is different, as it's designed to be hot swappable. The individual devices can be hot swapped, but the base hardware cannot. You would need to load the module for a USB controller at boot time. The USB devices that use it would be controlled through this driver, and wouldn't need to be part of the kernel.If you swap ethernet cards, then it will exist when you boot the kernel, and you can configure it to be loaded with the kernel, at boot time. Being a module doesn't mean it has to be loadable at run time.
So, I lay in wait and slipped in a bid of $251.55 right at the last minute, which is a good thing because a bid sniper tried to take it but his max bid was $250.
Why wouldn't you just bid the max that you were willing to pay for it? Ebay's proxy bidding system should keep you from worrying about bid sniping.It gets a lot cheaper when you buy more. We had four OC3's (OC12 equivalent) from AT&T and were paying $175,000 a month (though we also had a CTO with awesome bargaining skills).
Well about the only way you could realisticly disabled chunked encoding off the top of my head is force the server to only support HTTP 1.0 rather than HTTP 1.1 with a directive such as:
That's wrong. The bug involves chunked encoding on requests, not responses.Each daemon should have it's own UID, with file permissions set accordingly, ie. write access to the pid and log files, and usually nothing else, not even /tmp (if it needs a temp folder, it gets it's own.)
That's why you set the sticky bit onHow many people using Linux/Unix/X need that professional kerning or font management? I say maybe around 5% of users.. they are better off using Mac OSX of W2k, anyways. I'm happy with anti-aliasing certains fonts and leaving other fonts unaliased. Fonts have looked the same to me since last year in X compared to Windows
One of the reasons I use Windows 2000 on my primary desktop is because the fonts simply look better. If you do a side by side comparison, then you'll see that the smaller fonts (the ones that can't be anti aliased) simply look better.Applications that take data from untrusted sources, like web browsers, should course make sanity checks. So the error is in Mozilla, not X11.
They should in some, but not all, cases. That's why rlimits exists. Certain classes of applications should not have to check everything for themselves. For example, the qmail SMTP server can be made to allocate an arbitrary amount of memory by feeding it a huge list of recipients. This is not a bug. It is designed to be run with resource limits, usually set using softlimit. It is bad engineering to include needless checks in every single application, when the OS has this built in.BTW, if any sympatico users with debian boxes running ipmasq are worried about keeping Sympatico honest about their usage, apt-get install ipac.
Better yet, install MRTG. The mrtg-ip-acct program will read the IP accounting statistics directly from your iptables firewall. There is no need for the depracated ipchains compatibility module. Simply create a config file for it, such as /etc/mrtg.cfg:
WorkDir: /var/www/mrtg/
WriteExpires: Yes
Title[eth0]: Traffic Analysis for tourian
PageTop[eth0]: Traffic Analysis for tourian
Target[eth0]: `/usr/bin/mrtg-ip-acct eth0`
MaxBytes[eth0]: 12000000
You will need to have it run every five minutes, using cron, so add an entry to your crontab:
*/5 * * * * root /usr/bin/mrtg /etc/mrtg.cfg
Because the default page generated is named .html, for some reason, certain browsers (Internet Explorer) want to cache it no matter what you do. So the easiest thing to do is to wrap it in a PHP script, such as index.php:
My cable modem provider limits us to one gigabyte per day. After numerous arguments with my roommate, including how exactly to read and interpret these graphs, and because they do not cover a day exactly (we are supposedly counted from midnight to midnight), I wrote a simple PHP script to modify the MRTG output with nice, easily readable usage statistics:
http://david.maridia.com/mrtg/
The numbers at the top of the page are always live, since MRTG graphs are not. Note that the page has a latency of at least one second, because it takes two counter readings, one second apart, to generate the current usage rate. This is not always totally accurate, but should be close enough. Reloading the page a couple of times may give better results. The source to the script is available here:
http://david.maridia.com/mrtg/index.phpsI'm so frustrated by the culture of performance
I agree completely. The PHP developers seem very misguided. They refuse to fix many bugs in the name of "performance". Things like infinite recursion crashing the interpreter. At the very least, they could include a debug version, which would run slower, but help developers find errors. Things like the new private member variables silently erroring is NOT acceptable behavior for a scripting language, especially one that claims to be easy to use.
PHP already has enough bugs that make it hard to use, especially for those not experienced with it. It's really fun when a syntax error causes the parser to indicate that the error is hundreds of lines away from where it really is, or even in a separate file. Adding MORE bugs of this kind in the name of "performance" does not help to make PHP more professional. I really like PHP, and have been using in commercial environments for more than two and a half years, but these types of things are making me seriously consider switching to Python.
Yes, Everest needs to come to Lawrence and give Sunflower Broadband some competition!
They charge ~$50/month, dial-up ISPs charge ~$20/month for unlimited access. Since there are no other benefits to you other than faster speeds and always-on access, you're basically paying $30/month for bandwidth, for using a fat piece of their pipe (versus 5k/sec if you're dialup). So, how much bandwidth does $30/month buy?
I believe that you're forgetting what is probably the biggest benefit to broadband: decreased latency. Usually "faster speeds" means more bandwidth. For me, the 3-4k/sec that dialup provides would be quite usable for 95%+ of the time that I use the computer (which is a lot, considering that I telecommute every day). But the 200-300ms latency makes the connection unusable for me. Sure, web surfing is annoying when the connection is slow, but it's usable. Trying to work over SSH is almost impossible (and forget playing StarCraft).Yes, and there is a VERY good reason why it does that:
http://cr.yp.to/proto/verp.txtAre you sure that you want to be using Postfix? I don't...
http://cr.yp.to/maildisasters/postfix.htmlIf you're going to charge me between $40 and $60 a month for a capped service to begin with, I hope you're doing quality control.
Cable ISP's don't do the capping on the ISP end. The capping is done on the customer end, inside the modem. If you read the recent Slashdot article on cable modem uncapping, then you can see more about this.
Have you noticed that when hit your upload limit that the latency goes WAY up (say 30ms -> 1000ms)? This is due to the cable modem having an internal buffer and cap. You can fill up the buffer very quickly, but it takes it longer to empty. If the throttling was done on the ISP's side without buffering, then this wouldn't happen.Puh-lease. Like Comcast is paying $1K per T-1.
Umm, no. Bandwidth at OC-12+ rates is less than $400/month for 1.5mbps (T1) rates. And someone as large as Comcast is probably using significantly more, and thus getting a much better deal. Or they're a tier 1 provider and pay virtually nothing.
If ISP's were smart, they would peer with web hosting providers, and sell their outgoing bandwidth, since the majority of an ISP's usage is incoming.