Slashdot Mirror
Hardening Apache
Hardening Apache fills a huge gap in this sense, providing web administrators with a complete and yet concise book aimed to guide them from the very beginning of the installation process to the final steps of the server configuration. The author, Tony Mobily, is also the mind behind Professional Apache Security, a book published by Wrox Press which I reviewed on Slashdot about 17 months ago. Since Wrox's unfortunate closure, some of the material from that book has been moved into Hardening Apache. More specifically:
- The excellent chapter on "jailing" Apache is exactly the same;
- The chapter on XSS attacks has been slightly improved;
- The chapter on logging, which was nothing remarkable, has been greatly improved. It now includes a complete architecture to log on a remote host using encryption and a TCP/IP connection.
The first chapter of the book deals with deploying a clean and safe base installation, which will then be the grounds for adding extra functionality. Unfortunately, this task is often underestimated. What I liked in this chapter is the step-by-step guide to correctly downloading the source distribution and verifying its integrity (by checking its digital signature), as well as the clean approach to the creation of a lean, easily readable configuration file, which grants a painless maintenance. A highlight of this section is the use of Nikto to analyse and explain common weaknesses and to show how to fix them.
Chapter 2 presents some vulnerabilities and explains how to exploit them. The chapter doesn't have any "pearls of wisdom" (but it's nevertheless important to show that Apache can be vulnerable), and presents some important reference sites every web administrator should be aware of.
Chapter 3 definitely deserves a special mention: after introducing the "common" ways of logging and syslogd's architecture, the author describes a rational approach to realizing a complete logging solution which entails remote log servers, encryption of logs, and the use of a MySQL database to better organize them.
Chapter 4 is the only one which deals with the "programming" side of web security. It is not a comprehensive guide on how to write safe programs for the web, as it focuses on cross-site scripting attacks; it shows how to secure a simple and vulnerable message board written in PHP.
The following chapter talks about security modules: it presents an interesting overview of the most useful modules related to security, which will help administrators understand the importance of third-party modules and explains how to install and use some of them. I also liked Chapter 6, which deals with the installation of Apache in a secure, chrooted environment: the chapter does a great job in guiding the reader through the non-trivial steps required to get Apache, Perl and PHP working correctly in such a restricted environment.
The last chapter presents a number of powerful and well-written scripts which anybody can use to automate security and keep an eye on their web server (monitoring log growth, Apache's responsiveness, and so on).
What's to like Information throughout the book is very well focused and presented with a clean and friendly writing style. The book provides a clear and detailed walkthrough of the process of securing an Apache installation, covering both versions 1.3.x and 2.x and thus providing long lasting information. The book has lots of references and pointers to resources on the web, and - what's more important - instructions on how to read them. I also liked the "checkpoints" at the end of each chapter.
What's to consider Apart from chapter 4 on cross-site scripting attacks, the book does not cover secure web programming at all. It doesn't cover OS hardening either, which is out of scope but part of the game anyway. Going through the book requires some familiarity with Unix and Apache; otherwise you will have to resort to other books for the very basic steps.
All in all, I found this sort of "new edition" of the book by Apress to be greatly enhanced, more homogeneous and better focused than the previous book: I had been happy with Wrox's version, but I am enthusiastic about this one. This is a book which should definitely be included in any serious Apache administrator's bookshelf.
You can purchase Hardening Apache from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
In a way, Internet Information Services provides a more secure environment because an administrator gets a wealth of help and a decent initial configuration. In the end it's all about knowing your product, but it helps if the product helps you.
Try not. Do or do not, there is no try.
-- Dr. Spock, stardate 2822-3.
If security is not a concern, installing the Apache web server is a simple task even for an inexperienced system administrator Yet I still preffer IIS, according to research, it's more easy to install, and up to 70% more secure; according to research that is.
To Harden PHP while you're at it.
OpenBSD's Apache has a diff of 3 or 4 thousand lines over "stock" Apache. Why not just use that?
Comment removed based on user account deletion
You could also take the time to read about Hardening IIS. Come get me Mods =)
a cigar store wooden Indian? Sorry, I hadda say it...
To harden Apache, you just rub it the right way.
*I apologize*
How many of you so called admins do this:
%su
%httpd start
when we all know it should be
%su
#httpd start
- It's not the Macs I hate. It's Digg users. -
only a farking moron disable's ping and BREAKS many RFC standards.
idiots disable ping. It there for a reason, just because asshat's use it to find targets does not make it something you dont need.
Picked up Maximum Apache Security at Apache-Con last year and it has proven very useful. But any Apache administrator worth his salt knows most of this already. I don't see why you say it's fragmented and hard to find.
This is my sig. There are many like it but this one is mine.
but saying the least will always need less info.
And won't waste people's time (like mine)
who are going to react that your wasting people's time. Whoms reaction triggers more etc.
Ohhhh the horror.
the horror..
Hivemind harvest in progress..
Does anyone know whether StrongHold actually improves on Apache's security?
Jumpstart the tartan drive.
How many of you so called admins do this:
%su
#httpd start
when we ALL know it should be
%su -
#httpd start
I am a viral sig. Please help me spread.
The creators of Apache Server came up with the name due to it being based on a series of patches for httpd. "A Patchy Server." Get it? The name itself suggests its fragmented beginnings.
I wish web servers wouldn't advertise their version number be default (e.g. in directory listings or HTTP headers). It's like giving an attacker an exact list of the exploits that will work on your server.
You could always pick up something like PHP Triad, which installs Apache, PHP, mySQL, etc ... ... I use it, and it works rather well just as a server to try on a home network
php Triad
There is a fundamental flaw with security hardening being in a separate book, sold by advertising and word of mouth, read separately and in a different medium than installation documentation, updated asynchronously, and expected to work. Would you accept a word processor with a separate book on "Master's Secrets on Keeping Your Word Processor From Crashing"?
With any luck, many hardening techniques will migrate towards the Apache installation process, or at least the Apache documentation.
Don't let not knowing about security hold you back from hosting your own site. Experiment, learn, have fun. Put an apache box up on a DMZ, put stupid content on it, see what happens. Look at your logs, see what's going on, learn from any mistakes you make along the way.
If you're in this industry, and are afraid to be the "fall guy" who has do deal with the inevitable attacks, or the "fall guy" in general, you'd better fasten your seat belt...you're in for a bumpy ride.
He who makes no mistakes makes nothing at all
The older I get, the less I like everyone else.
you're apache install will stay safe because they'll be too distracted
You're all wrong:
%sudo apachectl start
apache drops root privs on startup.
i browse at -1 because they're funnier than you are.
Just use Apache 1.3 insted of 2.0 and it will stay up until you deside to take it down.
The least said, the better.
It's not offtopic, dumbass. It's orthogonal.
Simple. Dip it in tree resin and let it fossilize. It should harden into amber in . . . oh, a few million years or so. Don't be impatient, though, or all you'll get is copal.
For non-database logging, use mod_log_spread. This also solves the problem of merging logs if you happen to run a web farm.
Must-not-watch TV!
Setting up a webserver, or any other system should ~not~ be trivial. Administrators and Systems Managers have their positions because they are well-versed in the platforms on which they work upon.
Oh, wait. I just kind of re-read your troll, um, i mean post. I don't think I have to say any more. I'll get back to my 7th hour of the day keeping it up. Apache, that is.
The older I get, the less I like everyone else.
Very briefly:
Who is nominative
Whom is accusative
Whose is genative/possessive
Whoms (should probably be whom's, if it were a real word) does not exist
I know I'll get flamed for being a grammar nazi, but whoms is such a very, very bad neologism...
Boys from the City. Not yet caught by the Whirlwind of Progress. Feed soda pop to the thirsty pigs.
I have ping disabled on our system for machines outside of my subnet. It is something I don't need. It might be something someone else needs, but I don't care.
#ICMP traffic
einfo "Creating icmp chain"
$IPTABLES -N icmp_allowed
$IPTABLES -F icmp_allowed
$IPTABLES -A icmp_allowed -p icmp -s $NETWORK -j ACCEPT
$IPTABLES -A icmp_allowed -p icmp -s 127.0.0.1 -j ACCEPT
$IPTABLES -A icmp_allowed -m state --state ESTABLISHED,RELATED -p icmp --icmp-type echo-reply -j ACCEPT
$IPTABLES -A icmp_allowed -m state --state NEW -p icmp --icmp-type echo-reply -j ACCEPT
$IPTABLES -A icmp_allowed -m state --state NEW -p icmp --icmp-type time-exceeded -j ACCEPT
$IPTABLES -A icmp_allowed -m state --state NEW -p icmp --icmp-type destination-unreachable -j ACCEPT
$IPTABLES -A icmp_allowed -p icmp -j LOG --log-prefix "Bad ICMP traffic:"
$IPTABLES -A icmp_allowed -p icmp -j DROP
I do most of my work on a Domino server (say what you will, but its very secure and stable and I build customer apps really inexpensively) but Apache based servers (and they are myriad) keep intruding into my happy little world.
A year ago I wanted to put one public but found information on hardening it extremely limited -- or perhaps extremely disconnected.
If the book is indeed concise, it will be useful.
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
I designed the backend of www.babiesfirstchoice.com and we used apache 2. Its been hugely stable for us (the downtime we've seen was not due to apache problems) and lets us do everything we need to do on it. An IIS box would of cost thousands more due to licensing and the extra hardware needed to push M$ solutions (BFC currently runs on a athlon 1700xp with 512 megs of ddr and a basic ide hard drive, nothing fancy).
Lawyers, MBA's, RIAA? A jedi fears not these things!
I guess I can see how you would come to this.
/etc/conf.d I've been using the gentoo build ever since then.
I started using solely linux around 4 years ago. One of the first things I did was install apache and play around with that. I got it working with a little effort. After that I always built it from scratch. I eventually decided to try using apache out of the portage tree and it was set up as soon as I ran emerge and setup the configs in
Anyways I had to develop some webapp and my boss wanted me to do it in php on IIS. We had a hell of a time getting all of the authentication and settings, we couldn't quite get it all working. We had trouble with ssl and a few other things. so my co-worker and I dropped apache on it and got it working with SSL in about 10 minutes.
I guess the short version of the story is: Hire MS sys admins for MS systems and here Linux sys admins for Linux systems. Rather than saying one is easier than the other, it might be better to say that one comes more naturally to a given user. And since I'm at a school with a bunch of nerds and almost all of the people I know prefer to run linux, there's a good chance that we can probably get something like apache setup more easily than IIS.
i could not think of anything clever.
Or at least a good chunk of it is. Some of the patch is necessary just to take into account how OpenBSD handles various calls.
There seemed to be little in the way of practical material that gave specific and step-by-step instructions for installing and running Apache on Linux.
Maybe you missed the "documentation" section at the apache.org website? Or, do a google search for "linux apache howto". Tons of good stuff out there.
Apache on Linux requires you to spend 8 hours per day just to keep it up and running,
On what planet is this true? There's about 4 things to change from one webserver to another; you build one config file for your environment, and for the next one modify the listen, the user if you want, the document_root, and maybe servlet mapping if you're using that. Trivial and one-time.
and while its performance and security is fine if you have the time and staff for it, there is no way to just set it up and let it sit while installing patches when needed.
Our experience differs profoundly. Perhaps someone like you needs to hire someone like me to help you get set up. It's a trivial setup, configuration is well documented, and once it's up and running a webserver doesn't need any attention whatsoever until the next version comes out or you decide you want to change what it does. Arguing against Apache on actual factual grounds would be one thing, but "it's hard to set up and lots of work to keep running" is demonstrably false.
In short, find a hosting company with a lot of Windows servers nearby IP-wise. This is, sadly, not a troll or flamebait, it's my experience. Apache is not what you have to worry about, it's scripts you run. Every single non-targeted exploit of a Unix machine I've seen (that is, compromises that don't target a particular site or person) has been the result of a buggy script. And usually a script that's installed on a ton of systems, so the hackers can compromise many machines with the same attack. Most of them are looking for warez dump sites or launching points for DDoSes. It's not worth their time to target individuals.
Apache is extremely complicated and therefore a potentail rats nest of potential security holes. Please don't get me wrong here: I like Apache very much for its great flexibility, yet I helped a lot of security aware companies to migrate to thttpd because they wanted a code base that could be scrutinized in a reasonable amount of time. Most web apps really don't need the flexibility of Apache anyway, and those who do, will have to be run in secure environments like, say, jails or other virtualized environments.
cpghost at Cordula's Web.
%su
/user ALL=(ALL) NOPASSWD: ALL in /etc/sudoers
#httpd start
This implies:
%which su
alias su='sudo su'
and
I'm wondering where you got that '8 hours per day' maintenance idea. Seriously, do you adjust your configuration for 8 hours a day, 5 days a week, year round, or what? What exactly requires your interaction with the system, once it's been set up? Occasional patching does not take 8 hours per day, every day. Ohh, and you do know that on unix you can remove a binary of a program that's running, put a new one in, and do a quick kill/start? Almost no downtime in that case, and there are tons of other ways to do it without any downtime whatsoever. Overall, I say troll.
--- d'oh
If Apache and PHP can't fulfill those operations securely out of the box, then there is something wrong with either the design or implementation of Apache and PHP, not the experience level of the user.
In different words, the default should be secure, and users should have to go through extra steps to make it not secure.
The directive is [global] ServerTokens Min
It's all in the docs.
Ohh, and you do know that on unix you can remove a binary of a program that's running, put a new one in, and do a quick kill/start?
Actually, for most of the configuration changes (short of an actual version upgrade or SSL cert change), you can do an 'apachectl graceful' and it applies your changes to the _new_ sessions, while letting the existing sessions close in the natural flow of the users' use of your site. Nice for minor tweaks on the fly during the day, with zero downtime.
Heh. I plan to serve my own site to gain the experience of log-reading, config-file twitching, and all that; I want to break into the Web field, and see this as a good opportunity to learn about webserver/OS hardening by fire, as well as being just plain fun :)
I'll also ask a buddy of mine who's a security consultant to attack the server. Better to have a friend attack and tell me my mistakes than some pimply-faced kid 0wneR my machine.
I didn't think the house band in Hell would play this badly.
I wrote the book, so my input is biased...
I think the book would be great for you. If you do buy it, let me know what you think of it!
Merc.
(The book's author)
ohiolinux.org is having a speaker about hardening apache & apache2. and it looks as though its going to be a good feature packed lineup of oss speakers. hardening apache isn't exactly an easy task.. just don't think that you're going to be totally "secure." when you drop your defenses, you're at your weakest point. apache is a great server. and they have dominant market share. who says ms will win? 1 down, 100000 to go :)
Now, you still have to learn how to set up that package right, and keep up on updates, but diversity is a nice thing. Even if you just move some files (e.g. static content like images) off onto a side server, there's less to secure on the Apache box, so it's usually simpler.
PHEM - party like it's 1997-2003!
That whole thread on "no more apache updates" had nothing to do with security. It had everything to do with the OpenBSD team deciding not to keep using Apache HTTPD because of the new ASL 2.0 license. Personally I think the OpenBSD team is completely wrong on this issue and their attitude is incredibly offensive. Moreover, if you read the whole thread you'd find this response that the apache group has been responsive to the patches but that many of them are BSD specific and that's why they were not put into the main source tree.
Who said Freedom was Fair?
WTF are you smoking, I certainly want to stay away from it. I've had my computer run apache for the webserver for years without having to change anything in the configurations (Mandrake default builds).
Good programmers drink beer to relieve job stress.
Great programmers drink hard liquor and work best hungover.
There is one thing that makes apache very unsecure. Suppose that you have few users each having it's own virtual host. Apache is running from exactly the same UID/GID for each virtual host! There is no way in pure apache to prevent user A from looking into user B vhost files (containing for example php scripts, password to sql databases etc).
:/
You would need to run multiple apaches running from different UIDs for each user
That's bad. suexec it's not a option - it doesn't work with mod_python and other apache modules.
perchild MPM module that should do it doesn't work and apache developers are not interested in fixing that. No idea why.
metuxmpm MPM module was written instead perchild by external developers but it also doesn't work well unfortunately (for me it doesn't work at all).
following the only proven rule regarding security, which is:
secure == obscure
Which is why my webserver is only accessable through the octal represntation of its ip.
=================
Unix is very user friendly, it's just picky about who its friends are.
I would even accept an OS on those terms. There is a huge market for books telling users stuff that is not in the manuals. O'Reilly's whole business model is augmenting system documentation as was SSC's (Linux Journal Publisher). That having been said, I don't disagree with your basic premise.
Definitely, apachectl graceful is very useful. I was only talking about apache upgrades. By the way, if you add more ssl vhosts, you also have to do restart rather than graceful.
--- d'oh
http://www.bastille-linux.org/jay/Talks/slides-def con-securing-apache.pdf
This helped me with a few extra ideas.
Sigh, there are several ways to approach setting up an Apache server. All of them are easy.
First one is to start with an empty configuration file and then cut and past in portions of the standard file until you get a minimally working server.
The good part about this approach is that you get the least amount of bells and whistles added. Security via a small footprint is a good thing. The bad part about this approach is that you end up with a minimal server that may need more tweaking to get everything working as you need it.
The second approach is to take the original configuration file and start chopping things out of it. Test each deletion to make sure that everything you need still works. Use something as simple as RCS to keep track of your changes.
The good part about this approach is that you'll have a server until you break it. You will also have a nice record of every configuration change you've made. The bad part about this approach is that you may end up with a fatter server than you need. This violates a security maxim of making the least footprint on the net necessary to accomplish the task.
The third way to configure Apache is from scratch. This is somewhat more complex than the other two, and can lead to unmaintainable configuration files.
The bonuses for creating your own configuration file include understanding what goes on in the Apache configuration, and making a nice, modular configuration file. The bad part about this is that if you don't comment your file, you'll get an unmaintainable mess. Unfortunately some consultants think this is a good thing.
As for chrooting Apache, it took me less than 15 seconds via Google to find a step by step procedure http://www.faqs.org/docs/securing/chap29sec254.htm l to chroot Apache on a Redhat Linux.
Gee - you get this massive file to edit to get Apache up and running which is fine for some of geeky land (like the slash-dot crowd) but it creates a hurdle for your basic person who wants to have a running web site.
So why not create a smart program to generate the config file either as a script or as a GUI application that will have enough smarts to get your normal site up and running but with the possible security holes turned OFF and massive warnings written into the file so that when someone goes to edit it they have some clue provided.
Same for PHP.
Oh sure - it's soooo kewl to just edit the text file - and for the clued-in that's great. But something which can make a new safe file for you, AND check an existing file - seems like that's just smart.
Otherwise it's safety through obscurity - or insecurity through obscurity and neither of those make sense.
"...You can purchase Hardening Apache from bn.com." Or, you can purchase it for $10 less from amazon.com
YHBT. YHL. HAND.
Yeah, right.
Any RFC standard that breaks because it looks just like the computer on the other end of the line is turned off... needs breaking any way.
If you have a specific need to know I am here... it should be a part of your protocol.
Because, other than those things I want to work... my machine does not exist.
In short: I don't need ping replies to work. So, my machine does not do it.
--Phillip
Can you say BIRTH TAX
I actually like ping, I leave it on since my cablemodem isn't all that reliable and it's a nice easy way to check if it's up when I'm at work. If port 23456 stops responding, I can ping to see if it's just the application or the connection.
I also use it as a monitor from another location so I can redirect certian traffic if it stops responding.
Sure, I could go firewall nazi and only allow ping from certain locations, but hell, it's only ping.
Not to say you MUST use it, but I find nothing particularly evil about it that it MUST be blocked.
- It's not the Macs I hate. It's Digg users. -
Ctrl-Alt-Del
What, me Rush fan?
Whaddayamean "who cares"?
Perhaps I'm mistaken here, but the apache daemon itself still maintains root priveleges, it's the children it spawns which do not. If someone managed to stick their own module and load it in the config, I believe it still runs with root priveleges on startup.
Roflmao I did that once -> used apache as a reverse proxy. I love the simplicity of it, and the increased security. You can block a tonne of attacks using mod_security, and keep them from hitting your real web server.
Thing is, I forgot to lock down the allowable hosts.
Soon, I was seeing scripting engines requesting ads from various web servers in my logs. Took over a year for those scripts to stop.
Didn't take long for me to find it, but once you're on an open proxy list.... You're screwed.
09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
You know... a recent study at MIT (I think) together with Simon and Garfunkel is showing that there's no difference between hardened and unhardened Apache.
This might be slightly off topic, but I have a question for all you Apache boys. I get this weird error from my Apache/2.0.48 (Win32) Server: it installs fine, I set it up properly, I set the passwords and the users, I test it, everything works fine, people can access and download stuff, then a few months later they start getting broken downloads. Meaning that, if they try to snag a 4.8 MB mp3 for instance, they end up being able to download only about 300 KB (the beginning of the file/song) after which it abruptly ends. Although the file clearly says 4.8 MB on the web page, Apache only lets you download some 3-400 KB. I have no idea what causes it (registry rot doesn't seem to be it) and the only way to fix it is to reinstall Apache - ten minutes later all is good again.
Any ideas, anyone?
The power of accurate observation is commonly called cynicism by those who have not got it. -- G.B. Shaw
Similar to the anecdotes that I like to tell. I've been on a number of projects where the official decree was to use web server Brand X (where X is any of the Usual Suspects). After some weeks of futzing around trying to get it working sanely, I'd decide that I'd had enough of it. So I'd set aside a half hour to grab the latest apache, compile it, config it, fire it up, and link all our stufff into its htdocs directory. This never took the full half hour, and invariably it worked like a charm, never crashed, never had breakins, etc.
Part of the fun was always telling them every few weeks "Say, y'know, we're supposed to be using server X rather than apache. Maybe we should get working on it." And invariably, everyone else would say "Yeah, but we have a working server now; we have a deadline; we have a lot of stuff to finish off; let's put it off a while longer."
Some of those were years ago and they're still using apache. The idea is starting to get through the thick management skulls that maybe there's something significant here.
Of course, if they have any active content at all (especially SSL), I also like to warn them that apache's apparent security is an illusion. Maybe apache itself is solid, but it has no control over those modules that came from someone else. Any script at all could have a gaping vulnerability, and apache has no defense against that. Every piece of code they add to the apache core is a potential security hole.
Sure wish I could get them appreciate that. And keep up with the apache patches.
(And I've long known the etymology of the name "apache". Bad, bad pun. Well, as they say, the only good pun is a bad pun.)
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
There are others out there as well which do the same thing. I've never tried that one, but I'll dl it in a while. There is a list here and also here of others which aid in installing Apache, PHP, mySQL, Perl, etc (some also install tools such as phpMyAdmin, a bulletin board service and others go as far to include a mail service and FTP servers). I don't believe that the DeveloperSide.NET suite is listed there, but it looks like a good application as well.
> nslookup autoupdate.microsoft.com
Non-authoritative answer:
Name: a1076.g.akamai.net
Addresses: 199.44.34.70, 199.44.34.68
Aliases: autoupdate.microsoft.com, codecs.microsoft.com
codecs.windowsmedia.com.akadns.net, codecs.windowsmedia.com.edgesuite.net
MS uses Akamai for distribution, Akamai runs Linux. Side note, thank $_DEITY that MS does not use Connexion any more. Holy dog crap they were SLOW!
Of course it does... it couldn't bind to port 80 if it didn't start up as root. You are correct that it is the children that run with decreased privileges.
If somebody has gotten enough access to your box to add an arbitrary module to your httpd config, you're in pretty big trouble anyhow...
appending the - to the su command will create the environment as if the user had logged in directly from a login prompt.
while true ; do echo this is my sig; done
Interestingly tho, openssh has had more security advisories than the original ssh
http://spamdecoy.net - free throwaway anonymous email - avoid spam!
Sure, make it the default option, but I think something like RedHat's firewall options at installation (none, medium, high, setting iptables config) is even better. Apache gets run on intranets as well, and in some cases, full security is exactly the wrong thing.
This is easy enough to do; you ship several versions of the config file, and copy the one corresponding to the requested security level to the default filename. Include docs on using the others.
How many of you so called admins do this:
/
%su
#httpd start
when we all know it should be
%su
#cd
#rm rf *
It could be better (chroot), but the conf files are clean and easy to work with. Also, it's nice to be able to get up to date by simply typing 'emerge sync && emerge apache'.
Having recently left a fairly major ISP in my country, I feel obligated to correct both the Microsoft bashers and the Microsoft sympathizers on some of the key points.
The thing is this:
Microsoft Servers are Win2k3/IIS (for the most part. There may be some left on Win2k).
In part it has to do with bandwidth distribution, particularly when dealing with international downloads.
What happens when a million users (conservative estimate) go to download a 170MB file, such as Service Pack 2 (redist) all at once? Rather than having their server farm saturated with a million users, they use Akamai caching servers to distribute the files. Lets say they have an OC48 - 2.4GBPS which IIRC costs like $100k/month for a half decent one (conservative estimate for that kind of server farm, but you get my point). At a million users, you would be averaging about 2.4kbps. Not good. 56k users wouldn't notice, but us broadband users definately would.
So, what happens is, major ISPs whom happen to have partnerships with Akamai to distribute content, have these caching servers in their data centers all around the world.
What happens is effectively this: a client goes 'Hey! windowsupdate. microsoft.com! I need microsofthugeservicepack.exe', and since MS has a partnership with Akamai, the server passes it on like 'Wait a second... I've got a file here, but since your hostname is 202.202.202.202. newzealandisp.co.nz, you would be way better off getting it from nz01.windowsupdate.cache.akamai.net, cause that will save a shitload of international bandwidth'
So, in fact, it's not a "Microsoft Servers can't handle the load" scenario, it's more like "Using Akamai saves us a buttload of money and has a far better distribution network in place, so why don't we pay them like $100,000 a month, rather than building and maintaining one ourselves, which, despite being very cool, would be pretty much pointless and not as cost effective"...
Think -
1: Use Akamai bandwidth (like a transparent mirror, whose little yellow boxes just happen to run Linux or sometimes Solaris or BSD) for like $100,000 a month (guess), reach like 150 countries and 1-5 major ISPs in each country depending on it's size OR
2: Use your own for like $100,000 (guess) a month, have it go really slow at peak times, or even
3: Build own distribution network, for like $1,000,000 up front (say, 1000 servers @ $1,000 each, plus bandwidth costs from US, plus bandwidth costs for each country that has a mirror which could probably easily be $1,000 a month (conservative) in each country meaning ~$150,000 a month over all)
I mean, come on. They "duck and cover" behind Akamai for the very reason that when you, their user, is trying to download something off of their website/ftp server/auto update server, you want it to come through as quickly as possible. Microsoft, being the multi-billion-dollar corp that it is, is expected to deliver. You should be able to download that Service pack at 150KBps, so they utilize the aggregate bandwidth available from (in this case akamai) to deliver.
Just like when you go to major websites such as CNN, AOL, Google (et al ad infinitum), they too all use Akamai. The OS that the host machine itself runs is thus irrelavent. Netcraft basically does a fingerprint, and if their finger gets relegated off to an Akamai box, it could be because its a high traffic period, or, the Windows Server simply delegates finger probes (I know that sounds bad, but you know what I mean) to the Akamai box to perhaps give some sort of false impression.
You Linux Zealots should know better! I'll use Redhat as an example, here (though most other distros distribute their ISO images in a similar manner)
I know if I try to, for example, download the ISO or even any updates, off of say, the Ukranian Mirror for RH, it's not going to go as fast as if I downloaded off one of the New Zealand or Australian Mirrors. The only real difference here (in terms of who is using who) is that: Mirrors for L
Founder & COO, Hayai India (hayai.in) / USA (hayaibroadband.com)