Slashdot Mirror
Apache vs IIS in Performance?
Greg Merideth asks: "In the midst of my proposal to universally sweep all of our company web servers together I was handed an advertisement from DELL by our CIO with a big circle around two machines that DELL sells with an interesting note. They are identical machines, literally identical machines but the claim is that the Linux 6.2 Apache based machine only supports 20k-100k hits per day while the Windows 2000 IIS based machine (remember, same config) supports 500k-1M hits per day. Now if DELL is claiming that NT, with the same config will out-perform Apache in serving web traffic, how am I supposed to convince my company that Apache and open source is a great way to go? They don't care about open source or Linux, just the performance that they will get from the machines. Where can I get -credible- data to prove that Apache can outperform IIS?"
I'm using Infortrend Scsi to Scsi RAID modules on Dell PowerEdge 2XXX series machines. Running BSDI 4.1 I serve ~50k page views a day, HIGHLY dynamic, using mod_layout and lots of SSI. My load is usually .08 with an average idle of > 95%. Yes I run porn. It is worth noting that almost no unix comes optimized for what you want. Anyway, my machines do not suffer from "20 second" lock ups. Perhaps you've got a scsi timeout issue going on there. If you're using a lot of cache between your host adapter and disks, try removing some, big caches that don't clear quick enough cause scsi timeouts.
2) Stock systems are not what are used in the real world (i.e. webhosting companies) and therefore the results of a benchmark pitting stock system vs. stock system are completely useless. Web hosting companies DO hire talented system-administrators for NT or Linux.
3) your third comment is a complete troll. You should be moderated down.
Werd.
i have found similar results. the scheduler seems to have gotten a nice kick, as well. under heavy concurrency load, i saw performance increases in the order of 800-1200% (!) on identical hardware.
/me thinks apache + tux on that dell hw will perform nicely...
ydnar
The Cure of the ills of Democracy is more Democracy.
Erlang Developer and podcaster
I was testing on a Compaq Proliant Dual Xeon 800 1 gig of Ram, 8 Gigabit ethernet cards running traffic to 8 separate networks to 16 clients.
With 2.2.X the driver I was testing could only muster a total thruput of around 370 MBPS.
With 2.4.0-test8 ( ia32 )
My Cisco switch was the bottleneck.
Bitcoin pyramid: Join here: http://www.bitcoinpyramid.com/r/1427 it's FREE!
I don't want to accuse Dell of malfeasance, but if they really benchmarked these numbers, they must have badly misconfigured Apache. My guess is that they had Apache doing reverse DNS lookups on every query, and the queries all had to time out because there was no name server responding to them. That's the only way I can imagine that you could get such bad numbers.
> 'Three years without a remote hole in the default install!'
DOS attacks are not root hacks. OpenBSD has never claimed to be immune to DOS attacks. And unless you can control the connection end-to-end (or at least a hop away from your box), you can't ever be.
I've finally had it: until slashdot gets article moderation, I am not coming back.
You could also sell them on security and reliability, areas in which IIS simply falls on its face. A look through old Slashdot stories will give you ammunition. --Tom
Tom Geller
I have two links for you, from an os-agnostic source, the german computer magazine c't:
The first is in only german and show iis and apache are at the same speed (with 2.4pre-kernel IIRC).
The second article is really interesting (and in english). They measured downtime of business class webservers and compared nt to solaris and linux.
To quote the beginning of the article:
To Be Up or Not To Be Up
Analysis of Web Server Downtimes
Stability is one of the major criteria for web server performance. Although it is commonly accepted that Windows NT and IIS cannot match Unix and Apache servers in this field, there are hardly any tests to confirm this assumption. An availability test of the major German internet businesses clarifies the situation.
If you are looking for static page serving, then apache is not the speediest, it's fast, but not as fast as the competition. This is even the cliam of the apache group
For Dynamic serving, the story is a bit different, with apache puling up on par or ahead with the competiiton ( IIS, Netscape E, Zeus ).
if you want the absolute performance king, then the no question winner is zeus, it has been for a while the fastets serving static content, and a great contender on dynamic. The price is not a consideration, since it's not only well worth it, but when you are buying high end servers, a very little add on
Couple zeus with FreeBSD or solaris, and you got yourself a mean combination
.. and I don't know how Apache compares to NT, but since you're obviously wanting to use Linux, why not monitor the situation with Tux and khttpd. khttpd is the kernel webserver, and provides a webserver in kernel space, while Tux is a mix between kernel space and user space.
Khttpd is exceptionally fast when delivering static content, while it gives all requests for dynamic content to Apache. And apache being quite fast at dynamic content this works out well.
Tux however handles both dynamic and static content. It is also exceptionally fast at both. Take a look at this slashdot story.
Ah yes! We're back on this subject again!
Let's get started on the same thing for the millionth time.
FIRST! Dell server hardware can be summed up by one word: wierd. Thier PERC Raid controller blows goat nuts.
Second. Most likley this was tested on a great big honking 8 way Xeon with globs of ram. Yes, the 2.2 kernel has "good" SMP support, and yes 2.4 kernel has better support, but Apache 1.3 DOESN'T!!! It simple doesn't have muilt treaded support like ISS, or what ever. Apache 1.3 runs best on many single proccessor machines, clustered some how, sharing content through NFS. If you want to do the Apache vs, ISS on muilt processor machine bit, test with Apache 2.0. I think it was showed once, and it scored 4 times better then ISS.
Sure, one HUGE machine with no fail over is what you might want for your intranet server where only people will complain. However in a co-lo, running a website that runs your company, many small cheap machines, that can quickly be replaced, upgraded, and scale up is what you want.
Think about like this, you can spend 4k on a small one process machine with a ide drive, or 50k on this huge central heater with globs and globs of ram and a huge SCSI raid system. You'll need two of these space heater, that 100k. For 100k how many of these 4k cheap-o machine can you buy? About 25 machines. For about 4k, you can get a single processor 600 mhz and a 20g ide drive, about 1/4 a gig of ram, a nic, and a rack mount able case. If you only get 20 single processor machine, you'll be be able to flood your connection, and have money left over for a 250g RAID 10 system. My number may be off (WAY OFF), but my point is made.
If a 4k machine blows up, so what, you have 19 other machines. If five blow up, oh well, you have 15 left. If this 50k 8 way space heater blows up, you lose haft your site, and you'll be running at haft power for the next few days until the unsite monkey tech gets there.
MarNuke
If your website is crapping-out on you, the problem is likely to have something to do with your website. I.E. Your application was poorly designed, and the author didn't truly understand the platform he was designing for.
I'm not saying that IIS and MS SQL Server are perfect; They're not. But neither are MySQL and Apache, either.
-----------
"You can't shake the Devil's hand and say you're only kidding."
That's assuming that the hit rate is uniform across the whole day. That's an unlikely state of affairs, even for a site that is of worldwide interest and is getting hits from every time zone. There are always slower hours of the day, and there are always spikes in the request rate.
But you're point is well taken if we narrow the time frame. If we assume that all the visits come in an 8 hour time window, then the best performance (at 100k hits/day) is a mean of about 3.5 hits per second. If they all come within 12 hours, then the best mean performance is 2.3 hits per second.
Now, I've gotten Apache servers to serve up hundreeds of pages per second on fast Pentium machines! If you're averaging less then 10 hits per second over a whole day, then you have severely misconfigured your server.
I submit that the Dell figures are completely worthless without more details about the benchmark. Were they serving static pages, CGIs, servlets, PHP scripts, mod_perl scripts, FastCGIs, what? Were backend applications and/or DB queries involved? How did they tune the server? Did they tune it at all? Did they use a caching proxy like Squid? Without this information, the benchmark results are little better than random numbers.
Always keep a sapphire in your mind
Datacenter is only being sold on certified hardware you nitwit, and please explain to me why anyone would want a 8-32 CPU web server?
ostiguy, mcse
Ok, so I'm a few hours late to the party, but maybe one or two people will see this post. First, a preface: I use Linux, I don't particularly like Microsoft products. With that said...
...how am I supposed to convince my company that Apache and open source is a great way to go?
Well, you probably need to show them that Linux will make the company more profitable. As you said, the suits don't care about open source. And they really shouldn't. Personal agendas (e.g. "Microsoft is bad") shouldn't get in the way of what the company is there for: to make money. The same goes for you (the original poster). Just because you like OSS (and you should) doesn't mean that the solution for your company is to use it. There are far too many variables to consider.
Where can I get -credible- data to prove that Apache can outperform IIS?
That's really not a good way to go about it. If you're comparing an Accord and a Camry, you don't want to go looking for information that will prove that the Accord is better. You have to look at it much more objectively. I remember back in '92 when I was looking for a mountain bike. I decided what I wanted before I even started researching, and bought a bike that really wasn't the best for my money. I think the analogy fits this situation somewhat.
Besides all that, as I'm sure a million other people have mentioned, benchmarks aren't very valuable. You've probably heard a quote attributed to Mark Twain: "There are lies, damn lies, and statistics." Look hard enough and you can dig up factual data to back up most anything. But it's rarely the full story.
-- "Complacency is a far more dangerous attitude than outrage." -Naomi Littlebear
I have found that IIS is faster than Apache. However, IIS runs, for the most part, as a single multi-threaded process (inetinfo.exe) running in kernel space. Therefore when something goes wrong -- The entire process goes down, which kills everything. Ever try stopping the web publishing service on an NT box that is having a problem? It usually comes back with a message saying can't respond to control function, and you have to reboot. With Apache everything is running in user space as multiple processes - Speed Hit Here? - However, it is much more stable and reliable because if one process fails, it can be corrected without taking the entire server down. IMHO - I would take a speed hit as a trade off for uptime and reliability anyday.
The posted question peeves me a bit, since it asks: "Where can I get -credible- data to prove that Apache can outperform IIS?" rather than: "Where can I find independant validation of these results and/or data to refute them?". If you're really concerned about finding the best solution to your problem (web serving) you should find facts and then choose a solution, not pick a solution and then find facts to back it up. That's not fair to your employer and it's not the way shops should be run.
-User
Emacs is for experts. Pico is for beginners. VI is a disease.
Don't forget, IIS runs in kernal space, apache in user space. That gives IIS a boost, but also causes it to take the OS when it dies.
Vintage computer games and RPG books available. Email me if you're interested.
Say you run a superbowl ad with your URL. Say 20,000,000 decide to check out your site after the game. Those hits arn't going to be streched out over the 24-hour period that comprises super bowl sunday -- you'll get hammered HARD at half-time and after the game. If you are running an online brokerage, 99% of your hits are going to come during market hours, with probably 1/2 or more of those hits coming in during the first hour after the market opens. You've got to be able to handle PEAK loads, not just the sustained average loads.
Why is it that the proponents of "one nation under God" are so eager to get rid of "liberty and justice for all"?
I think the thing to keep in mind is that any given test is most likely not going illustrate useage on YOUR server.
The only way to do tht is with tests performed in your environment, with your data, your pages, and your customers.
Tell your boss: "Would you buy a car without test driving it and the competition? Why buy a web server without doing the same?"
If you need web hosting, you could do worse than here
Step 2: Show your boss OpenBSD's listing at Bugtraq (read: "...This was fixed 5 months ago..."
Step 3: Use OpenBSD for security (with lowers costs)
Burn Hollywood Burn
If I were you, I'd get Dell (or the reseller/vendor, whoever) to drop off a couple of identical machines, configure one with RH6.2 and one with NT, and let you guys test them. Do not be afraid to tweak the Apache server based on your experience and knowledge. That will show management where their skillsets lie.
Run one for a week or two, move the content, and run the other for a couple of weeks. Then bring up the MS licensing costs, the Apache statistics on web presence, etc, along with an "Oh by the way I'm a Unix guy" and see what they say.
If Dell refuses to supply the test machines, make sure and bring that up to your management- Explain that basically the vendor is unwilling to justify their claim. Then you might be able to pick another vendor such as IBM or Compaq. I understand that Compaq has a fairly liberal test/loan program for such things.
Don't sweat the petty things. But do pet the sweaty things.
This page on Dell's site might also be of interest.
Bingo. Dell itself reports July 25, 2000 SPECweb99 results. Compare items 2 and 7 to find a common platform score comparison.
(rank, vendor, system, score, #CPU's, OS)
2* DELL PowerEdge 6400/700 4200 4 Red Hat Linux 6.2
7* DELL PowerEdge 6400/700 1598 4 Win 2000 Adv Server
Go to the SPEC web site. Then search for the SPECweb99 results Dell submitted.
Apparently, they are using Redhat's Tux server, not Apache. I don't know whether they are related but the combination kicks IIS's ass. You can't get much more definitive than the manufacturer's own tests using the recognised industry benchmark.
This page on Dell's site might also be of interest.
Yeahim sure this is informative, its really good to know, I did not know this, but this seems like a typical response from a Linux Enthusiast.
:) But my first thing to do is to take Linux off and throw FreeBSD on there :)
"Oh oh we dont have this feature yet but it WILL be in next version....."
It just seems like well I dont know, and I know Linux is progressing much faster than it has ever before, but I could go do the same thing with FreeBSD and im sure I can max out a switch.....
The main thing I want to say is just that it doesnt seem fair or rather helpful to this guy that a kernel that isnt even production level does what he needs it to.
I suppose you got modded up because you were showing facts about Linux, but Apache runs on many platforms not just Linux
So I dont really know what the point of the post was, other than the fact that a Dell comes with Linux 6.2 => or is that RedHat 6.2?
Jeremy
It's true, IIS has run in ring 0 since 4.0. I think they did this to get a leg-up on apache/zeus/netscape/etc...
:-P
It provides a performance boost, but putting something complex like a webserver (or browser for that matter) into kernelspace is just asking for it.
Having it take the OS down with it on a crash is the best thing you can hope for... what if it doesn't take the kernel down on a crash, but decides to trample all over memory and data instead? It could misbehave and mangle SQL queries from there on in and it could do signifigant damage before it's noticed. I hope that database wasn't too important...
---
Where can the word be found, where can the word resound? Not here, there is not enough silence.
"Where shall the word be found, where will the word resound? Not here, there is not enough silence." -T.S. Eliot
c't (german magazine) compared apache vs. IIS in a quite exhaustive test and made some very interesting points. Though not Win2k was involved the article may give you some hints and it is even available in plain english here
I also found this bit interesting, if you look at the machine that is configured at the top of the ad it has 256 MB and is quoted at 20K-100K hits, and directly below that is the exact same machine with only 64 MB and surprisingly it is quoted at 20K-100K hits. So apprently quadrupling the amount of available hard memory does nothing for the performance of a machine....uh....er something.
Funny and I thought Perl == Paid employment recently located
I test nic cards and no mater what I use gigabit or fast ethernet. I can't get a total thruput of more than 370 MBPS.
I tried similar test with kernel 2.4.0-test8 and nearly got fired for shouting.
I was able to max out my switch and there is no upper limit in sight with the Cisco hardware I curently have available to me.
The 2.2.X TCP stack is NOT multi-threaded.
The same bechmark on the 2.4.X kernel will take your breath away.
Bitcoin pyramid: Join here: http://www.bitcoinpyramid.com/r/1427 it's FREE!
There have been a lot of good posts on this subject, but all that I've read missed the one obvious problem with Dell's claim (or the post): ;)
100k hits per day is just over ONE PER SECOND. A 486 could do better than that.
Even if you assume that peak time on the site is a 6 hour period, you're getting close to four and a half hits a second, which is no big deal.
For some good information on the state and progression of Linux, look at http://www.kegel.com/mindcraft_redux.html
I know, love Open Source, hate MS. But are you open to the possibility that Apache may not perform as well?
This is not a troll, but a serious point for all--make certain that your loves, hatreds, and desires don't over-rule what may be a valid goal.
If the goal for your project (as specified by the powers-that-be) is performance over ethical ideals, then be certain you know what you're really looking for before you go looking.
But then you know that since you are a techie.
:-(
One of the hardest jobs a techie has to do is convince a clueless boss there is more to a job than a rigged benchmark on a 4-color marketing sheet.
Here are some points to bring up:
Since there are more Apache installations than IIS installations, there are more engineers on the market who understand the technology. You can even throw in a few MCSE horror stories for good measure.
IIS will tie you into an NT platform, and in 5 months (when M$ gets the appellate court to delay their case for a few years) the licensing fees are going to shoot way up. Every major consulting firm has given exactly the same prediction to their largest clients, when M$ wins or delays its antitrust case, licensing fees will increase 2x to 10x, and to reserve a major portion of budget for it. If you choose Apache, later you can switch reasonably painlessly to Solaris, linux, or any other system as costs or management changes dictate.
If you are going to serve only static pages, then IIS wins slightly. If you start to generate dynamic content, Apache blows past IIS. Go search the web for some of the other comparisons.
Apache installations are far more stable than IIS, and there is a lot of anecdotal evidence on the web to help you back this up.
When it comes time to add a custom feature to your web servers, an OSS solution like Apache is likely to have it covered, but with IIS you are at the mercy of M$. If a new feature doesn't exist for IIS, you don't stand a chance of convincing M$ to add it. They have a long history of doing only what they want, not what their customers are demanding.
Get creative, or you will be stuck with IIS, and it will be time to find a new job
the AC
Hemos is like...sci-fi fans;he thinks technology is cool, but he hasn't bothered to understand the science it's based on
http://www.flyingbuttmonkeys.com was a 250MHz AMD k6-2 with 32MB of ram and a 4GB IDE hard drive during the past month when it's been repeatedly slashdotted. Twice, it served over 120000 hits an hour, at an average packet rate of more than 1000/sec, without a problem. Lately it got unreliable because of a failing network card, but that's not exactly Apache's fault. 100k hits a day, right. If the slashdotting had been in effect for the whole day, my little Linuxbox would have taken and served 2.88 million hits that day. the load stayed around 0.20. Imaging what a machine with some actual memory and CPU could do!
________________________________________
Napster-to-go says "Fill and refill your compatible MP3 player", which is a lie. It's not MP3. It's WMA with DRM.
I'm SURE Dell has the documentation to confirm that ad. Call and ask to see it. Dell is very good about things like this and you should be able to look it over yourself.
Also, ask to borrow two boxen for evaluation. They'll do it.
With Apache everything is running in user space as multiple processes - Speed Hit Here?
Yup, The apache processes run round robin (far as I know), and that means that you have to flush the CPU cache for each new connection (rotating the memory space). Note, this won't be noticable for anything but the heaviest loads (since regularly scheduled OS context-switches will have the same effect). Additionally, unless you put in enough memory, you're going to have to swap to disk throughout the rotation. This is likely when you have large mod-perl code (I've gotten apache up to 20 meg / worker-process). Additionally, AFAIK, all IO caching is redundant with MP. I THINK there was a shared memory segment that resolved this, but I don't know.
An MT version has no such problem, since the cache lines are valid for all threads, AND if you had 40 threads with 4 - 12 Meg each, it's unlikely that you'll push yourself out to disk.
We're assuming, of course that nobody is using flat-out CGIs. If anyone was, then Apache would win out-right. In a single-tasked, MT environment, you have to initiate an entirely new process space, and before that, you'd have to configure your environment. In Apache, you have ready-to-go worker-processes than can easily exec the CGI, a new worker thread is recreated at a later time. In Windows, I believe there is less process-creation overhead than in UNIX (assumption based on the non-forked model of windows), so the distinction may not be as great.
I definately agree with the added stability inherent with discrete processes. However, it can actually be harder to debug when you have faulty embedded applications (mod-perl), since each worker will have a seperate work-space, you get a different answer every web-page access. Course, on the other hand, MT perl is even less secure.
With MT, you have all the fun of race-conditions and hidden shared variables that clobber each other. Debugging that can be a nightmare. IIS gives you their equivalant of perl / visual basic. I don't really know much about it, but I speculate that it's more integral to IIS than perl is to Apache, and thus is more efficient. Not to mention, perl wasn't designed for the web whereas ASP was.
Now I know we're not focusing on perl here; you could very easily substitute php, or SSI static pages. I discount servlets, since most of your processing time supposedly is taken up in the servlet, and that's independant of the web-server. I assume that servlets are available (in one form or another) for IIS. Hell, if you really liked servlets, you would use it AS the web server.
I've heard that Apache is comming out with an MT version, BUT, here's the problem. In a windows platform, you still have all the negatives of being in server windows platform. Not to mention, Apache is a text-file configuration while windows is a gui based configuration. The two do not mix well. In the UNIX environment, however, our kernel MT model leaves much to be desired. I vaguely recall reports that showed how poorly Linux did in MT compared to other platforms (obviously Solaris, but I think we even lost to Windows). If you compare MT Apache on Linux to Windows, you might get a nasty surprise. I don't know if the situation has been improved for the 2.4 kernel. Hell, Linux isn't even POSIX compliant with MT.
For simple static-page serving, MT apache should accel. Highly optimized C code in tight loops with a fixed number of worker threads could do magic. This avoids all that BS VC++ MFC stuff. But unfortunately, you can't make a sophisticated business around static pages (unless you're into pr0n I guess).
-Michael
-Michael
Is there such a thing? Really?
As well, what's the difference between "supports" and "speed?" Not to mention cost. (I'm sorry, but I just put two Linux servers in place yesterday simply because Microsoft wanted over $5,000.00 for the software licenses alone; Secure Apache came with Red Hat for $170.00.)
The server I have up seems to foot the bill quickly and correctly. What kind of bandwidth are you going to have? Can your BANDWIDTH support that many hits? Do you expect that many hits? Contrast this again with the advantages of Open Source as you see them.
All in all, I thought Linus had to eat crow because IIS really *did* outperform Apache on Linux. Wasn't there a story about that on Slashdot about a month ago?
-- Talonius
My reality check bounced.
I have no idea what the hell you are talking about NT crashing once a day for an IIS installation, most of our production servers have 4 to 5 month uptimes. I've never seen an NT server that is simply serving pages, dynamic or static, crash. Ever. I have seen the box slowed down, but not because of microsoft, but because of Sun's shoddy JVM and Allaire's crappy JRUN installations.
I've witnessed and worked on deployments of sites that bang massive loads, and you know what? If your site is pulling a million hits a day and you're attempting to box that into one server, then you're a complete moron and deserve to suffer those crashes. No OS is going to save you from that.
For huge sites, the network design is just as important as the application design, and if you fail on implenting a solid design - regardless of what OS you are serving off of - you're going to run into problems.
This was taken from Apache's own Performance Notes Site.
IIS may be faster. I actually don't know, because I've never used it. But I will say this: I worked at LinkExchange when we were the number one company in Internet reach (52%). That's right, more eyeballs than AOL, Yahoo, and MSN combined. And we did this using Apache; both for our site, and for our banner network.
Not to start a flame war, but there's also a chance that the performance bottleneck is Linux, and not Apache - LE was using FreeBSD. There's an excellent benchmark of various Unices, which may indicate as much. It well done, but doesn't get going until page 8 or so...
Anyways, be sure to take administrative costs and bandwith constraints before making a decision.
This shows how a bleeding edge webserver, TUX 1.0, running on a tweaked 2.4.0-testX box can outperform a virtually identical box running IIS 5.0. Curiously enough, these are DELL boxes, and the tests were performed by DELL.
I understand that it is hoped that the advanced features of TUX 1.0 will eventually make their way into Apache.
Apache has been designed for correctness and it *chugs* compared to IIS, Netscape/iPlanet or Zeus. But it's stable and works well.
If you need a speed boost try putting a Squid proxy in front of it - it'll really help on static pages / images.
You may find that the Apache 2 Alpha/Beta software performs better than the 1.x line.
If your web server is down then it ain't going to be handling any traffic now, is it?
Do you *really* believe the numbers and statistics handed to you on the glossies? Wouldn't it make you a rather credulous person if you did? These are *salespeople* that make these things up yyou know.
Government of the people, by corporate executives, for corporate profits.
Where can I get -credible- data to prove that Apache can outperform IIS?
I know I'm risking a troll rating here, but shouldn't the question be "Where can I get -credible- data to prove which is better?" If you don't have access to the information that Apache kills IIS, how do you know that it is, in fact, better? I'm not saying you should take Dell's advertising at face value, or that IIS is better, but you are presupposing the answer to the question that you (admittedly) don't have the answer to. I suspect Apache would be better on many, if not all fronts, but I don't have any data to back it up either.
+5:offtopic,but anti-American