Slashdot Mirror
Red Hat/Apache Slower Than Windows Server 2003?
phantomfive writes "In a recent test by a company called Veritest, Windows 2003 web server performs up to 300% higher throughput than Red Hat Linux running with Apache. Veritest used webbench to do there testing. Since the test was commisioned by Microsoft, is this just more FUD from a company with a long history? Or are the results valid this time? The study can be found here."
Looking at the first page of the benchmark report, I see that they're using the exact same setup as in their highly contested samba benchmark, with a specific ancient version of Red Hat running on a specific hardware setup that version is known to have performance problems on. They could have at least tried a different server last time, or a modern version of Linux. Under fairer circumstances, who knows, IIS might have still won, but this rigged benchmark has nothing to offer us in deciding which server is faster.
Out of the box Apache doesn't do too well. But take some time tuning it, and your OS's TCP/IP stack, and you can easily outperform even Zeus. Read some of the tuning guides.
Let's see. A test commissioned by Microsoft says IIS is faster than Apache. The link for more information goes to microsoft.com. Is this really "news"? Seems more like a thinly-disguised press release...
Simpli - Your source for San Jose dedicated servers and colocation!
At least they're up-front about it these days.
Other Veritest-Microsoft fun:
http://www.veritest.com/clients/reports/microsoft
http://www.microsoft.com/windowsserversystem/fact
http://www.gotdotnet.com/team/compare/veritest.as
In short, this is a company paid by Microsoft to make reports/whitepapers that make Microsoft look good. Nothing wrong with that as long as everyone's aware
rooooar
Faster to get infected.
Faster to get rooted.
Faster to get used as a warez server.
Nothing new here.
So does that make SMS on Windows faster than morse code on Linux?
Windows 2003 server running on skynet is 300% Faster than Ye-oldie redhat -12 edition from 1723 running on an abacus.
This reliable Expensive test paid for by Microsoft to show how much better windows 2003 server is(the payment came with a clause stating such).
The only things certain in war are Propaganda and Death. You can never be sure which is which though
It very well could be. However, let's try 1) an indepedent test, paid for by neither competitor, and 2) the most recent version of IIS against the most recent version of Apache, and 3) the most recent version of Windows against the most recent version of Linux. I can guarantee a win in any test so long as I am allowed to dictate all of the conditions. I wonder how many combinations they tried before they found one that IIS6 could beat?
The web page says it was published May 5, 2004, i.e. a year ago. The report itself is dated from April 2003. The test was done using RH advanced server 2 and Windows 2003 RC2, i.e. a pre-release version. Since then, both RH and Microsoft have published new releases, for example the service pack 1 of Windows 2003. Why is this posted now?
Sheesh -- with such outdated news, I almost felt like I was reading the newspaper or something.
Here's an article from linuxworld. Dated but shows that Veritest makes mistakes in setup.
I like Microsoft, and I like when somebody defends them.
I've been in IT for about 17 years. I've seen MS destroy "the little guy" time and time again, with thier power and yet with all that power, money and developer base, deliver garbage year after year, to this day.
Then I compare them with offerings like Mac OS X, the BSD's and Linux and wonder, how on Earth someone can say, "I like Microsoft".
Seriously now, what is there to like about them?
300% is pretty hard to believe.
Apache was never optimized for serving lots of small, static files so I can easily believe it falling behind in some benchmarks, but not 300%.
It doesn't take much computer to saturate a lot of bandwidth, which is why most people don't care, but big sites will often have a Zeus (or similar) server set up for serving images precisely because Apache isn't as good for that. But you've got to be huge before you get to that point.
Dynamic content put Apache where it is. It has the support, the tools, the libraries, and the widespread expertise to do dynamic content pretty damn well. It's not better than everyone at everything there either, but it's a very good solution for most cases.
I rarely criticize things I don't care about.
Anyone do the math to see what that would cost.
It's conventional wisdom that Google has about 100,000 servers. If google went with Windows 2003 Server Enterprise Edition (which costs $3999 ) That would cost google about half a billion dollars.
Extending the logic to use SQL Server Enterprise Edition as their search database, at $25000/server the price would go up to about $2.5 Billion.
Every CEO likes to be like Google and likes talking about numbers like billions of dollars; so this is a fun set of numbers to throw around when your're discussing microsoft partnerships with the CEO.
(Note, however, that in the true spirit of Team99, I must say that Longhorn will make it well worth the price, though, and I wouldn't be surprised to see Google switch)
People keep saying, 'When are we going to get a real benchmark?" Well, why don't we roll our own? Seriously.
Here's my idea:
Slashdot has strong zealot^H^H^H^H^H^Hsupporters for both Microsoft and Linux. Let's have a contest to select the best qualified from each side, have them work in teams on identical hardware. Let them make any changes, tweaks or optimisations they can dream up. Then, let 'em rip.
I'm dead serious about this, by the way. Let's get off this endless roundabout and for once make a clear comparison.
For bonus points, once the first contest is finished, we should take the two servers, leave them exposed to the Internet and see which one gets 0wned first. 8^)
Crumb's Corollary: Never bring a knife to a bun fight.
Microsoft argue that Apache is slower because CGI is slower. They say that it needs to spawn a new process for each request, which is correct.
But how many years have mod_perl and mod_php been around now? Does anyone actually use CGI on Apache this decade?
Perhaps a more fair comparison would have compared CGI on IIS with CGI on Apache. And I'm pretty sure that for various reasons (spawning processes is slower on Win32 than on Linux) IIS would lose horribly.
Karma: It's all a bunch of tree-huggin' hippy crap!
If someone publishes a benchmark about your software, and finds out your software does not perform well, don't whine, don't behave like a child, don't start kicking and screaming, don't tear his hair out. Behave professionally.
Good starting points:
Let me summarize what I think about their test. First of all, I believe their numbers. Apache sucks performance-wise, in particular if you run a busy site with dynamic content. That's why people are using squid in local accelerator mode before Apache. This is a good indication that some performance tuning is in order. But no, people rather wait for Microsoft to find out and then they start thinking about fixing it.
If this test was meant to be unfair FUD, they would not have tested TUX, just Apache.
But now to my questions above:
Question 1: is their setup relevant?
No. Sites who answer more than 5000 requests per second are not using a single web server, they are using a load balancer and a cluster.
Question 2: Can their numbers possibly be true?
The point I find least believable is that IIS had better CGI performance than Apache. Creating a process is really slow on Windows. Their result should be independently verified.
Question 3: What weak spots about the competition does their test reveal?
They did not test a single-CPU webserver (which is what almost everyone is using).
They did not test FastCGI or APAPI dynamic web pages.
So if we wanted to do a more balanced review, we would look at these.
Question 4: What can we do to improve the results.
Document APAPI better, I'd say. Almost nobody is writing their dynamic web page modules with APAPI.
Everyone is using PHP or mod_perl. Benchmark Apache in real-world scenarios. Document best practices.
> 1. You rejuvenate and dance when you hear a windows flaw exposed, but you conveniently ignore the thousands of security flaws exposed in linux.
"Rejuvenate" means "renew, appear to grow younger". Did you mean "become jubilant"?
I don't become jubilant when anybody's security flaw is exposed. In the case of Open Source apps, patches are generally available in a couple of days.
> 2. You yell loudly TROLL! at any person's post or at any person you see posting facts that you do not want to hear about your oh so cool linux.
No, just the ones that misstate the facts or are attempts at FUD.
> 3. You know it's a classic case of penis envy, you don't have all the support, software and hardware available for linux and you have to let that anger out somewhere, but you don't have the brains to admit it.
Um, Linux supports all my hardware just great.
> 4. You hate windows, hate Microsoft, but race to emulate windows, have programs to run office from within linux, and spend a $300 on a Windows emulator, only Windows fools.
> I run Linux, Windows, and Solaris machines. I use OpenOffice.org and so have no need for Microsoft Office. But if I did, I could run it using WINE, which I can get for free. Unlike MS Office.
> 5. You cannot admit that you don't have professional usage of Linux outside server markets.
I use Linux *professionally* on the desktop.
> 6. You cannot admit that most of the joe user out there when told that there is linux will respond, what is that?
Sounds like there's a need for some consciousness-raising, then. Alothugh I've noticed that more and more people -- even Joe Sixpack types -- don't go glassy-eyed when Linux is mentioned these days.
> 7. You cannot admit that there is no professional printing capabilities in linux.
I don't have any problems printing from Linux.
> 8. You cannot admit that you are a masochist (otherwise why would someone spend hours playing with scripts, and recompiling programs that are available for Windows?)
Well, it did take me about 30 seconds to learn how to type "./configure - make - make install - make clean". Or if I'm feeling lazy, I can just double-click an RPM file icon in Konqueror.
> 9. You cannot admit that there is no professional desktop publishing done on Linux.
Sorry, mate, you're talking to someone who does just that for a living.
> 10. You cannot admit that no one in their right mind would do professional video editing in Linux.
I honestly don't know about that. But I do know that lots of movies' special effects are being generated these days using Linux-powered render farms.
> 11. You cannot admit that linux sucks when it comes for gaming/home entertainment or education.
There are tonnes of educational apps available for Linux -- many of them come with commercial distros. There are still more on the Net. As for games -- if I want to play games, I'll buy an X-Box.
> 12. You have problems in understanding Windows, and you will blame your own incompetence on Microsoft.
Over the years, I've used and administered Windows 3.1/95/98/Me/2000 and have no problems doing so. But after just 6 months, I can install, configure, and administer a Linux machine faster and more reliably.
> 13. You have problems in pointing a clicking, but have no problems in wading through cryptic scripts written by lunatics.
Pointing and clicking has its place. But there are lots of things that are actually easier via a command line. For instance, I'd much rather run a MySQL server that way than use the GUI tools. Nice thing about Linux and Open Source apps in general is that you've a choice in the matter. If you don't like the command line, don't use the bloody thing.
> 14. Nothing will get past that shit that fills your head, you will not admit to any facts.
Can't respond to an assertion that's semantically nil, sorry.
> 15. Yo
Il n'y a pas de Planet B.
You are mistaken on some Apache concepts and how threads (?used to?) work on Linux.
This is because for each request, Windows must create a new process (the CGI program), and destroy the process when the request is complete. While the execution time is low, the process management overhead dwarfs the actual page runtime, because Windows doesn't do that sort of thing quickly. This is why CGI has long been blacklistedon Windows systems by good web devs, and this is one reason that Apache 1.x was such a dog on Windows. Apache 1.x creates a new Apache process for each request.
No.
Now Linux, on the other hand, creates processes about as fast as it creates threads, which is to say, really damn fast.
Yes, but only because pthreads does this by creating a new process (that just happens to share some things with its parents, like address space). Ergo, creating threads is just as fast as creating processes because they are nearly the same thing.
The NPTL in 2.6 might have changed this, but I have not read the docs yet.
Yet Apache is still back here creating a process or thread for each and every request (note that there are some ways to speed things up. FastCGI comes to mind, but I don't want to get into the gory details that I don't know enough about). This is not the brightest way to do it in terms of performance, but then, Apache appears to have been designed for universality and configurability over raw throughput.
No, Apache does not create a new process for each request. It creates a pool of child processes which sit waiting for requests. The parent monitors this pool and creates new spare children when too many child processes are busy. This way, most of the time a request comes in there is already a child process sitting idle waiting for work.
CGI does indeed require forking a new process, but there are already great ways to handle this. mod_perl, mod_php, mod_python all do it by embeding the interpreter inside the server. FastCGI keeps a version of the program running (much like apache does with its spares).
You are correct in that your description isn't the brightest way to do things. That's why operating system designers solved these problems years ago.
For static content, again, Apache creates a new process or thread for every request (with some exceptions). If you'll forgive a bit of an oversimplification, it's like writing a program that prints text to the screen. One program calls printf() in a loop. The other program executes a second program which itself displays just one line, and runs that in a loop.
Again, no. Apache will usually not need to create a new process or thread for every connection. The correct analogy would be the other program spawning the required number of children, and then asking them to all printf at the same time.
Why couldn't IIS be faster than Apache?
It could be. However, this test is severely flawed in that they performed registry level optimisations to the Windows setup, yet equivalent optimisations that are well documented for Linux were not performed. Therefore, we don't know.
Surprise, they enabled DisableLastAccesS on windows but did not mounted linux filesystems with noatime
noatime disables the update of the "last accesS" field of files, and improves the performance a lot for some workloads. If you check the latest article about the kernel.org servers, they found that they reduced the system load to the half by just using this option
This analisys is biased. Who cares, anyway?
A lot of people would say "if you don't like that, don't use MS spanners". Fine. Done deal! :)
Just a little way down the metaphorical road, there's a shop that sells spanners at a fraction of the price that MS does. They may not be as pretty, and for some jobs they aren't quite as exact - but they've been getting better for years and the difference is scarcely noticeable these days. And if you can do without the fancy packaging, you can go online and get that same tool free..
And it's then yours to use legally, wherever and however you wish - so long as you don't try and claim you designed it.
So the question is: by what criteria do you evaluate best? None free software, security holes, forced upgrades... with many people these things carry a hefty negative.
Don't let THEM immanentize the Eschaton!
Actually the ultimate test would be for an independant party to Sponsor a challenge.
Each would team would get(windows and linux):
$5,000 in cash with which to buy hardware and software. All purchases must carry a receipt and all parts must run to spec. No overclocking.
Garunteed 5 9's power.
Each Team's computer will be housed in the same independant facility maintained by Sponsor.
The contest can last no longer than a year. Each team will be able to maintain their own server throughout the competition.
The scoring will be simple. You won't lose points for having down time. Your score is simply the number server pages(the kind to be determined) you've properly served before your first moment of downtime. So if your server crashes before the year is over, the number of pages served up to that point is your score.
Maybe someone has an idea for what a good server is to run.
The "big players" can contribute cash, but not hardware. It is too easy for them to contribute hardware specifically enhanced for their product.
#1. Each team gets X dollars and no restrictions on what it can buy. After all, that should be how businesses run their shops. We aren't comparing hardware, but total systems.
#2. Each team must purchase the software off the shelf.
#3. No team is allowed to recompile anything or to use any drivers, etc not available from a public server for the past 12 months. This might sound like a bad deal for Linux, but it will also stop Microsoft from re-writing the drivers. Again, most companies do not have access to that level of expertise so that won't be allowed.
#4. Each tweak or configuration setting must be documented and a reference for it shown on a public website or manual. Again, businesses only know what they can read.
#5. At the end of the competition, the other teams will critique each team's configuration. We've all seen the "tests" where Windows is running on a RAID 0 array which is beyond stupid for real production work.
That way, each team can deploy the best system they can think of for the test. I'm sure you all remember MindCraft and their massive single server "test" for webservers when anyone else would have run multiple cheaper servers and gotten higher throughput.
So, a test in run and the Windows team buys the biggest single system they can afford for the money. While the Linux team fields a dozen boxes booting from CD and one storage box.
Which system would be "better"?
Which system would be faster? Would that be the same answer under different loads?
Which system would be easier to maintain?
Which system would have higher uptime?
Which system would be easier to scale up?