C't NT vs Linux benchmarks : Linux wins

Anonymous Coward writes "Go check out this benchmark of Linux vs NT in a real life-situation. C't makes a pretty good point here, showing Linux/Apache to be ahead of NT in performance in daily life! Also compliments the Linux community for its responsiveness: "Emails to the respective [Linux] mailing lists even resulted in special kernel patches which significantly increased performance. " This is the C't benchmark that's been bouncing around lately-translated into English, for all of the German-impaired out there.

Comments

Re:NT Crashing?

[edgy]

Reminds me of that joke where this patient goes to a doctor and says something like:

"It hurts when I do this"

"Well, then don't do that!", the doctor replies.

I think NT's reliability has been rehashed over and over, and I hear more complaints about NT crashes than about Linux crashing. I hear more glowing reports about replacing NT with Linux than the other way around.

Then again, YMMV.

Yes, but Mindcruft isn't completely Wrong

[redelm]

c't (IMHO the only independant mag left) has done much more realistic testing (page sizes, static vs CGI,load, SMP) and reported their full results. At less than 1000 hits/second, Linux soundly trounces NT.

But look toward the end of the article: with dual 100 NIC's and 1000+ hit/sec loads, NT pulls ahead. Clearly, something could be optimized further in the Linux TCP/IP stack or ethernet drivers. Perhaps finer grained kernel locking? Maybe we should thank Mindcraft for helping debug Linux! I'm sure it was by accident.

Linux Unix and all the rest

[Jute]

Being a complete newbie, can someon point me to a site or tell me exactly what makes Linux et el different to other os's and how it is used as opposed to say (I hate to say this word) Windows?

Ta from a hopeless girl.

Re:Linux Unix and all the rest

[attila_the_pun]

Hmmm. Better than all the rest? I don't think it's necesarily better than all the rest.

Compared to most other operating systems it has pros and cons. Compared to windows however it's significantly better in a number of key respects. The much decried text interface is the key here.

First Windows integrates the windowing system into the basic operating system. If your windowing system is FUBAR you can't do anything except reboot. Text based operating systems OTOH allow you to log in and use the text based utilities to find and fix the problem. It's possible to stop and restart the windowing system without rebooting the whole machine.

Secondly the use of plain text gives you great flexibility. The utilities supplied with most Unix like systems, including Linux, generate and process plain text. If you want to find all files containing a particular string and change that string you have a utility that finds and lists files according to certain criteria, e.g. filename ending in .txt. The output from this can be fed to another utility that checks whether those files contain the string you want to change and a third utility that actually makes the change. In windows on the other hand everything just creates another window. There is to take three programs that each do part of the job and chain them together to do the whole thing.

Generally Windows makes the easy jobs easier (provided you want to do them the Microsoft Way (TM)). Unix and Linux make the hard jobs easier.

Competition

[EisPick]

An important point gets lost in all the discussion about these benchmarks: Both NT/IIS and Linux/Apache perform astoundingly well, and both perform much better than they would if the other didn't exist. Developers for both sets of products borrow good ideas from the other, and both race to make improvements to keep up.

MICROS~1 flacks like to blather on about needing their monopoly position in the market to protect their "freedom to innovate," but where they have no competition, they don't innovate. Why can't they acknowledge that the only reason IIS doesn't suck is because Apache exists?

I wish they had similar competition on the desktop. If they did, maybe I wouldn't need to reboot my Win98 4x/day.

Re:That's funny (tm)

[edgy]

That's funny, QWin2k beta 1 hasn't been around for 666 days.

Service Pack 5 vs 4 vs 3

[IntlHarvester]

Using SP 4 instead of 5 makes sense, because Service Pack 5 was intended to be more of a bugfix release than anything. It's more stable (IMO) than SP4 or SP3, although there are known problems with RAS and a couple other things.

However, the tests used Service Pack *3*, which not only is seriously old, also misses several enhancements and security holes along the way.

They did manage to upgrade the Linux kernel to 2.2.9, noting that the stock 2.2.5 kernel in their distribution was slower.
--

Re:Service Pack 5 vs 4 vs 3

[IntlHarvester]

Oops!
--

Re:BAD TEST: NO SERVICE PACK 5

[IntlHarvester]

Do you have a URL that indicates that Microsoft used SP4 instead of SP5 for the ZD tests?

--

Re:No big difference with only 1 NIC

[IntlHarvester]

I don't know if the processor-NIC affinity is so ridiculous. It actually does make alot of sense for those bizarre moments when you will have >2 NICs.

Apparently it will be enabled by default in Windows 2000, so I wouldn't call it cheating either.
--

Re:Good point but..

[IntlHarvester]

Unless you are serving to an Intranet, in which case you might actually have a switched 100Mbit network.

Multiple NICs might not be that common for web serving, where speed can be gained through HTML+application design more easily, but it's used all the time for other things.
--

Re:Rebooting NT

[edgy]

Gee, seems like it takes more skill to run an NT box efficiently than a Linux box.

Linux SMP

[IntlHarvester]

I found it kind of strange that Linux performed better with SMP turned off than with it turned on, unless you applied "special patches".

http://www.heise.de/ct/english/99/13/186-1/pic10 .jpg

Consider that there's been a bunch of SMP improvements done the Mindcraft test, and that it took an unreleased SMP kernel to beat a released Uniprocessor kernel. (Are the "special patches" in 2.2.10).

No intent to spread FUD, but perhaps Linux's SMP support isn't quite ready for prime time. The numbers seem to look that way.

(Perhaps this could explain the cognative dissonance between the Mindcraft/ZD results and the average Slashdotter testimonial? How much better would Linux have done if they just turned off the SMP support?)
--

A rather dangerous trend

[Anonymous Coward]

I think denying benchmarks until you find ones that match what you want is a Bad Thing(tm). While these may be 'more fair' Linux still has a number of problems. Scheduling, and obviously some IP stack probs.

Completely Wrong. The Facts:

[FutileRedemption]

Read harder.
And think harder.

NT was significantly faster than Linux in one "pretty much" unrealistic benchmark.

Linux was massively faster in two "somewhat" unrealistic benchmarks.

Linux was slightly faster in the other benchmarks.

So please tell me why do you think that NT/IIS is "a better high performance web/file server"???

The point is that NT is optimized for one single case, possibly only needed by something like a mega high volume porn site (static pages, as Alan Cox pointed out), and linux does better in all other cases.

And if you want to server MANY files, you need to buy TEN NT boxes instead of ONE Linux box.

For the T1: Linux is a fine solution for a 100MBit site. A T1 is 1.5 MBit.

Please do not confuse the facts.

Re:Something that's been bothering me...

[tgd]

Well a few points:

1) NT tends to not handle high loads as gracefully as Linux. Linux/Apache tend to slow to a crawl under high loads, but I've never managed to crash the server. I've done that a bunch of times under NT (usually when running SQL server and IIS on the same machine... but running various combinations of Oracle, Sybase or MySQL with Apache under Linux doesn't seem to cause a problem...)

2) I've found the most unreliable NT servers are the ones that people have been hacking around on, tweaking, etc. Vanilla NT with everything else carefully installed seems fairly stable. Mind you, I'd never run a serious application on one, but you CAN get them working reliably. Its harder to keep non-administrators (ie, clueless management) from messing around on NT servers than Linux servers. (I once built a sandbox system that ran a clone of the system in a chroot'ed sandbox with the logins on the first six vc's pointing at it, with one on nine pointing to the real system -- I guessed that the owner of the company I was working for at the time was monkeying around in the system and that's why Linux kept crashing. After doing that, the sandboxed system kept flaking out, but the production one stayed up!)

3) Buggy COM objects and ISAPI objects are prone to crashing various parts of NT, like IIS and for whatever reason, causing bluescreens. Lots of sites use not-so-stable third party COM objects and ISAPI's and from experience, it can be a real bitch to figure out whats causing the server to crap out under high loads in that case. The last major website I built using NT, I ended up rewriting all the COM objects we'd bought in Java so I had source and could fix the bugs that were my fault. :)

4) Slashdot isn't all that reliable. I have problems with it all the time. (Connection refused, partial pages, broken HTML, extra blank stories, etc...)

#4 isn't a flame at slashdot -- god knows I spend enough time on here commenting on things and reading the site. Slashdot is amazingly stable for the way its architected (running on a single server, no redundancy at the server level or the hardware level, etc...) I wouldn't run a high traffic site I was paid to build like that, but they're doing great for bootstrapping the site themselves.

Re:uh, yeah..

[raistlinne]

No, I know both C and perl and like both quite a bit. I use them for different things, but when speed isn't an issue and text processing is, I absolutely love perl.

Re:You're making up those instability claims.

[raistlinne]

Was w2k beta really released 1 year and 10 months ago? And they still haven't released the actual w2k? And this is a real server which gets used significantly all the time?

Something that's been bothering me...

[kwalker]

Something that bothered me about the Mindcraft studies that was partially explained in the earlier article posted here about saturating a T1/T3 on a single-processor Linux box, and still further explained in this article...

If NT is such hot stuff running a webserver, how come so many NT servers die horribly when they're slashdotted, yet slashdot (P2x2 256MB ram if I remember correctly) has enough processor time and bandwidth left over to customize the interface and most of the pages that it spits out? I have seen so many high-traffic NT sites bog down and sometimes just not respond when they get busy, yet most Linux/FreeBSD servers keep chugging right along.

I wonder if there's a way to benchmark that...

Time is the test both C'T and Mindcraft forgot

[pmancini]

I love the web - it is the great equalizer. Bad benchmarks like Mindcraft can be shot down in quick order. However there is one test that would have crushed NT in BOTH tests. It is simply this: Conduct the test over a 6 week period.

Having worked in an ALL NT house and now in an ALL UNIX house I can tell you that the NT/IIS server will crash NO LESS than 8 times in 6 weeks and require hours to fix/restart. That has been my experience at a company that had 80+ NT servers doing real life web application work.

I used to complain that LINUX/APACHE was no match for NT/IIS because the application platform from Microsoft is simply amazing. I've since seen something called PHP3 and that looks as good if not better than IIS. Does anyone have any experience with PHP3? Is it very powerful?

--Pete

Re:Time is the test both C'T and Mindcraft forgot

[J-F Mammet]

Yes PHP is an amazing programming langage for the web. I use it every day at work, and every day I find a new functionnality I did not know and I'm happy. The developpement is still going very fast on PHP3, with PHP4 on its way to the beta for the end of the month.

For example with PHP you can generate on-the-fly gifs, but also on-the-fly pdf files ! With database integration think about all the possibilities you have. I'm working (among other things) on a FULL template system which not only allow you to change the HTML of a page, but also all the images/buttons, without having to recreate them all. You can use true type and postscript type 1 fonts in you gifs. The only problem with the generated gifs is that they are RLE compressed so I pipe them to gifsicle and I get a maximum compressed file.

Btw do not compare IIS with PHP. IIS is a web server, PHP a scripting langage for apache (and CGI). If you want to compare, go with ASP. There are many pages comparing ASP/Perl/PHP. I hate ASP so I won't speak about it. I'm a C developper so for me PHP with its C syntax is a dream (no mallocs !). Perl is not structured enough for my liking.

I think every web developper should give PHP a try. It changed my life 8)

J-F Mammet
webmaster@softgallery.com

BAD TEST: NO SERVICE PACK 5

[Anonymous Coward]

How come they didn't use Service Pack 5????

They use the latest and greatest Linux version.
Yet they failed to use SP5 for NT?

SP5 has many performance enhacing features
for multi-cpu configurations.

Zealots?

[Ristoril]

I hate Microsoft. I think their software is bunk.

With that out of the way, I do have an observation that I believe is worth consideration.

When Mindcraft came out with their benchmarking tests, this place (as well as their mail server) was flooded with 'what a bogus test!' 'you MS whores!' and the venerable 'go f*ck yourselves!'

However, when these benchmarks come out, and say that Linux beat NT, they are automatically heralded as The Truth. Now, I really do like the fact that Linux has been 'vindicated', but what guarantees do we have that these tests were any less biased than the ones that said NT won?

I know a lot of you will think I'm a heretic, but we need to present an image of being clear-headed observers. The way not to do this is to automatically discount every benchmark that says NT is better while automatically accepting benchmarks that say Linux is better as God's Own Truth.

Just so I can be sure you guys understand, I'll reiterate:

1. Linux rules
2. Microsoft sucks
3. A benchmark is not trustworthy merely because it agrees with your beliefs

Ristoril

It's ASP and other dynamic content

[Matts]

ASP's are slow. See the benchmarks done by the mod_perl people on perl.apache.org. NT is notorious for slow dynamic content. Unless you write everything as ultra-optimised ISAPI dll's you'll suffer the same fate - as I've experienced to my great embarassment - gladly I've vowed to never take *that* route again... :)

Matt.

perl -e 'print scalar reverse q(\)-: ,hacker Perl another Just)'

Minor little point

[DonkPunch]

IIRC, the "486 and higher" PCs were client machines making the http requests to the test server. An http request from a 486 taxes a server just as much as a PIII's request.

Six Points

[FutileRedemption]

- pretty much everybody uses one of the configurations c't tested

- pretty much nobody uses or will use mindcraft's setup (raid 0, 400 mbit net connection, 4 way xeon server to exclusively serve static pages)

- like probably no other magazine in europe, c't is renowned for independence, objectiveness, competence

- what is known about mindcraft is that they did another test some time ago, seemingly with a setup advantageous for NT (against Novell Netware)

and:

- the mindcraft test was payed by microsoft, mindcraft conducted the test in a microsoft lab, mindcraft used microsoft email accounts

- the c't test was payed by Heise verlag. And by the way: Heise runs Solaris.

If you are really objective now, what will your conclusion look like?