Slashdot Mirror
NT faster than Linux in tests
Mike_Miller writes "The lastest Mindcraft Study claims that Microsoft Windows NT Server is 2.5 times faster than Linux as a file server and 3.7 times faster as a web server. Their white paper shows that NT beats Linux on every test. " Anyone have a critique?
Fished out of dejanews -- the Mindcraft folks used the pseudonym 'will@whistlingfish.net'
(If this was posted earlier, I didn't see it...)
Can anybody here respond to this?
Hi Everybody,
We're considering using Linux + Apache as a web server. The hardware
is a 4-processor 400 MHz (Xeon) server with 1GB of ram, a RAID controller,
and six disks. We have Redhat 5.2 installed and compiled an SMP version
of the 2.2.2 Linux kernel. For the web server we used the latest 2.0.3
version of Apache.
The scenario: we're bangin' on this web server with a bunch of clients
to try and get a handle on its capacity. Simple static HTML requests,
no heavy CGI yet. My Apache server is tuned up, MaxClients is 460.
I recompiled with HARD_SERVER_LIMIT set to 500. Limit on number of
processes is 512, imit on file descriptors is 1024.
The problem: the server performs well, delivering in excess of 1300
HTTP GET requests per second. But then performance drops WAAAY
off, like down to 70 connections per second. We're not swapping,
the network isn't saturated (4 x 100Mbit nets), disks are hardly used,
but the system is just crawling. If it were saturated then performance
should level off, not drop like this. Neither vmstat nor top show
anything unusual. No error messages in the web server. Its puzzling.
Any ideas? Any tips, suggestions, or pointers would be appreciated.
Thanks!
Thanks. I just checked that out. It does appear that they asked a single question about Apache performance. I remember seeing that posting myself and blowing it off because there wasn't enough info to tell him anything and I didn't feel like going into the give-and-take to get enough info to do something. (I do enough of that supporting my own customers!). Now, in hindsight, knowing what he did not do to Linux, the answer is obvious: he was running out of file handles. Do the math. An inactive Apache server has 8 file handles open. 127 servers max * 8 = 1016. Default file_max is 1024 for Linux, of which 150 or so are usually open while the system is at a rest. Apache could not bind a socket to a file handle for incoming connections because there were no file handles. So Apache was basically deadlocked, waiting for file handles to come free so it could accept() the socket but it was already holding all the file handles!
If there had been questions about general tuning of such a large system, that would have solved the problem because someone would have remembered about file_max. But one cryptic query that didn't give enough information to get help does not an honest effort make.
Anyhow: I guess I have to post a partial retraction. They did post a *SINGLE* query to the net.
-- Eric
Send mail here if you want to reach me.
Set OPTIM = "-04 -m486" before compiling
on 4 x 400 MHz Pentium II Xeon
Samba 2.0.1 Configuration
wide links = no
That creates a bottleneck in Samba performance, see here
the following processes were running ... (kswapd), /sbin/kerneld,syslogd,
not sure if that means something, but why they run kerneld with 2.2 kernel ?
On NT side:
Tcpip\Parameters\Tcpwindowsize = 65535
that makes huge boost on network performance, but only on local network where packets don't get lost
Set Logging - "Next Log Time Period" = "When file size reaches 100 MB"
Logs on the F: drive (RAID) along with the WebBench data files So basically server does much less logging than Apache - and since it's many small requests, and since Apache writes logs on a non-RAID disk all together it'll be a big bottleneck
Anyone noted anything else wrong with this benchmark ?
From all my experience it looks like pure crap
P.S. Why they needed NFS ? inetd ?
Obama 2012: our incompetent asshole is slightly less of an incompetent asshole than the other incompetent asshole !
Yeah, this study is sponsored by Microsoft, if you read the fine print:
Mindcraft Certification
Mindcraft, Inc. conducted the performance tests described in this report between March 10 and March 13, 1999. Microsoft Corporation sponsored the testing reported herein.
Looks like you can buy anything you want with enough money. It doesn't make it a true indication of a real-world situation.
I think that there's enough evidence to the contrary already out there, and this will only serve to discredit Mindcraft.
According to a posting on Linux Today by Jeremy Alison of the Samba Team, it seems that the Mindcraft study crippled the Samba server in the tests:
:)
From Andrew Trigell (original Author of Samba):
They set "widelinks = no" now I wonder why they did that
In case you haven't guessed, that will lower the performance enormously. It adds 3 chdir() calls and 3 getwd() calls to every filename lookup. That will especially hurt on a SMP system.
Okay, folks. So we have a bit of egg on our face for this one, because nobody (to my knowledge) has really stepped forward with large-server Linux benchmarks which demonstrate anything differently. It may be that Mindcraft royally screwed up, or it might be that Linux really is slower than NT for a certain set of benchmarks -- the truth is more likely a combination of these factors.
If Linux is going to be treated as a serious operating system by the majority of the IT community, it's going to have to step up to the plate and demonstrate scalability and performance which does rival NT server in this area. Most of our knowledge about Linux-vs-NT performance is somewhat anecdotal -- we haven't really "put our money where our mouth is" and shown objectively that Linux can outperform NT in these areas.
Rather than dismissing this study as FUD, I think we could learn a few valuable lessons from this study. We should seek to understand why the benchmark results weren't as great as we would have liked. We should fix any obvious bugs or misfeatures in Samba, Apache, and the Linux kernel which stood in the way of higher performance. And we should stive to improve the entire system to make it be a true NT rival.
We have a lot going for us. First of all, we can innovate at a much more rapid pace than Microsoft -- so hopefully within just a few short months (and I'm being pessimistic!) we could demonstrate a high-performance Linux file and Web server which kicks NT's butt all over the place.
Nobody said building a high-performance, scalable Internet server operating system was easy. Let's get to it!
Matt Welsh, mdw@metalab.unc.edu
The net posts asking for help that are mentioned in the white paper appear to have been most likely made under the pseudonym:
will@whistlingfish.net
Use DejaNews.
No-one seems to have done that and talked about it. I did; h ere's the relevant link that lists all the messages from this guy on Usenet. Take a look at them and post what you think about them. It seems to me he hit a strange, obscure bug in GNU, Linux, or Apache, and it might have something to do with network adapter or SCSI adapter problems.
--jon. Postel is dead. May we all mourn his, and our, loss.
So they have 4 network cards in that nice little Dell machine, and they specifically mention "Used the affinity tool to bind one NIC to each CPU", so my question is whether they even bothered to use the other 3 NICs under Linux.
It seems to me that Linux with one network card doing only 2.5 times under NT with 4 network cards sounds about right. Give Linux 4 network cards and you get performace that easily blows NT away.
To use multiply NICs you have to use the network driver as a module I believe, did they bother to do that? I can't imagine Red Hat's installer "How many network cards do you have, but then again maybe it does, I'm a Slackware kinda person...
---- Gee, I wish I could read Kanji...
The beowulf newsgroup had a couple short threads a couple months ago about consistently abyssmal performance on redhat 5.2 SMP machines running 2.x with > 512 MB of RAM. The two threads [ one, two] deal with users who had horrendous performance problems with their new machines (both running 2.2.2, the same kernel as in the report) when they used more than 512 MB of ram, but the performance jumped right back up when they used 512 or less. Check out the articles to see how bad the performance was; it's pretty surprising, and presents an interesting opportunity for detractors of linux:
Linux definitely has some hardware/kernel combinations that would seem OK by design on paper, but exhibit peculiar behavior in practice, especially with SMP. I wouldn't rule out the possibility of the testers (or financial backers) hand-picking kernels/hardware configurations that could affect results while seeming perfectly viable to the layman.
It seems very likely to me that if Microsoft did not outwardly donate the hardware to the testing company, they at least made suggestions on its configuration. The open nature of linux development and bug disclosure could easily be used by companies wishing to stage biased demonstrations; Microsoft almost certainly does a thorough job tracking linux kernel development and bug reports.
-- Scott
Please submit any inconsistancies you see in this document (and if you don't see any, please shoot yourself in the head) to lwn@lwn.net. They are readying a response as we speak.