Slashdot Mirror
NT vs. Linux: Again
Jeff Molloy writes "The results are here
link " It's a
shame Linux didn't win, but it looks like the tests show where Linux might
have some deficiencies. Overall, it looks better than the original test, though.
← Back to Stories (view on slashdot.org)
If you can't help program then go out and test all this new stuff and send in bug reports. Let's have Linux set the standard again. It seems like, acording to the article, it was this way once and we lost it because Microsoft has pushed the bar a little higher and we lagged behind.
It's so nice of Microsoft to pay for this apache advertising. Just as a point of reference, 1800 hits/sec is the same as 155,520,000 hits/day. I think that it's safe to say that noone in the world gets more than 150 million hits per day of static content. Wait, there's a better way:
1800 hits/sec * average 2k/hit * 8192 kbits/kbyte = 29,491,200 bits/sec, or 29.5 MBits/sec. What's that now, a T3 line? I know that a T1 line is 1.5 MBits/sec. Ok, so apache on one of these boxes can fille the equivalent of 19.6 T1 lines by itself. If (a bit more realistically, how many 2k files get those types of hits) those are just 10k files (let's not get into pictures), that's 147.5 MBits/sec, more than filling a T3 line, IIRC, and definitely filling up aapr. 98.3 T1 lines.
What's the problem with Linux/Apache, now?
May I suggest, if you can afford this sort of bandwidth, that you buy one of those 32CPU sun E10000 servers and call it a day? (or a server farm of linux boxes, since you're serving up static files.)
Oh, if you're serving up >1800 files per second of 2k files, who are you?
Oh, one more thing. If this is all on an intranet, you'll still need Gigabit ethernet if you're serving up the 10k+ files, so the sun box still applies to you.
They laughed at Einstein. They laughed at the Wright Brothers. But they also laughed at Bozo the Clown. -- C. Sagan
All of your stuff is completely relative.
Price/Performance
It's all related to how much you pay your admins and how well they administer your system. This isn't a function of the OS. Yes, Linux costs less out of the box, but an NT admin is going to have a harder time (and thus charge more) to set it up than he is an NT system. If a business currently has functioning NT systems and competent NT sysadmins, why should they switch to Linux?
Clustering
How many small businesses who are choosing between Linux and NT need to, want to, or care about the ability to cluster? People who care about this benchmark are not the same people who need to run clusters.
Other Hardware Configurations
How much would it cost for a company to build a Linux-happy system? Most systems built today (and the systems that we want Linux to run on) are built for Microsoft. You'd need a custom-built, custom-designed solution to truly grab all of Linux's power, and that costs money, either in man-hours or purchasing power. The results of this test would've been far more atypical if they had built both machines finely tuned for Linux. At least this time around, they weren't blatantly geared towards Microsoft.
Security
Security, I'd say, is 75% system administration and 25% OS. Linux has its security problems as well, most of which can be plugged up with effective network management. Many of NT's can, too. MS may be a lot more apathetic to security concerns, but they don't run the systems, they sell them. I don't consider Linux or NT any more secure than the other.
Stability
Stability can be completely a function of management. I've heard stories of Linux systems stay up for months or years. Guess what, I've heard the same stories about NT as well. I've also heard stories about unstable Linux systems. I've seen no long term studies done on system stability, so everything I hear about stability I file away under anecdotal evidence, not hard verifiable data.
Change real world needs
What's good for the goose is good for the gander. I don't see how this benefits Linux. Change the system and, whoa, Linux might perform worse under that setup. It happens to both types of OSes, and before you say, "It happens to Linux less!" find some hard data, not stories.
The Future
Past trends do not determine future performance. I doubt Linux will keep up its 212%/year growth and Linus has already said that upgrades aren't going to be as drastic as 2.0 to 2.2. Don't assume that Linux will advance in the next three years as it has in the past three years.
From CMP's "Information Week" June 21, 1999:
"When every minute of downtime can mean millions of dollars in lost revenue, companies generally rely on applications that run on OS/390, Tandem NonStop Kernel, Digital OpenVMS, or Unix operating systems. But Windows NT is increasingly being deployed... so IT managers must find ways to increase the availability of their NT environments. To do it, they're adopting products and services that promise to provide extra protection..."
" 'Any system with lag time is unacceptable for running the application' says William Harris, NT Administrator for the Ohio Utilities. 'Money wasn't even a big deal. I's rather get quality and reliability and availability'. The organization...paid $75,000 to implement the (third party protection) system.
Translation (for those who need it): Management is telling IT they have to transition to NT. IT says, in order to be stable, we have to add third party help. Management says: "Here's a blank check."
It goes on to say that Unix, w/o third party software or service achieves "availability in the 99.9% range, as opposed to 97% for NT."
Now, what's the difference to a business between 97% and 99.9%?
IBM's NetFinity Availability Program guarantees 99.9 w/ NT. Cost: $220,000.
HP Mission Critical guarantees 99.9 with NT for a mere $300,000.
Imagine going to your boss and saying "Hey, how'd you like to save $300,000?"
JL Culp
Business Technology Consultant
Chair, LPSC
I think this feature explains, at least in part, NT's superiority in multiple-CPU raw service.
A side note to flamers: please, PLEASE don't treat these results as suspect or corrupt. I don't think they are. Don't think of them as a defeat, think of them, like ZD said, as a roadmap to show where Linux needs improvement.
I used to run these benchmarks, and worked with the people who wrote them. They were designed to work best on Intel hardware with NT and IIS. It was intentional, and to use a ZD benchmark in this type of comparison is laughable.
------------------------------
NT has put many services in kernelspace and has largely bypassed their HAL in favor of multimedia performance - especially video.
NT uses a multithreaded process model for IIS and SMB file-services that results in higher throughput but less stability. A single thread of the main process may die without completely destabilizing the server but if the main process dies then all child threads die.
Linux divorces the graphical user interface from the kernel thus ensuring stability (framebuffers are available for video enhancement though) and implements most services as userspace daemons.
Linux uses the forked process model to provide services to multiple users. This modem achieves stability in that if one process dies, the others continue as if nothing had happened. Both Apache and SAMBA operate in this way I believe.
NT has chosen performance over stability.
I believe that with kernel enhancements and profiling, any bottlenecks in the networking system can be eradicated causing Linux to perform much faster and possibly even beat NT in tests such as these.
Codifex Maximus ~ In search of... a shorter sig.
But here is our chance to show the world [and MS] why Linux and other OSS projects are such a good idea. By quickly implimenting fixes to the problems brought to light by these test, we can prove how much better OSS is.
Proposal: Annual or semi-annual benchmarking of NT [or the current MS server platform] and Linux [and any other OS's that want to compete I suppose]. By doing similar tests regularly, we can show how efficient OSS can be at fixing current shortcomings [as if 24hr bugfixes aren't enough].
Just a thought. ;)
BTW: Sorry for the overuse of the "OSS" buzzword
I'd help with implimenting fixes myself, but I'm not exactly an expert coder [I don't think "Hello World" will help Linux beat NT]
Ender
If at first you DO succeed, try not to look astonished!
Nothing to see here
But I don't think that this means that speed for web serving should be any more important. Getting back at Microsoft is not a reason to improve Linux in my book. There is are many other fronts that Linux heading toward like the desktop, embedded devices, and hand helds. I can imagine that if Linux is tweeked for web serving more than normal that some test will find Linux useless for embedded devices or something else that is important.
Microsoft right now sees Linux as direct competition as a server. It will be nice to see Linux compete back but don't expect NT to stand still. There are other servers also. How does Linux compare to Mac OS X?
And no more excuses. Linux is not the fastest. Deal with it.
For now.
--
"In the other corner, two cardboard boxes; one labeled 'Windows NT Server,' the other 'Microsoft IIS'..."
This all inspired by:
(Yeah yeah, apples to apples...)
-m
I find these studies inadequate as data to inform a purchasing decision. While MS will claim that they have proven NT to be better than Linux for Web and file serving in the general case, I disagree. Here's why:
These studies do not address price/performance. P/P is one of the most important metrics in making a purchase decision; these studies measured only peak performance. That the prices of the Linux-based and NT configurations tested are not given indicates to me that Microsoft wishes price to be disregarded as a factor in purchasing decisions. To do so would be an irresponsible act for any purchaser. Consider that NT license fees increase dramatically with number of clients, while Linux's price is constant and lower than any NT option.
These studies do not address options such as clustering. Clustering is a common solution to the problem of constant high client load. It may well be a better solution (in P/P and in peak performance terms) than simply boosting processing power with multiple processors. It also has reliability advantages.
These studies are not generalizable to other hardware configurations. While MS will claim that they prove that "NT is faster than Linux" inherently, they do not. The HW configuration was selected for the first Mindcraft study, which has been proven to have been engineered to favor Microsoft. Hence the hardware configuration itself is suspect. An across-the-board comparison on various configurations, with P/P as well as peak performance measured, would be a more reasonable comparison of the virtues of the OSes themselves, and would also highlight particular combinations of HW and SW that are worthy of consideration for purchase.
These studies do not address security. The release version of MS IIS has outstanding security holes, including the recent one disclosed by eEye. This was a root compromise which took eight days for Microsoft to admit, and two more to fix. Microsoft classically avoids the subject of real-world security, preferring the proven-worthless tactic of security by obscurity. Security, of course, is a major consideration to be made in purchasing.
These studies do not address stability. Stability, like P/P, is an important metric for purchase decisions. It helps one determine how expensive a system will be to maintain -- one that requires regular resetting or reconfiguration in order to keep operating will cost in manpower; one which crashes a lot will cost in downtime. Downtime costs money in an enterprise situation, and hence should inform purchase decisions strongly.
These studies do not address changing real-world needs. A real server system is rarely left serving static Web pages forever. When needs change, performance will likely change as well. Building a system to meet a single, narrow-minded need is likely to lead to a dead end in terms of scalability.
These studies demonstrate nothing about the future. Based on past trends, one can expect the situation for Linux-based OSes to get better and better. The next version of Windows NT will likely offer decreased performance on the same hardware (due to increased resource consumption by the OS itself) whereas future versions of Linux will likely improve performance. Buying heavily into Windows NT leads one to platform lock-in which may damage one's ability to escape the expensive effects of bloat.
In short, I do not believe that MS has demonstrated that there are advantages to purchasing an NT system over a Linux-based system for real-world file and Web service. Wise system administrators, IS/IT managers, and CIOs should stick with the proven security responsiveness, stability, price/performance, and scalability of Unix-based systems, possibly including Linux-based systems, rather than betting the farm on the Johnny-come-lately Windows NT.
First thing.... These tests where much better but they still manage to miss the mark.
.htaccess files on commonly accessed files nested five directories deep. Not .htaccess file .htaccess files as possible. This
.htaccess file that is already slowing things down also has
:)
Ok, just in case anyone still thinks these tests are worth a shit. I'd like to clearify that this is pure and unadulterated shit. There now that the
childish remarks are through. I'll do some intelligent speaking.
First off, I don't doubt this to be shit from the get go. I'm an MCSE (my work paid for it) and I know the insane amount of system reasources it
takes to run a NT Server alone. Yes, I know how to properly configure an NT Server right down to the streamlining of the registry. Plus, we have
all been through the multiple restarts and memory that applications won't let go of after using it. Not to mention all the swapping and overhead
processing. Don't get me started with IIS 4.0.
There is a new bug found almost on a daily basis that spells doom for these servers. Plus, IIS 4.0 doesn't have near the amount of features and
configuration possiblities as Apache does. Next Apache needs someone who knows it inside and out to configure it. This is due to Apache's
extreme flexiablity.
Say that average joe smith sets up his Apache server and uses
uncommon with big sites where management is broken up. Well, for every request on the document Apache will check with each
per directory. So if this file is accessed 100 times. Apache will check 500 times for the rights to that file.
Because it will check the root to the next directory to the next. And merge the config files it finds along the way. Making Apache check 5 times
per document requested. But, on the up side if you need infinately specific rights to files. This is a god send that can be reduced by placing
commonly requested documents near the root of the (don't fork the directories too much)server. And using as few
is why you should try to place as much configuration as possible in the gobal configuration files and preferrably in the server configuration file. I'll
explain the last part of that last sentence next.
When Apache is looking into what the rights are for a requested file. It checks certain files in certain orders. And within those files it checks it
against the directives in the order they are placed in the config file. Meaning if that same
the most requested file in the directory near the bottom of the config file. It will take longer. Maybe not whole seconds longer. But, enough on
heavy sites to make an impact.
These are just two of the many configuration tips for Apache a person can pick up when they rtfm (Read The Fucking Manual)and even reading
the source.
And all the rest of the way IIS doesn't have as flexiable a rights system. Nor does it handle dynamic pages as well as Apache. Infact IIS 4.0 will
work fine if it isn't that compilcated a site, the pages are static, and the machine is so big it won't ever see a processor load near 100%.
Apache has that complete control rights system. It handles dynamic pages bueatifully. And doesn't freak when heavy loads hit. It will just keeps
chugging away.
As for file serving? I can't say. I'm not anywhere near an expert at samba. But, I do know that my Linux box boots faster, handles heavier loads
better, and memory management is bueatiful. And to make another remark.
RedHat should not be the version of Linux they are pitting against NT. Sorry, this isn't a direct RedHat sucks type deal. It's a use Slackware or
something and so you can minimize the system to do only what it is suppose to do. And recompile everything to be optimized with the systems
hardware. Maybe not even Slackware. Just something streamlined. Redhat is actually a great system for the home user. That's the way they
seem to be heading nowadays. And I applaud them for it. My it's now easy enough for my mother to use it.
Personally, once again you can look at the source of the tests and wonder why the outcome is the same. These companies are heavily dependent
on Microsoft products. And some have been funded by Microsoft. Mindcraft even did there tests in Microsoft's labs. Of course they aren't going
to say anything bad about Microsoft.
The real test should be here is X amount of dollars. Put, together the best system you can. Linux would kick the fucking shit out of MS. For the
amount of the software alone you could put together a Beawolf cluster that would crush any NT Enterprise 4-way SMP box. I know, I tried this
before when installing many NT systems to upgrade a hospitial. Personally, I won't go there if I'm shot. But, that Linux cluster is up to this very
day without a reboot performing critical storage and access control for CAT scan images. On the other hand the NT clusters (if you can call it
true clustering) are constantly having parts of them rebooted.
Whatever, don't believe this stuff. It's just FUD and the media looking for conflict.
Eros -- I know what every file on my box is there for..... Do you?