Slashdot Mirror
madddog on Linux v NT Benchmarking
BogoMips sent us an interesting tidbit running in Performance Computing currently. Jon "maddog" Hall explains some of the benchmarking issues associates with the DH Brown reports, as well as the ubiquitous Mindcraft tests. Very well written article, IMHO.
Last month, the German Magazine decided to check
the results of the last MindCraf Benchmark.
However, they decided to test performance on
many hardware configurations.
The results were this:
1CPU + 1 NIC --> Linux wins.
Several CPUs + 1 NIX -> Linux wins.
1 CPU + Several NICS -> Linux wins.
Serveral CPUs + Several NICs -> NT Wins.
99% of servers don't use only one NIC (with one or many CPUs).
Conclusion: Linux is faster than NT in 99% of the hardware configurations.
Meanwhile, Linus already told that he is addressing the botleneck caused by the "global lock" in the networking sub-system, that caused this problems in the MindCraft Benchmark.
The solution (Linux 2.4) is expect to the end of the year.
Yes, we are talking about the little free Linux, agains't the expensive proprietary NT giant...
The problem with linux and large enterprise servers is mostly in it's style of development. Most people don't have access to an Ultra Enterprise 10000 ``starfire'' and several disk arrays to just sit around and play with -- they are stuck with (relatively) cheap PC's and Suns and Alphas. Only large companies can afford to shell out millions to buy the equipment, and millions more to pay programmers to develop for it. That's why Sun Enterprise equipment is almost mission-critical-environment ready -- and linux isn't. You can't yank a processor board out of a machine running Linux and still have the thing hum along. Yet.
-- Erich
Slashdot reader since 1997
Of course, most of the "enterprise" settings I've worked in have featured NT with "admins" who couldn't find their ass with both hands. It always amazes me that companies are willing to pay big bucks to people who, whenever a problem comes up, just go running off to a pay-per-incident help 800-number anyway...
----
Every year during my review, I just pray the words "slashdot.org" aren't mentioned.
----
Every year during my review, I just pray the words "slashdot.org" aren't mentioned.
Now, to install almost anything, and have it completely optimized for my system, I go into /usr/ports/, pick the application, and do a "make install distclean" and it's installed seemlessly and optimized for my K6 :-)
I was refering to _every_ level of software, I suppose Apache would be the most obvious example with all the httpd comparisons going on lately. It's not just the kernel you can optimize.
And you don't need to make the changes instantly, you can do make, and then later when your ready (after a backup, and work has stopped for the night), do the "make install."
If your into the Linux "kernel of the week" syndrome, well, optimizing or not doesn't make any differance in how often you reboot, only the speed after the reboot.
If that is true, Intel could make a great marketing move. They could recover a lot of the "Anti-WinTel" fallout where people are moveing to Linux/AMD by open-sourcing thier compiler and helping merge it into egcs. It's like $700 now though? But, if you could make absolutely outragesly fast binaries for Intel CPU's out of normal open source apps, they would gain a lot of support in the Open Source community.
After that, to install, all you need to know is three words, "make install distclean", and even that third word is optional.
It does bring up the point that it should and could be easier. What is really needed is an expantion on the basic "uname" call, to include much more system information (specifically, the CPU, amount of memory, etc..) Then, it would be possable for open souce compilers to just make it one generic flag (like SGI's compiler does, -Ofast) and it would grab the system information for you and figure out the best flags on it's own. That is a realistic possability, and if something like POSIX, the LSB, or some other standardization body would implement this type of system call to get hardware information, it could potentially benifit the UNIX community in a way that Microsoft can't keep up with.
System tuneing is IMPORTANT. Important enought that it can make one OS faster than another. I think we should be pointing out that open source is much more tunable, not only in the ability to modify the code, but also in the ability to optimize it for specific hardware. (quick note on some stuff I tried to see the diffrence, click here)
Why can't someone do some intellegen testing on this, and give credit to the people who REALLY make GNU/GPL and all of open source a success, the folks who write the COMPILERS! Linus did write some nice stuff, as well as many others, but without the right compiler, it's worthless.
If I can do the work of a big enterprise server on four P75's, that's still a savings. Linux is proving that for some tasks, a cluster of small machines works just as well as one huge machine. And there's reliability projects in the works as well based on the same principle: have an extra cheap machine which grabs the load if your main server goes down.
Jon did mention what is needed: the ability for the system to say up and available 24x7 REGARDLESS of disk failures, CPU deaths, and motherboards frying.
Linux is good at the low-end server and desktop role, ans Jon and DH Brown state. However, I don't think your company is willing to run it's General Ledger, Web Server, or other critical systems on one of the Sys Admin's PCs. If they are, you need better Line-Of-Business people to whack upper management around a bit.
Linux is good. It's just not good ENOUGH yet.
This space for rent. Call 1-800-STEAK4U
The one thing about this post which redeems itself as being not a flame is the following line:
/* That's the way it is. Don't like that? Go work towards changing it. Change is good. But till things change, what I say will hold true. */
So this isn't FUD because it's open to the idea that Linux will improve over time.
The fact is Linux is, for lower end machines, excellent. Linux on an Intel against Solaris on a Sparc of comparable speed, for lower end stations wins hands down in ease of administration and (with the porting of Oracle, etc.) gaining rapidly in number of applications.
But until Linux develops a journaling file system (a real one: think AIX, not NT) or more scalability in SMP, clustering and HA (think VMS, not Janus), it will not take over the datacenter, which is where the real money and durability is.
That having been said, I disagree with the poster as to the "inevitablity" of it happening -- I think Linux, *BSD, etc will improve because you can't stop the desire to make free software better. Plus Free Software never will go out of business, by definition.
Give Linux 2-5 years to develop good HA and clustering. Give it 5-7 years more to get a reputation to compete with AIX, UNICOS, and OpenVMS in the datacenter.
Three Step Plan:
1. Take over the world.
2. Get a lot of cookies.
3. Eat the cookies.
I think you'll someday find that Microsoft is feeding the press with data to show Linux's weaknesses. That is how they 'compete'. Unfortunate for them, they aren't 'competing' with IBM, its Linux. Weaknesses will be patched quickly and tested by the community too quickly for Microsoft. They won't be able to wait for a liquid cooled CPU to become the norm so NT v5/2000 can beat Linux in future tests.
Fair benchmarking and reporting is not the norm in this industry. Recently IBMs Warp Server for e-business was hammered on InfoWorld (the link is now broken to the article....). It turns out the guy who wrote the column is the Senior Contributing Editor and Columnist of Windows NT Systems magazine. See "Hatchet Job"
Complaining/exposing a injustice is how we open the eyes of the unknowing. Example: At my stock investment club meeting last night, one member insisted that NT was faster then Linux in all cases and that Linux had a weak GUI. I booted OpenLinux v2.2 on my P120 laptop an he started questioning his beliefs. The rest of the goup was surprised at the polish they saw. All but one member are professionals in the technology industry though mostly embedded/realtime systems.
"Anyone who stands out in the middle of a road looks like roadkill to me." --Linus
Yeah, when it doesn't hang your system. Mandrake is a good example of this.
Mandrake 6.0 has a hdparm line in /etc/rc.d/rc.sysinit, which optimizes HD performance on most systems. However, on some systems, it causes the HD to hang.
By booting without starting up init (linux -b at the LILO prompt), I was able to find and comment out the hdparm line. However, since the system hung during the first boot after install, the RPM database got trashed. I ended up having to completely re-install it again. That was a pain.
Mandrake should not have put in an unnecessary optimization in as a default. Looking at the discussion about this on their mailing list, they didn't seem to care, though.
This is kinda ridiculous. Why is it that corporations are expecting the free(Linux) to be as "enterprise server ready" as the expensive(NT)?
Linux, as far as I know, wasn't designed with that in mind. It was originally Linus' hobby, for crying out loud. Also, it is a very young OS. It is right now good at what its intended use is... squeezing good use out of lower-end, and sometimes antiquated PC's.
I mean, try to make a web/mail server for your corporation with Windows NT on a leftover 486/16MB. Can't do it. But I did that with Linux, and it runs fairly fast for its hardware handicap.
But it has not yet been coded to do what NT is aimed at... taking over the high end server market. Linux developers (whom are gentlemen that deserve our utmost respect) are now beginning to seriously address Linux' lack in this area. I imagine that since Linux is already king in low-end servers, now it will turn its attention towards the higher-end market. Thus mindcraft and company have helped shape the development of Linux in a good way... instead of us madly shouting "unfair benchmarks!", we need to simply begin working towards what NT has already claimed... higher speed on the big iron. Hopefully we can do so without the bloat and instability that has plagued NT, though.
----- if ($anyone_cares) {print "Just Another Perl Newbie"}
warn "Just Another Perl User" if $anyone_cares;
Just be sure to have a more than adequate disk space unless this is just some network-router-like thing. I feel that the linux community has moved away from backwards compatability to just support all sorts of new and really cool stuff and ignores problems with fixing stuff that may not necessarily be the rage is still a problem. Basically I see it like this. NT is worried about getting all these new kick ass features out the door to impress people. Network downtime is expected and most people just nod sagely when the network goes down and think that such an act of god cannot be stopped. They moved away from 486's as soon as win95 appeared (they moved to pentiums remember they don't call the platform wintel for nothing)
Linux then hits the scene and says that they want to FIX the problems that NT left out when it was trying to impress people. I just wish that all these $toonice features would be reserved for people who really want them and not for people who don't want/need them.
A really nice feature that should be allowd in the "real" kernel is e2compr support. I have a really crappy hard drive that is just big enough to get what a normal sane person would need with some small ammount to squeeze out (about 340Mb) this is extremely limiting. I am faced with running a development kernel that isn't part of the standard distro because I have to get it recompiled. Sometimes I don't even have the space for the kernel so I start to get out of date currently I use 2.2.7 with an e2compr patch when the devel version is like 2.3.10 or so (haven't checked in a while)
The death of one man is a tragedy; the death of a million is a statistic --Joseph Stalin
You know, I liked this article. It echoed what I've been saying for ages.
Linux isn't for mission critical large-scale. NT *DEFINITELY* isn't. If you want that, you'd better call your IBM RS/6000 VAR today, because sales are going to jump now.
Linux is not the be all and end all of unix. Period. It never ever will be, as it will more than likely collapse upon itself before we see ext3.
I use Linux. I've used Linux for about 4 or 5 years. I think Linux is great.
But it's not an enterprise OS. Period. Flat out. Never. It's good for small to midrange stuff, sure. Hell, our primary DNS server is Linux, as is our webserver currently. (It *will* be moved to an RS/6000 H70 as traffic increases.) Our secondary DNS server will be Linux. But our network monitoring system will be an RS/6000 43P-140 (aka Model 140) workstation running AIX. Why? Because if hardware starts failing in Linux, I'm screwed. AIX will scream bloody murder before it gets anywhere *near* the point of no return.
Linux has no place in the enterprise as an ERP server. It has no place as your 1,000,000 hit/hour webserver interfacing with SQL and doing dynamic pages. Period. Those of you who find this offensive, kindly contact a proctologist so that you may have your head removed from where it is. That's the way it is. Don't like that? Go work towards changing it. Change is good. But till things change, what I say will hold true.
-RISCy Business | Rabid System Administrator and BOFH
your company here.
shelby != ford
I, for one would like to see a set of benchmark results for Linux that would help a person to make decisions about the configuration of hardware platforms for systems. I would like to see a set of test results for different types of system activity, such as compiling code, raytracing/graphics/visualization, file system access, network bandwidth, combined network/filesystem access etc. This set of measurements could then be run on a variety of hardware types providing the basis of cost/performance decisions in the implementation of systems. One could answer questions like how does a K6/233 compare to a P5/233 ? How does a P5/233 compare to a P6/450 ? How much difference is there between an ULTRA ATA disk system and SCSI? Does the difference change with different processor speeds ? Does 1MB cache make a difference with what I want to do ?
If implementing a cluster, does saving $2K per box make up for the difference ( providing money for more boxes ) if you use ATA and slightly slower CPUs, rather than higher end platforms ?
I would love to see a single ( and evolving ) location for this kind of info. Hell, I would love to work on compiling it. I have seen some sites with benchmark info, but nothing that seems to try to answer specific questions. The sites that I have seen present specific objective numbers but it was hard to derive any context for the differences between systems.
enough is too much
Call me a cynic (after all, I am paranoid for Linux's sake), but MS is probably holding off on moving hotmail.com and homepages.msn.com to Win2000/IIS until the optimal marketing moment sometime during Win2000 rollout.
"See, we've dropped Apache and are moving to our super-reliable, super-scalable Windows2000 with IIS. And we're now making that same power to your company for just $200!*"
--LP
"* Terms and conditions may vary according to the license purchased."
I liked the fact that instead of reading an
article about how messed up and skewed the
benchmark was, I read an article that suggested
things that can be done to better arm ourselves
the next time something like this pops up.
Good Job.
I wonder where all the hunters are today. --Daffy Duck
People like Jon maddog Hall should be in charge of benchmarking. Then the world would be a better place.