Slashdot Mirror
NT vs. Linux - Mindcraft Vindicates Itself
MauricioAP writes "the new benchmarks from mindcraft, or the NT vs. linux, aren't good for Linux, especially for RH guys.
Check this out." Reliability and "bang per $$" aren't addressed by this test or the results might have been quite different. But within the limited parameters, which may or may not accurately reflect real-world conditions, it looks like Mindcraft has been quite fair. (Please read carefully before judging.)
Let's face it, Linux isn't perfect... it's just a lot better on average than the alternatives...
Denny
Police State UK - news and
what a crying shame...
ohwell... maybe next year boys..
yeehaww...
i'd like to see a freebsd comparison actually..
chixdiggit.
-r.
Ok, at the risk of opening myself to flames, I would say that the tests (for once) look valid. Doesn't look good for Linux, and we all know Microsoft is gonna have a blast in the PR dept.
I think it is important to note though that in MS you have NT 4 to pay for, then IIS, and all the rest. So Linux certainly is cheaper, and it's uptime is better. And Linux has better SMP support. Also add in tech support (assuming you outsource for both NT and Linux) and Linux still kinda-sorta comes out on top.
IMHO (strapping on the asbestos) they both have their uses. Like it or not, there are some things that NT is simply better for. And ditto for Linux too. I personally believe Linux is a better server platform despite the Mindcraft tests because of uptime and efficiency.
Just my random unorganized thoughts.
These benchmarks were released on the day of Gates' Comdex Keynote? Coincidence?
Bah, who am I trying to fool?
lets see there is a newer version of redhat. complete with a new glibc contains alot of speedups. there is also a new kernel with a big crapload of smp improvements. this isn't news. mindcraft will never come out with benchmarks showing linux as the victor. These benchmarks don't even say where improvements need to be done since they don't use the newest software. I'm sure the nt server had the newest service pack.
It has been statistically shown that helmets increase the risk of head injury.
I couldn't care less how Linux performs compared to NT, or any other OS. The simple fact of the matter is, I get far more work done in Linux than in Windows. I haven't tried any BSD's yet, purely through lack of time, but I would guess that the same rules would apply. For a programmer/sysadmin such as myself, the benefits of a Unix-like environment just beats the pants off NT. There's no way I'd use NT to do what I have to do. I'd go insane within two weeks...
So Windows NT is a faster file server and marginally faster web-server on single-processor machines. I don't think anyone expected the results to be reversed for the second test. But look at it this way: NT's strength is (currently) in raw performance and that'll take a while for the Free Software community change. But what'll never change about NT is the price while Linux servers continue to improve their performance. Linux is currently able to take a substantial slice out of NT's customer-base, and it's a slice that's getting bigger as Linux-based software develops. What are MS going to do to win back Linux converts, then?
I'm curious as to whether anyone reading (okay, biased readership, but stil...) has actually decided Linux is not the solution for their business, and decided that, all in all, paying for NT is a more cost-effective solution, rather than deciding to go from NT to Linux.
Matthew @ Bytemark Hosting
They wrote:
:(, like they don't know 2.3 is devel ....
"Why didn't Red Hat use a Linux 2.3 kernel in Phase 3?
They told us that it was too unstable for them to be sure of getting it working in the short time we had to run the Open Benchmark
"
Yeah right
The guy who wrote the FAQ really could go into politics someday... He's go at bending the truth.
I would really like to see mindcraft publish some benchmarks on the *BSD operating systems, more specifically FreeBSD. After all, its used on such networks as hotmail, yahoo, and cdrom.com
dox
If they had suddenly told the world that Linux was faster, they would have called themselves liars. They had to come to this result (which is the direct result of non-realistic test results, and probably giving Microsoft enough time for foul tricks (knowing about the test, they probably created a special service pack optimizing for this situation)).
I'm quite sure a Red Hat 6.1 box with an updated kernel (2.3.28 is MUCH better with SMP and also has some very nice TCP/IP improvements over 2.2.x) would do quite a lot better. (Anyone still using 5.2 in real life, by the way?)
This message is provided under the terms outlined at http://www.bero.org/terms.html
Do you think, that the choice of client might make any difference?
BTW, It sees,e that 1processor performance is about the same both for Linux and NT, right?
Where I work we have 4 kinds of computers
hpux
nt
solaris
linux
hpux sucks. Those computers sit there and suck and crash all day. People are begging others to take them
nt. these make good workstations and ok file servers. they don't suck as bad as the hpux's and can actually do some work
solaris. these are the big babies. they sit there and work all day. these guys do the big shit. webservers firewalls and other bigshit
linux. these sit around similar to the solaris and do everything. weather they are work stations or dhcp or file servers. when you don't need anything to high end.
bottom line is that you don't put nt on a 4way smp box. you put solaris on that.
It has been statistically shown that helmets increase the risk of head injury.
Note the date at the top of the referenced page - June 30, 99. (Which explains why they are using old builds of Linux and old NT service packs.)
-Blake (who didn't realize the Linux crowd hadn't already looked at this updated benchmark
The document is from june 30th and has been discussed at slashdot before. I really don't understand how this could pass through as a slashdot story, or why people suddenly 4 months after feel a need to discuss this as if it was a new thing... Let's kill this thread now shall we :-)
i seem to recall quite a few disparaging remarks directed against mindcraft and others.
/.
perhaps an formal apology is in order. oh, wait --never mind; this is
The one thing I couldn't determine was whether they were striping the network traffic across the cards on the NT based systems.
:-)
Since these machines look network bound this just might make a difference
Conclusions
Mindcraft's credibility and reputation have been vindicated.
While dont disagree with the results, I think the above conclusion might have been the one they were aimming for.
There is nothing new about these tests. Yes, it shows once again that NT beats Linux at serving static pages over a 100 Mbit connection. Again, dynamic web pages are not included in the test. Do I have to say how unrealistic this is? Most major websites serve pages with active content these days and very few af them have a 100 Mbit connection.
C't magazine has so far posted the most realistic comparison so far. I agree that work has to be done on the TCP/IP stack and kernel locks in the linux kernel but I am convinced that these issues will be resolved soon. Then microsoft will go and find another situation in which Linux performs worse than NT and will focus their PR machine on that.
Note, however, that the tested kernel (2.2.6) is one prior to the single-threaded-TCP fix. I would like to see these tests done with a more recent kernel.
Again, we must concede that on unrealistically high loads, in an unrealistic test scenario, a professionally tuned very-high-end PC with 4 CPU will outperform an older Linux kernel.
However (sorry Microsoft), that doesn't matter to me. What is also important is reliability, maintainability, cost, support, standards-compliance, and a host of other things. For me, Linux still beats NT when all these factors are considered. Also, if I wanted a very high-end SMP box for web serving, I'd probably choose Solaris anyway. Microsoft, you're barking up the wrong tree. Let's see this test repeated, but compare NT with a Sun UE450 next time.
--
--
The early bird catches the worm. The worm that sleeps late lives to see another day.
I can't imagine anyone, even a lunatic, would ever think that MS's products are the best. What will 2000 do? Crash. The code base for windows is an ugly mess full of kludges. That is why they continuously push the release dates back. Windows 98 was originally windows 97. Linux on the other hand has a much more easily manageable code base. Any advantage NT has will be short lived at best. I'll bet you own stock in Microsoft, right?
Muslim community leaders warn of backlash from tomorrow morning's terrorist attack.
Mindcraft:
The major performance problems are with the TCP stack, which is single threaded in the 2.2.x Linux kernels, and with large-grained kernel locks that degrade multiprocessor performance. The Linux community is addressing these performance problems and others in their 2.3.x kernel series.
Well, I'm glad that they recognise that work is being done on this. It is very much the case that Linux does have SMP scalability problems, and I think we all knew that prior to this report.
Regardless, I still stand by the old motto, there are lies, damned lies and statistics, run Linux, run BSD, run NT, do what you will, but be sure to be happy, with what you run. I would like to see how NT fares against Linux & BSD in the real world, how about this test:
The test will last for one year.
The machines will be under constant varying Web & File serving Load.
The NT box will also run a 16-bit application.
I think we all know what's going to happen over time here...
You can't test NT performance over 15minutes of file/web-serving, NT may have only leaked 15Mbs of the available 1Gb in that time.
OK, I don't know whether they tested for 15minutes or not, but I did look and cannot find anything regarding the duration of the tests. Can someone please comment on this?
Ok, so we get another look at Linux (which has been developed primarily on single processor machines and started as a test system) compared to NT (developed by a company with effectively infinite hardware resources for its developers) and really expect the SMP scaling to compare?
Let's remember that the 2.2 kernels were just the start of SMP for Linux where NT was written for SMP from the beginning. Ok, so Linux lost this round? So what?
These tests are good for the industry (both Linux and NT) - they show each where they need to improve. Linux needs improvement. It will improve. Lets run the benchmarks every 6 months and watch the way things develop.
John Wiltshire
Fear: When you see B8 00 4C CD 21 and know what it means
You're right, anybody can make an NT boot, but only a few understand the security model thoroughly. Even most MCSE's don't, so your choice here is:
1. Buy NT, get somebody who understands it, and can see through it's obscure "happy, happy" colors GUI, pay a truckload for this.
2. Download a linux dist, get a geek who lives for it, and understands it.
Fact is: NT's GUI only serves as a way of making you believe you can administer it properly, while you can't.
-- Andreas
It wouldn't have been fair to use a 2.3.x kernel -it is accepted, even by the Linux community that:
a) 2.3.x kernels are development kernels and may be unstable
b) 2.3.x kernels are essentially beta software releases and not mainstream releases - the equivalent competitor to 2.3.x is whatever alpha/beta software Microsoft happen to have got for NT5 whenever [if] it surfaces
As a Linux user I have to concede that it looks as though Mindcraft have made every effort to be fair in this test.
So the questions are,
* What can or is being done to [safely] jack up the performance of Linux ?
* Did the test identify any specific bottlenecks ?
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
Everything looks fine to me too. All I can say is that I am disappointed at Linux's performance. Still won't stop me using it though.
However, I do have a question that maybe someone here could answer - if in testing NT proves to be faster than Linux, why then in the real world does Linux always feel faster? Web sites that run on Linux/Apache always seems more efficient and seem to load faster than ones on NT/IIS, but the tests here show otherwise!
Don't get me wrong - I'm not saying the tests are fixed. If there were Redhat Engineers there doing the tuning (and I'm sure if they weren't really there we would have heard about it), I certainly can't say that the tests were fixed. But the tests certainly don't seem to reflect what I seem to witness in the real world? Maybe, just coincindently, the NT/IIS servers that I connect to happen to have lower bandwitdh than those with Linux...!
Anyway, here is an example:
NT/IIS: http://www.dvdexpress.com
- this is one of the site where they have multiple servers to handle the load. I think they go from www1 to www9, maybe higher. And it always seems slow...
Linux/Apache: http://slashdot.org
- this, AFAIK, runs on 1 web server (I think the config is 1 web server, and 1 oracle server). Correct me if I'm wrong. However, it definately isn't 9 or 10 webservers. And response time is always good.
Granted, the back end on both is completely different - DVDExpress runs on SQLServer, and slashdot on Oracle. But there is still a noticeable difference.
Anyone care to comment on this? Why does the real world never reflect 'scientific' testing?
T.
Benchmarks were made on low-latency network and with four network interfaces. In reality HTTP server can get load this high only with very high latency of the network -- clients simply can't be that close to the server by the network topology -- backbones cause huge delays. I doubt that with high latency network (can be simulated in laboratory) and single gigabit interface instead of four 100Mbit/s the results will be the same.
Contrary to the popular belief, there indeed is no God.
Reminds me of an old story where they set up a contest between a hand shearer (of sheep) against the new-fangled electric shears. The manual champion won and everyone thought the electric shears was going to be tossed .... then the electric shearer got another few pounds of wool off the hand-shorn sheep, ie less wastage == more profits.
While people might not think a few percentage makes a lot of difference, it should be pointed out that in high volume businesses, companies like Wal-Mart sustain a long-term competitive advantage over their peers by adopting a pervasive mindset to control their costs. While Linux may not be a gas-guzzling speed champion on pre-slected race-grounds, the lack of restrictive licenses (operational cost less dependent on #connections) and the ability to control your own environment (ie upgrade at your own pace) offer value in other ways. These savings would add up when hosting very large web farms.
Different horses for different courses.
LL
Tests for 1 processor systems with NT clients are to conspicously missing. This, conincidentally, is where Linux should beat NT.
Mindcraft hasn't made up their results, but there's no need to when you select your benchmarks carefully.
In light of the Mindcraft results maybe this story should have had a different logo
A skinned Tux being eaten by Bill Gates perhaps ?
:-)
P.S. Do not regard this as Flamebait - I use Linux ! Honest !
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
We've got a K6-233 setup with NT server and a 386-40 setup with linux as an internet gateway. I chose NT for the server mainly because linux doesn't have any support for netbeui. With NT I can turn off netbios over tcp/ip and use netbeui for the local file sharing. This way if someone were ever to actually break into our gateway from the outside, they wouldn't be able to get to any of our business information. Also one of the programs we use is dos based and has a server component. I don't have the time to play around with trying to get dosemu to run stably with networking support just for the thrill of running linux on the server. NT works and does what we need it to without any real drawbacks. Unlike many of my friends I'm not a linux or open source zealot. I prefer linux over any other operating system, but I'm not foolish enough to think its the best in every way and in ever situation. If it were NT wouldn't even be on the map. I can't comment on whether NT is faster on our fileserver than linux would be, but I can say with certainty that linux is faster on the 386-40 than NT would be, especially since its running off 8 megs of ram. Not exactly a supercomputer but it does its job of ip masquerading fast enough to deal with our 56k connection in real time which is all you can ask of any computer doing that job.
I like NT to tell the truth. As a simple fileserver it does a good job in my experience. But it has flaws in its stability and security. I wouldn't use it someplace where you had to really rely on it. I'd use something else, maybe linux, maybe not. I would of course depend on the task and which tool was the best solution.
Muslim community leaders warn of backlash from tomorrow morning's terrorist attack.
So you're saying that RH6.1 is a factor of two faster than RH6.0?
Come on.
Sure there is currently a better version of about everything. But, given the time scale upon which linux evolves, there will ALWAYS be new versions of components by the time a place like Mindcraft finishes writing up a detailed white paper about what they did.
If those updates rectify the factor of two the benchmarks see, then bitch for a rematch. Until then, accept reality, and work on making the next version better.
Has anyone else considered setting up their own machine to try out some different benchmarks? Admittedly, fileserving and webserving are the two main apps you want a server to perform, but what's the reason we all use Linux at home and NT at work?
Surely there are some more real world comparisons we can make to push the point that Linux is a more usable, sturdy and fun platform to work with.
The web is a great leveller, so why don't we start putting up our own "official" (!) pages detailing where linux beats NT hands down. Then we can really put the willies up MS.
Why the hell are we discussing a benchmark ran on a hardware config designed especially for NT:
1. MindCraft once again used a quad ether (but skipped anouncing it) and the infamous "EtherStripping" break your switch stuff.
2. Mindcraft once again used the Dell machine which has a RAID running better under NT than under Linux
The benchmark is faulty by design:
1. If you want these speeds you use a Gig Ether on the server in full duplex mode not a questionable technique that actually breaks lots of real networks.
2. If you want real OS becnhmarking you use an architecture that is equivalently supported by bothe OSes.
Overall:
I have tested Linux with GigE (it can almost pull physical speed on machines much cheaper than the Mindcraft Dell monster) and NT has been officially tested by most GigE manufacturers. The results used to be available at the packetengines site butit looks like they were dropped when moving the site to alcatel. Anybody a link please? I would not quote them so nobody blames me for flamebaiting...
It will be rather interesting if someone finally does this benchmark on a sanely designed network (no etherstripping BS) and with proper hardware.
To conclude I expected better from RH than accepting a doomed bench (on hardware and in a network setup where they cannot win).
Baker's Law: Misery no longer loves company. Nowadays it insists on it
http://www.sigsegv.cx/
I find this tit-tat issue funny. DOES IT REALLY MATTER? For example while they mention that the Zeus server has the same problem, at least there is a choice on Linux.
NT is a good OS. And yes IIS is a good Web Server. But there is NO CHOICE!!! And that to me is a bigger problem.
You see problems can be fixed in both Linux and in Apache. But what happens if there is a problem in NT and IIS? Can I switch Web Servers? Not easily. Can I fix IIS? Not at all.
"You can't make a race horse of a pig"
"No," said Samuel, "but you can make very fast pig"
NT performing better than Linux, I guess I can accept the results. But in what sense are the benmarks a Solaris-level comparison. Don't Microsoft try to compare Windows NT and Linux on points where useually Solaris is the only winner? OK NT performs better than Linux, does everybody need that performance? Just some little doubts I have
Bizar technology?
The speed of Linux development renders most of these sort of contests invalid before they hit the street in any case. Anyone with a modicum of coding knowledge could tune the Linux TCP stack and SMP threading to smoke NT in those trials (and much has been done since 5.2 toward that end). You just cannot say that about NT (and not be a flaming liar, anyway).
So E is relatively prime to (P-1)(Q-1)... Odd, that.
Obviously Microsoft is the one who is behind all this. They have their programmers and developers sit down and come up with a benchmark spec that they believe they can tweak NT into performing well on. At the same time they try to find areas where Linux is not as strong. After a few months of coding we get service packs for NT and mindcraft conducts its "independent" study. Of course NT comes out ahead, big suprise.
It's the same kind of thing that Apple does when it compares the toys they sell with PC's. What I really like are the photoshop benchmarks where the Mac is so far ahead. What they don't ever bother to tell you is that apple long ago made changes to their OS and put in system calls specifically for Adobe Photoshop. Then there are the straight CPU benchmarks where they take the few instructions from the powerpc that are significantly faster than an equivalent on x86 and say that the processor is faster overall by this factor. Some powerpc instructions are slower, but then they never tell you that. It's the same thing here. Rather than get bent out of shape we should spend our time doing an honest analysis of both platforms and outcoding the sons of bitches.
Muslim community leaders warn of backlash from tomorrow morning's terrorist attack.
According to Mindcraft's tests, the bottleneck was the Kernel. In particular, the TCP/IP stack is single threaded.
". Download a linux dist, get a geek who lives for it, and understands it. "
Sadly, this is _not_ a recipe for success. A geek who knows all the command flags for ls by heart and prides himself on being up to date with _all_ the latest bind vulnerabilities is not your ideal sysadmin. You need someone who sees the wood as well as the trees, and administrator who can think strategically as well as perform competant operational tasks.
In this light, you realise that a good sysadmin is not someone who understands an OS thoroughly. It is someone who understands the aims of you IT systems thoroughly, and knows how to implement those aims properly. There's a world of difference.
That said, yes, I think the TCO of *nix is generally lower once you are talking about large installations and Enterprises. For smaller organisations I'm not at all sure that is true. A 20 person company with a need for a file and print server is perfectly suited to an NT box.
-----
This sounds a lot like all the things people said the last time: "Wait until feature X is ready in Linux".
Face it: The current version of Linux is tested against the current version of NT. Reading the article, it seems that enough people were there to tune everything on the Linux side, so just believe it: NT is better in some things than Linux. And surely, we can think of other circumstances where Linux or another OS is better than NT.
Linux is not the answer to every question.
-- Nothing is as subjective as reality --
Linux is updated on a daily basis in many ways, and using a newer distribution with new glibc, newer apache, samba, kernel, etc. Could be a little more realistic. Granted NT might still be faster but who's making more improvements faster?
Show how fast linux is catching up by using updated software and do another test today!
I was wondering if anyone has the hardware available
to perform the same test on FreeBSD? I suspect that
if the test was (MS's fastest OS for the job) vs
(The fastest free OS for the job), MS might well lose.
perl -e 'fork||print for split//,"hahahaha"'
From what I can tell, c't used fork()'ed CGIs for their comparison against NT which is unrealistic. No high volume site will rely on out-of-process CGI's any longer. All they proved was that Linux fork() is faster than NT's CreateProcess(). But NT excels at threads, not processes. The real test needs to be mod_perl vs ActiveState's PerlScript on NT.
Someone needs to perform a standard transaction-oriented dynamic web test, for instance, maybe a simulation of a auction web site, with simulated buyers and sellers.
We could then benchmark mod_perl vs php3 vs servlets vs ASP vs aolserver vs LiveWire and see which languages and which platforms come out on top.
Alot of people are expecting Linux to win on the dynamic version of such a test, but I wouldn't be so sure. IIS does a lot of unsafe things to boost performance, and I bet its ASP execution path is faster than Apache's (although not as safe/stable)
NT, for all its flaws, has a very efficient threading and asynchronous I/O mechanism which helps it scale very efficiently for some scenarios. Still, I'd love to see a dynamic benchmark.
This is bad news for Linux, but not for the reasons noted in previous posts. :) to keep NT in the minds of those who make software policy in business. And unless RedHat starts investing some of that whacking great IPO in advertising (haven't seen _any_ in the UK), to get the message across, then this sort of thing is the only press which people see Red Hat in. And that will be bad for Red Hat and Linux in general.
Face it, how many of you fellow nerds out there are regularly consulted on HW/SW choices? And even if you are consulted, you will probably be overruled. Like it or not, your PHB controls the purse strings. And (s)he won't be in a position to explain why "free" is better than "corporate standard" to the Directors' Board, until more people understand what Linux is. Most non-nerds don't give a toss about what their servers are, just as long as they keep files & send mail, and let them look at web sites. This is what we need to focus on.
OK, NT beat Linux in a staged test. That's fine, gives the kernel dev team somewhere to aim their efforts. But for the non-nerds, all they take away from this is "NT is better than Linux". I'm not going to go into whether the test was fair or real world representative, just that people see NT beating Linux.
The problem here is the PR, getting the users & PHB's clued up a little on what goes on when they click their mouse buttons. The MS juggernaut has all of the tricks (and has been caught for a few of them
Strong data typing is for those with weak minds.
the page says they used rh 5.2
Jilles
Don't forget you need 3 times as many NT admins to keep X number of NT's running as unix boxes.
I get these figures from my last three employers where the comparison could be made.
Roblimo is right. This test is not about "bang for bucks" it's about sheer performance. The tests speak for themselves. NT beat a RedHat Linux installation fair and square.
The important question here is not "Are Microsoft behind this?" but "What can we learn from this?". All the kernal hackers and module developers need to read this very carefully and work out a stratergie for development on a global scale. Were does Linux progress from here? What should be the main focus for development in the future? What can be done to overcome these test results?
We all have the answers to these questions but do we all know where to send them?
Mindcraft seem to be saying that because they've rebenchmarked everything "fairly", their previous benchmark is vindicated. Doh? I'm sorry, Mindcraft, but your previous benchmark is just as completely rubbish as it always was regardless of all the spin put into this new one.
An important thing to remember is that a statistic is a representation of exactly the data you collect - in this case, it's representative of what you get when you benchmark an old Linux kernel against the latest NT server with hardware that Linux is known to not get the best out of (4 processors). The good thing is that now these results are known, something can be done about it.
I would enjoy Red Hat asking Mindcraft to rerun the benchmarks when these problems are addressed in the Linux kernel.
I'm just a rank beginner with linux, but have been a WinNT programmer for some years now, and have been long told that Microsoft's C compiler has a way better optimizer than GCC does. Is this true, and if so, how large an effect do you think this will have on the performance of the kernel and the modules tested (apache, samba, etc) Thanks.
Alas gallinaceas de urbe bovis volo
Some people may flame me saying, "You don't care anymore because Linux is losing." Wrong answer.
Here's why: I LIKE LINUX
I genuinely like it. Yeah, so these benchmarks say it is not as fast or as "good." What is good anyway? Good to me is: reliability, configurability, usability, extensibility, scalibility, inexpensiveness, fun, etc. Linux shines in all of these areas and more. Yes, that's right, I said Linux is FUN. Linux is plainly more fun to use. I don't care what any some benchmarks say. We all know that benchmarks are unrealistic. They don't test "real world" conditions and situations. I think we should use their criticisms (only if valid, of course) to help Linux be a better operating system, not to beat some other OS.
----------------
"Great spirits have always encountered violent opposition from mediocre minds." - Albert Einstein
Co-founder and designer at Music Nearby: http://musicnearby.com
I'm sorry, but that post is just so wrong it is laughable. If you had found some site that ran NT and was faster than Slashdot (not hard to do) you would be flamed out of existance.
Where to start?
Slashdot does use multiple webserver - it caches static pages, and
Slashdot does not use Oracle it uses MySQL. Big difference in websites.
"The response time is always good" ????? Not from where I am (Australia) it isn't. Subjectivly, dvdexpress seemed faster to me. Anyway, what does that prove? You are closer to Slashdot than dvdexpress? Slashdot has more bandwidth?
Dvd is graphic intensive, and takes longer to render in Netscape, too.
You can't compare two totally dissimilar sites, on totally different hardware.
I bet I can find apache sites that seem slower than NT/IIS sites. EG: www.Apache.org always seems very slow to me. What does that prove? NOTHING!!!!
Look, I want Linux to be faster than NT as much as anyone, but we can't even be seen trying to spread FUD like MS does. Imagine if MS stuck that up at Comdex as by "a Linux Hacker, posting on the Linux nerd site slashdot.org".
People, please think for a moment before you post, and before you moderate comments like that up. Ask yourself this:
Reader of Slashdot don't need to see arguments for Linux like this, we need to see the opposing view, so we can learn what we need to improve.
Damn.. I just know this will kill my karma, but that is crazy!
--Donate food by clicking: www.thehungersite.com
That said, yes, I think the TCO of *nix is generally lower once you are talking about large installations and Enterprises. For smaller organisations I'm not at all sure that is true. A 20 person company with a need for a file and print server is perfectly suited to an NT box.
I work in a small company. I set up the network there among our four machines (I got duped into having the file server on my machine, too). When it comes down to it, I don't have time to set up any security or strategy, because it isn't my job - I'm a programmer. I can't talk them into letting me devote a day or so to set things up 'right'. So my boss (who is great but not that intelligent about computers) runs executable files attached to e-mails send from friends all day. I know it isn't going to be long until he wipes out my computer and all the other ones on the network, but my hands are tied.
In the situation where you have a limited number of computers, are running NT anyways, and/or cannot get someone to commit to actually implementing a strategy for security, NT is going to win, because it is designed so that morons (like myself) can point and click repeatedly until you have a server set up on the network. But the company across the street from me (now up to four employees!) has a linux box as their server, because it is just plain better and faster, more stable and more secure. You just got to evaluate the need-vs-time-investment-vs-hardware-investment graph, and choose.
I am surprised so many people haven't realised there is no such thing as a non-biased benchmark, and that, shock, horror, Linux is perfect (yet).
Benchmarks must reduce the scope of tests and make assumptions, which are not always true, so as to be possible. They also need to be done at a point in time, and not wait 'for the next version, which is so much better'. Doug Ledford of RedHat was there for the tests and has his spin on the tests, where he talks about the difficulty of getting a meaningful benchmark. The Tranaction Processing Council are continually revising their benchmarks to remain meaningful. The big guns, IBM, Sun, HP, Oracle, Sybase, Compaq and Microsoft all use different TPC benchmarks to try and gain ammunition for sales staff. At some point Linux people will need to do the same.
The Mindcraft benchmarks look to be as fair as any I've seen. The reaction to the benchmarks is far more informative than the results themselves.
Linux can still be improved, it isn't as strong as other operating systems in some areas. The fact there is development occuring proves this point.
If you don't like the results, find a benchmark and configuration that gets the results you do like! Where there is a real deficiency lend a hand and be part of the solution.
First of all, as far as I know almost every major company has a habit of cheating in benchmark tests. For example video card drivers detect that a test is being run and enable code that skips most of the drawing primitives. This is easy to do in code that is not open source since it would take a major effort to reverse engineer the device drivers. It might be possible that NT has a feature that detects different kinds of tests and optimizes its performance accordingly (if you are for example testing throughput you would trade throughput for latency times). While this is not cheating in usual sense I think that this would be quite useless in normal mixed load situations.
The second thing is that Microsoft is quite a large company. If it wants to outperform Linux then all it needs to do is install Linux, tune it to its limits and then analyze its performance and find out weak points. Then it makes the same thing with NT. After that it just puts hundred well paint workers to make NT faster than Linux. This is made easier by the fact that if Linux works faster than NT they can just look at sources and figure out what Linux is doing better than NT. Also, it is possible that Microsoft would look at the weak points in Linux and would publish only those benchmarks where Linux performs significantly worser than NT. Anybody who does those same benchmarks would get similar results and the original benchmarks would be considered objective.
Third thing is that those benchmarks might only test peak performance - performance under high load. It is also possible that the structure of the load is untypical. This is true with most benchmarks; they rarely test systems under realistic conditions. Since I have not looked at those benchmark programs I do not know if this is the case. Anyway, peak performance is important if you want to identify bottlenecks and see what are the limits of programs. Peak performance does not tell how programs work under normal every day use.
Last thing is that I think those benchmarks are already outdated. What I would be more interested would be performance of cutting edge Linux system against similar NT system.
As a conclusion I again state that I think those benchmarks look valid. It seems that Linux kernel (and possibly also Apache) still has bottlenecks in its performance. I'm not sure if those have been fixed since this benchmark. However, I think that this benchmark should be thought of as a challenge to improve the performance of Linux. I actually think that Linux did quite well; performance differences are not THAT large when you take into account my comments above.
I have read on www.benews.com that there is an effort to port apache to BeOS. As you know, BeOS is pervasively multithreaded, which means it will use multiple processors much better than even NT. Now, Apache 2.0 is a version of Apache utilizes this feature and, therefore, rund much faster. There is an alpha version of Apache 2.0. I guess once Linux kernel gets the pervasively multithreadedness, apache 2.0 will run on it, too. But I wonder how much of a re-engineerig effort this is for Linus?
BTW, there is a Apache 1.3.9 port for BeOS; but the 2.0 is the thing to go for, for BeOS.
here is info on Apache for BeOS
So, the really cool hting would be for Linux kernel to have pervasive multithreading.
Sigged!
Though the PCWeek tests favour NT (for reasons well covered elsewhere), they do not do so by the ludicrous margin the original tests gave, and the Linux's community's cries of "foul" were entirely just and accurate: these tests show that Mindcraft did indeed load the die.
Furthermore, Weiner has never managed to justify the claim that he had asked for help in "several Linux discussion groups" when setting up the first test: searches show that he only posted *one* article, and that was met with requests for clarification that was never forthcoming. So as it stands we're quite justified in believing that Weiner is a flat-out liar on top of his other sins. That's not vindication.
--
Xenu loves you!
The real problem with the Mindcraft benchmark has nothing to do with most of what they cited: the graphs are painfully clear that the limited resource is network bandwidth. That's why it's so funny when they say "We'd never test a server that's resource-limited. What's the point?" That's what I'd ask them now.
Note that they test with one and with four processors, but do not test with one or two ethernet cards. In fact, they never mention the complete hardware configuration of the machine, so we just have to assume they used the same f*cked-up four ethernet card configuration.
There were actually benchmarks put out by c't explaining this, with graphs, and real tasks. Linux performance generally did much better until that second ethernet card was added. I'll believe them, that it's a software limitation in the TCP stack, but I'll also believe that they were exploiting a known problem in the Linux kernel--that only happens under these strange conditions--to their ends. Until they show some benchmarks with the ethernet cards mentioned as a factor.
NT vs. Linux Server Benchmarks: informative and interesting, but most of all truthful, with a link to the c't article I mentioned, and many other more realistic benchmarks.
---
pb Reply rather than vaguely moderate me.
pb Reply or e-mail; don't vaguely moderate.
You moved your mouse. Please restart the computer to complete the changes.
What I'm saying here is this. You can't use NT in a 24/7 environment unless you want to spend twice as much for a spare server. You make a simple change on an NT server and you have to reboot. No so good for the enterprise...
webmaster: http://amazing.divingdeals.com
--- Think of it as evolution in action ---
Yes - but for the "phase III" test, where the RH guys were allowed to put in all their latest updates, the end result probobly looked a lot like the soon-to-be released RH6.0.
Also note that their updating produced at best a 14% improvement over the phase II system. So, it would be fair to assume a similar increase in the next round of evolution.
Certainly things would be even better (but good enough to win??) if they'd used 2.3.x kernel SMP stuff - but that's not a fair test, as the systems companies might be using as web servers will seldom be beta.
The was a quite unbiased benchmark done by the German c't magazine (unfortunately, it seems they removed that article from their online archive) that showed Linux and NT having pretty much the same performance in static webserving, NT losing disastrously in Perl-based CGI and Linux losing disastrously when serving more than one NICs. Note that this last test is very unrealistic, and Mindcraft basically repeated this with their new benchmark. OK, we know now that Linux doesnt handle more than one NIC well. Big thing.
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
I'm disappointed by the number of posts here that show a stubborn insistence of "Linux is better, NT sucks, we don't care what those Mindcraft people say." Let's just face the facts, people. Linux did not perform so well in this test. But instead of wasting our time and energy to criticize NT/MS/Mindcraft, why not we do something to improve Linux?? I don't know about you, but for me, the fact that Linux is being criticized is a good sign. Why? Because it shows there is room for improvement, and that we can still make Linux better. Instead of moaning and whining over it and spinning conspiracy theories against Mindcraft/MS, let's just take advantage of this criticism and improve Linux to be a better piece of software.
Remember, "if it can't kill you, it'll make you stronger."
mikre he sophia he tou Mikrosophou.
actually, that article is still up, someone else posted the link: http://www.heise.de/ct/english/99/13/18 6-1/.
The illegal we do immediately. The unconstitutional takes a little longer.
--Henry Kissinger
I can't believe anyone would make this assertion. While I agree completely with the other half of your statement, that a good sysadmin must understand the aims of yout IT systems and know how to implement them properly, I would say that thoroughly understanding an OS is a vital prerequsite for that second clause.
Anyone who has worked with an operating system, a programming language, heck, a make and model of car, knows that there are essentially four levels of competence. First, complete incompetence. You have no knowledge, you try things and you screw things up. Second, basic competence. You have some knowledge. You successfully carry out basic tasks. You use the system without damage, but there are vast areas about which you know nothing. Third, competence. At this level you know your way around. You know how things work. You know what all the parts do. Fourth, high competence (guru). You not only know how things work, you know why. You develop a holistic sense of the system/language/automobile. You can imagine how things work. You can be presented with an unfamiliar situation and you can figure out what to do about it.
Most people with whom I have worked in IT (and I've been working professionally as either a system admin or a programmer/analyst for over 12 years now) are at what I would call level 3, and a fair number are at level 4. Thorough knowledge of a system is required to be at level 4.
The notion that one does not require deep knowledge of systems to be a systems administrator is tenable only in a system with nothing ever happens that is outside the training materials. No such system exists.
If you are arguing that deep knowledge of a system is not required to be a sysadmin, then I sure don't want to work at your company. If, OTOH, you are arguing that deep knowledge of a system is not in itself sufficient to be a good sysadmin, well, then I've been wasting your time and I apologize, because I agree with that...
If you were going all-out for performance on a Linux web server, why would you choose Apache?
Wouldn't Zeus or thttpd be a better alternative?
Doesn't Roxen outpace Apache?
Don't get me wrong. Apache provides a level of flexibility and power that few web servers can match. But this test was about performance, not the real-world advantages of Apache. (URL rewriting totally rules, for example).
So why did Red Hat choose Apache?
--
Peter
Wake up people! The whole point of OS, FS, you name it, is that nobody's got you by the balls, as we know of certain companies too fond of squeezing... Performance is important, but it's not first on my list, as it comes after freedom, robustness, security, and the what-can-I-do-with-it factor, at least.
Just my take,
JM
I had a friend in high school who's parents bought him a brand new Ford Mustang. It outran my poor Honda Civic, which was about 8 years old, even though my car was regularly maintainance by certified Honda technicians. I'm absolutely positive that if I had been driving a new Honda of any model that I still would not be able to keep up with the Mustang as far as raw speed is concerned.
But as far as total cost of ownership, the Mustang loses by a long shot. My annalogy is a bit flawed however. Linux is becoming more of a Corvette at a Honda Civic price each and every day. Bah to those that think commercial software developes faster or more efficiently.
One other point, the white papers are dated June 30, 1999. That means that the test was run how long ago? All this means is that the latest version of NT outperforms old Linux technology. I'm happy that these tests are on the up and up, but I see absolutely no reason to be impressed by NT.
My $0.02 worth.
Windows NT Server 4.0 beat out NetWare 5, Solaris 2.6, and Linux (where is BSD?)
http://www.mindcraft.com/whitepapers/index.html
Mindcrust benched them, so it MUST be true. Yeah, right. Does anyone else find this is a little suspicious that NT beats all 3 platforms?
Actually, I would say that your arguement is good against the tests. In the real world, the main thing that causes differences in speed is bandwidth. Not processors. If you can scale to the max of the bandwidth, anything else is just extra. As an ex-Girl Friend use to tell me, "more than a handful is a waste" ;^) More than a bandwidth is too a waste. It seems to me from the tests, that the Apache/Samba/Linux can handle the majority of bandwidths that are out there.
So I guess that until there is high speed optical switches everywhere over the Internet, we can stick with Apache.
Second note, Where are the tests on Dynamic Web pages? Although I read the Mindcraft White Paper quickly and might have missed a statement that they did do it.
Anyhoo, this was good to let the Linux Kernel hackers work out the bottle necks. I can't wait till "Mindcraft III", where we get to test the 2.4 kernel.
Third Note: When are we going to use something other than that damn Dell? Isn't that machine made for NT. Although there is problems with the Linux SMP (soon to be solved), this machine I once heard (don't know if this is true though) is tuned for NT servers. Lets run this test on another High speed machine. Or is this the only server that NT runs on?
Steven Rostedt
Steven Rostedt
-- Nevermind
There's a long road from "point and click" and thinking you've setup a system to "point and click" to setup a system. And most NT admins I know/have heard of don't know how to do the latter.
-- Andreas
.. as all SW included is compiled with bad optimization (and feature-sets) to start with.
;-) ... noone gonna use. And at the same time some vital utilities/docs are missing.
Next: there are plenty of stuff included that almost noone (I would've say just "noone" but there's always a guy complaining
EG: It could be tricky for newbie (and even not-so-absolute-newbie) to figure out *proper* hdparm parameters for given configuration. Not to say that default RH's doesn't understand UDMA things at all..
Say: on my old(er) PC without UDMA capable drives -u1 crashed filesystems on both of drives (on the same channel).
At the same time -u1 is vital on my new(er) hardware.
In fact -u1 saves me from seeing smth like:
... blah-blah error (sector not found)....
... disabling DMA mode....
on regular basis.
Who knew? Took me a lot of time and russian fido archives searching to figure this out.
It's a pity that half the posts here are stil I-can't-and-won't-believe-it posts. Face it, in artificial ultra-high performace benchmarking, NT is just faster than Linux. Hopefully it'll change soon, but for now that's it. I'm not happy about it, but it won't stop me from using it in the real world, because it just works better.
So please just stop the same whining we've been hearing since the first Mindcraft benchmark, and go on with your happy Linux lives!
-JF
Well, now it's accepted that NT is faster than linux in 15 minute spurts using insane loads for a few specific applications. Big deal. It's still BUTT UGLY!
See you, space cowboy...
Yes, somebody on Slashdot commented (after Mindcraft I) that the RAID controller used had very poor Linux support, due to the company not willing to release its driver specs. Mylex cards was recommended instead (just look at the Mylex README files in the kernel source -- these guys ARE nice). I can't remember anything being said from Mindcraft about this, though, and it certainly is an important part of the benchmark.
That being said, I think the parts about "the Linux community is working on these issues" tremendously adds to Mindcraft's credibility.
/* Steinar */
(This comment is of course GPLed.)
The point the first poster was making, I think, is that by using a 4-way box, and crowing about their advantage on ludicrously high server loads, MS is aiming at that market. That is, once you step into the realm of 4 processor machines, testing NT vs. Linux is just silly, because who in their right mind would use either one for such hardware? It's like saying that my Cessna is a better stealth fighter than your Piper Cub, and ignoring the F-111 because "we're not competing in that market."
The fact remains that this test only proves that for applications where you should have been using a high-end OS (and, apparently, where stability doesn't matter), NT can pump more bits down the pipe for as long as it manages to remain up. As for "real world scenario," this test sure ain't. I'm a little disappointed that RedHat actually sent people to compete in this test, since we all knew what it would show anyway. They should've just pointed out that the test was silly and they have more important things to do. Corporate pride and all that, I guess...
----
Morning gray ignites a twisted mass of colors shapes and sounds
There is no K5 cabal.
I am not the real rusty.
I think this test will help linux developers optimize code for round III. What I dont understand is why test on a 4 way processor? why not a single processor?
Microsoft aggravates my tourettes syndrome.
Just commenting on one point...
If you want to maximize throughput, you can easily tell the kernel to do just that (and trade it for latency) -- it's a simple setsockopt() call. Alternatively, you could use ipchains to do it, if your server program doesn't. I would be very surprised if Apache and Samba didn't do this already. (I think Samba lets you set individual socket flags in smb.conf, even.)
/* Steinar */
(This comment is of course GPLed.)
Actually, 4 NICs are quite common for mission critical web servers. You have a routable IP on one NIC and a non-routable IP on the other to communicate with the application server, or database, etc. That alone requires 2 NICs. Now for basic redundancy you need 4 NICs in total.
"The purposes for the Open Benchmark were:
To confirm that Mindcraft's previous testing was unbiased and representative of Linux's performance."
It seems to me that their main intent was to validate their results. If the specific intent is to validate previous results, then I will guarantee that bias exists. Will Mindcraft publish results that validate the suspicions that they were biased from the start? No-- that would totally destroy the shreds of credibility they have left.
But I'm not going to totally dismiss the results, either. They followed a definite process, and we can't simply ignore what they say because we don't like it. I think that Microsoft *does* have some areas where it beats Linux-- we need to work on these. Mindcraft is only bringing it to the public eye.
But if Mindcraft is bring problems with Linux into the public eye, that means that developers can show why Linux *is* better. Bug fixes are going to pour in. Changes will be made, enhancements created, performance tweaks and changes will make their way into the kernel and into Apache. Performance will improve. The developers will gain more experience, allowing them to enhance their creations even more.
Damn, it's almost like they're doing us a favor!
What is new with this test. Look at the date (30-June-1999). Did I miss something here?
I beleive that there were some issues with the RAID controller being used in the Dell machine?
In this case, as the Mindcraft chose the hardware (to use a poorly supported controller,) I still see this as a rigged test.
A better test would be:
"We've spend xxxxx$ on this box from Dell, which has these features. Come up with an equivalent priced box, and go head to head with us."
Oh yeah... Mindcraft vindicate Mindcraft's reputation. Ha.
Mindcraft Certification?
Excellent. I haven't had such a good laugh in weeks!
Mindcraft certify that the conditions chosen in the above benchmarks were specially chosen to make their biggest buddies (Micro$oft) happy.
What? Was I supposed to take this seriously?
Deleted
/. is about five months late on this benchmark. i guess they are under pressure from the money people to get content, so that drag this thing out again.
the point that linus and others have made about the pc week sanctioned tests is that it wasn't cheating on the benchmarks that flaw the results, it was cheating on the benchmark itself.
the benchmarks are designed to do one thing and one thing only, show NT's strengths against Linux's weaknesses. only idiots take these results to mean anything at all other than mindcraft has done it again. they are paid to make NT look good, and they do. they change the test parameters, the benchmark itself, depending on whether or not they are sliming novell, or sun, or linux.
the real question is why is
Actually I have. On a banking environment I had a machine that connected to the German stockmarket on 2 nics (in case one died), and the other 2 nices were to bridge up to our mainframe, again in case 1 died.
Not a common situation I'll admit, but it happens, so don't make such sweeping statements. On all my web servers, no matter what OS I have 2 nics, a big raid, 2 power supplies and so on. It's good practice.
So they weren't allowed to use a kernel still under development at the time? Thats not old software. Imagine if MS had ran an NT kernel that was in beta and it out performed a Linux install. You'd be complaining then that MS had an unfair advantage running beta software.
Maybe they should get Mindcraft to hook up NT to hotmail, maybe *they* can make it work there.
:)
We can get all excited about how fast NT is and how fast Linux is but lets take a step back. Why run so fast if you will just run into the bars.
Linux and the rest of the GNU project, BSD as well is about giving us freedom. Without ownership of the tools of out profesion we are little more the cogs in the wheels of someone elses machine.
Get a grip we are winning. We are winning and they don't even realy know how much we are winning. In fact the only way we can loose is if we stop fighting our war and start fighting there's.
Let them run there benchmarks then we can go out and fix the problems. In the end we will be better in all the ways we need to be.
IF you think this is impossible, or even just unlikely, keep in mind that MS engineered Windows 3.x specifically to bomb out when running under DR-DOS in order to keep MS-DOS sales up.
What I find most amusing is that they claim that they've "vindicated" themselves.
Well, I don't think they did, as the CONSIDERABLE DIFFERENCE in the two results shows that they did _not_ do what they could have/should have done in the first place -- tune with experts from both sides.
First benchmark:
WinNT is 2.5 times faster than Linux in file serving
WinNT is 3.7 times faster than Linux in web serving
Second benchmark:
WinNT is 2.7/1.9 times faster than Linux serving to Win95/WinNT clients.
WinNT is 2.2 times faster than Linux serving web pages
Now, I don't know about you, but I feel that 150% is a massive difference -- and then there's the fact that the file serving is all screwy too.
I see no vindication, except on our side -- Mindcraft screwed up the first benchmarks, exactly as it was said they did.
OK, I build networks. Lately I have been building e-commerce networks in co-location facilities. Here's what's wrong with the benchmarks ... they don't represent real-world conditions. Most benchmarks don't. Some specifics.
... I haven't looked in a while), as is Apache, Zeus, etc. The OS choice to run your web server can be made on performance, reliability, scalability, ease of use, security, and all the other external reasons why you use an OS for any server. No, it isn't clean cut, and it doesn't say "OS X is better than OS Y for web servers", but it is reality.
CGI: NT, somehow, in all the benches, gets good performance numbers out of active content (isapi dlls, in the case of webbench). Sun w/Netscape's Enterprise Server, as a reference, spawns a new version of perl every time it takes a CGI request for perl-based content (apache, on the other hand, using mod_perl, will not spawn all the extra processes.) I don't know how the NT kernel deals with these requests, but it pulls better benchmark numbers, likely because it is using isapi dlls.
All of these, of course, are entirely dependant on middleware server performance. As webbench doesn't actually query a database, I'm not sure how accurate of a description of active content serving this is. Page serving lag time is as dependant on your backend database as it is on isapi, nsapi, or perl/cgi calls.
SSL: SSL processing is a bitch. This is just reality. Several studies show that you can peg 4 beefy processors running a few (teens) SSL requests against them. WebBench runs around 8% SSL. Curiously, NT and Linux show about the same performance numbers (IIS vs. Apache) on SSL processing, but the resource drain that 8% SSL hits create adversely affects the rest of the system in unpredictable ways. I can't say with any surety what the final impact on web serving systems are by putting the SSL in webbench, but my standard builds do not run SSL and static/dynamic content on the same server.
Then, there are the real variables. Active to static content percentage, database to web server connectivity, page size, graphics load, etc. etc. etc.
A real OS-to-OS benchmark is going to be using the same method to query active content (everybody does CGI, not everybody does ISAPI), and is not going to load down the servers with superflouos (sp?) SSL processing. Even then, the only real indicator of performance is to run site-specific, application specific benchmarks on your own web site. The real story is that IIS is a decent web server platform (if they've cured the memory leaks
Cheers.
good. fast. cheap. (pick any two, you can't have all three)
-Legion
What really matters is what can an OS do for the user -- specifically in this instance where the OS is a server. IS people need to be comfortable with their environment and if it works, use it. Especially when you have to have cpmpatibility
I've heard a lot of you say "both OSs have their strengths and weaknesses. Both can do a job and do it well." This is the paramount truth. BOTH do have their strengths and weaknesses.
The most interesting thing I've seend so far from Mindcrafts latest tests were (and I maybe reading the phase 1/2 configureation wrong) is that all the client machines were Windows OS machines.
Duh Huh. Do you suppose they might work better? I certainly do. Lets try this on a homogenous system (though thats impractical unless they have ported NetBench and WebBench to Linux/Unix).
The truth is: Yes Linux needs some work. Yes Mindcraft used a 2.2.6 Kernel. Yes you would expect that MS would work better with MS. Yes 95% of the personal computer market is MS. Yes NT and Linux have their strengths and weaknesses.
But the bottom line is what do you as an IS person (and I say this because a majority of us Linux users are IS people) want out of a server/OS?
Bang for the buck? Good argument. Holds some water. Reliability? Better argument. Holds more water. Potential? Best argument of all. A wellspring.
If you must use NT for a purpose, use NT. Its not all that bad. It works well in its environment and by nature its going to work best with other MS products.
Otherwise, get Linux. Use Linux. Make Linux better. Forget Benchmarks. Forget this silly NT Vs Linux crap. Its all a PR ploy that MS has ccoked up to hit us Linux (AND ALL OTHER OPEN SOURCE OSs) below the belt. They can't beat us in the marketplace, so they have to get us from somewhere. And that somewhere is in our confidence.
INSERT INTO comment VALUE('Doh!') WHERE user='you';
This is rather old news. Blah blah yes NT's faster at the moment when using multiple network cards in a single system. The kernel developers know that NT's threaded TCP/IP stack is making the difference here and they're implementing that in the next kernel. Any speed advantages NT has at the moment will be gone well before Windows 2000 comes out.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
The problem wasn't with Linux's SMP support ( although Mindcraft would like people to think that ). This was a test of Network horsepower in both the single an quad CPU tests you had to pump lots of static data out to a pretty decent PC clients.
Linux got whooped on because it's Kernel has a single threaded TCP/IP stack and single threaded Ethernet drivers. In other words with 4 NICs in the machine it wasn't getting 4x the data throughput because it was addressing them 1 at a time. This is what I got from the SaMBa people ( ask them about SaMBa on Irix in that kind of configuration ).
The good news is that the relevant subsections have been completely rewritten for Kernel 2.3/2.4 and informal benchmarks show massive speedups ( I have herd _unconfirmed_ rumors of 3x and more in _this_ configuration ).
A test of SMP performance would be database work so head on over to SAP where they can tell you about Linux and NT getting essentially the same performances running R3 on 4 and 8 CPU boxes. Note the 4 CPU tests were with 1Gig vs 4 Gig for NT and Linux. Linux didn't support large memory at the time but the 2.3.25 Kernel dose. Up to 64 Gigs on x86.
Performance is something that can be bought with time and tweaking of Code. MS claimed NT was a Unix killer and started to tweak it for this kind of work from the beginning. Linux was designed as a desktop / small server OS so it's only since these benchmarks that people even noticed these shortfalls. They are being worked on. It takes time. Let the hackers do their thing.
In the mean time head on over to ftp.kernel.org if you are an admin in a place with a large pool of boxes and grab the development kernel. Install it on a bigassed server, take precautions to protect data and benchmark it. grab as much debugging info as you can and talk to Andrea, Alan and the rest. They will tell you what they need tested today, but only if you ask.
--= Isn't it surprising how badly I spell ?
What I mean by this is really pretty simple. In a Windows Operating Environment, people are encouraged to use very different toolsets than they are in a Linux operating environment. Instead of perl, apache, and MySQL, they tend to use ASP, IIS, and Access/Jet through ODBC.
ODBC alone is a performance and stability nightmare, especially if it is not setup perfectly. ASP is a piece of junk. IIS is (I guess) okay. In Linux, perl is (arguably) pretty good, so long as you use mod_perl, MySQL is the fastest thing under the sun for those tasks it can handle, and apache is (like IIS) pretty good if not designed for speed.
We aren't comparing operating systems, we are comparing operating environments (or at least we should be). And testing static pages only totally discounts the afects of operating environment.
Another thing to look at is "culture". Linux users tend to like carefully crafted point solutions -- that's why we're Linux users. NT users just want to get it done, stability be damned -- that's why they're NT users. I think that this difference has a lot to do with Linux reputation for speed and stability. Even a novice sysadmin, exposed to the Linux community, starts to soak up the ethos of the community. Even more important, the support (including full source code) is in place to allow him to do as well as he would like. An NT user soaks up the "get it done, screw stability, hardware is cheap" ethos of the NT community. And the resources to do it right are often /not/ easily available.
Anyway, the point is that we are /not/ just comparing kernels. If we were, then we'd all probably be running some custom TCP stack on embedded hardware.
-- Slashdot sucks.
I am just wondering, with all the great things I've heard (and know) about linux...why would this happen? Is it simply a matter or SMP being immature? Is asynchronous IO not yet fully implemented. I'd have to say, when MS ignorantly claims that linux is based on "decades old technology", these two must be a thorn in its side. Sure, most linuxvolk probably never encounter this cieling...but is it being worked on by somebody somewhere? We can't just say every time somebody does a benchmark "Oh, well, you forgot to use the latest kernel that just came out an hour ago, along with X number of requisite patches".
(I'm not trying to be a troll...just wondering)
It's 10 PM. Do you know if you're un-American?
People who haven't rebooted since they installed, could still be using RH 5.2. One of the funny things with NT, I tend to stay more on top of the fixes because I have to reboot the NT servers more often. (Might as well try a the latest SP while I'm at it) The Linux servers rarely even have a monitor plugged in. I do all the administration remotely, and they just run. So I'm sure there are a number of RH 5.2's that have been up for a year that nobody wants to reboot for the upgrade.
Just remember that. So *of course* every benchmark you see will show NT is superior. If you ever see anything different, make sure the MS lawyers don't. :)
"Evil will always triumph over good, because good is dumb." - Dark Helmet (Spaceballs)
This only points what areas of Linux need to be improved.
-- That which does not kill me only pisses me off
This post encoded with ROT26. If you can read it, you've violated the DMCA. Handcuffs please, sergeant.
When I read NT vs Linux comparisons like this, I find myself laughing for the fact that the two are even being compared. The mere fact that Microsoft cares enough (or is scared enough) to compare the two is a success. The mere fact that people consider it a viable alternative is a success. Other x86 OSes -- commercial x86 OSes -- would kill to get top billing beside NT, even if they lost the benchmark race.
Like many of you, I've been using Linux since about 1994 -- since before Windows NT or Windows 95 even existed. At that time, Linux was little more than just a cool hobby. We were making and using our very own operating system. For me, just being cool has always been enough for me. Back then, we didn't care how it compared to commercial OSes because we had no need for commercial OSes.
In the "early days" of Linux, we were exstatic if Linux merely got MENTION in a magazine article about operating systems. Now, you can't get away from it. Everywhere I look I see Penguins on magazine covers and cars and T-shirts. This would have been unthinkable when I first became enamoured with Linux.
Someimes I worry that we're unknowingly hurting Linux by making it come into the parlour and perform amazing tricks for our visiting guests, like a child prodigy. But I think in the end, as long as I have a cool operating system, that is MY operating system, and as long as I can continue my fun little hobby the world of Linux will be well, I will be happy.
Micros~1 can be split up, go bankrupt, or fall into the Pacific ocean. Linux will always exist as long as people use it and want to improve it.
--
In a one-against-many struggle, the individual always loses in the end. What is Micros~1 going to do? Hire more people?
Why are we rehashing this? This is the June 15th press release. PCWeek did its tests AFTER this. The kernel used was 2.2.6. Am I missing something?
A great many people think they are thinking when they are merely rearranging their prejudices. -- William James
I suppose you want more leggos for Xmas, eh??
send flames > /dev/null
Only 'flamers' flame!
So what if NT can throw out static pages faster? I'm sticking with UNIXish systems because when I use them, I know I've got more power at my fingertips than NT's pop-up happy dungeon could ever give me. I can change the code on most systems... I can craft my own tools by combining existing ones... and I can find the answers to my questions on the Web rather than having to pay "per-incident" fees.
On top of it all, if anything does go wrong, 99.44% of the time I can fix it without standing in front of the server. NT doesn't have that capability. If a service needs to be upgraded I can do it without affecting anything else on the box.
UNIX is power because you can insert yourself at virtually any step along the way of any process. You want to do anything in NT, you must do it with whatever APIs MS thinks you need, or pay them to make more.
I'll stick with the power, thank you.
But still, you'd end up having to tune both servers. And tben it boils down to `which OS do you know best'. Besides, some parts of the disk are able to deliver more data quickly than others (outer/inner sectors). I don't really see how this should help anything.
And, `set up another machine to bang on them with wget or something' is a TOTALLY unrealistic perspective. One machine, downloading 100 copies of the same file, and never accessing a single CGI script?
And how are you going to benchmark Linux and NT at the same time, with only one machine? If you use it as a webserver box, no doubt the load would be higher at some times. There really are too many unknowns in such a test.
/* Steinar */
(This comment is of course GPLed.)
The last paragraph really tells the story, doesn't it? Even if we lost this time, we are regrouping and will have a go at this again.
``Linux file- and Web-server performance appears to be bottlenecked in the operating system kernel, not in Samba or Apache. This was demonstrated best when the Red Hat engineers ran the Zeus Web server. Zeus performance topped out at about the same place as Apache, using fewer resources. The major performance problems are with the TCP stack, which is single threaded in the 2.2.x Linux kernels, and with large-grained kernel locks that degrade multiprocessor performance. The Linux community is addressing these performance problems and others in their 2.3.x kernel series.''
Let's keep up the good work and we can be sure that even if this benchmark is unfavorable, no one can really match the flexibility of the Open Source development model, and that even if we don't win now, there is always tomorrow.
Check Linux Magazine #1 where they benchmark Samba, Linux kicks NT's ass. :)
Was anyone else amused that Mindcraft found that samba running in user space was a bad thing? (Look at the phase three report.) Hey... I wonder if this means someone should hack up a ksmbd? :)
I am curious, however, to see what the performance/reliability difference is when we're looking at a four-way linux cluster vs. the four-way SMP NT...
-_Quinn
Reality Maintenance Group, Silver City Construction Co., Ltd.
OK This is really lame. I hope next time Rob & Co. will actually check the validity of the article before posting it. It's already been covered before:
1. They tested on a machine with 4 100BaseT betwork cards. Show me a web server that has 4 * 100 Mbits/s bandwidth and uses it to serve static content.
2. Even on a file server it is extremely unlikely that it would have 4 network cards.
3. Contrary to what the tests show, Linux does not have pathetic scalability. Just do the same test with one 100baseT network card and notice the difference. Also, if you want to test real-world performance, you got to test dynamic content. C't did just that and Linux came out on top.
___
If you think big enough, you'll never have to do it.
Or else you'd never get more than a 50% improvement on the same hardware. Never heard of an NT box that served pages fast in the real world.
NT was designed to do this sort of thing - keyword designed. :P.
/tmp/X11-unix lock files? ERK. :P, the registry is a good thing. Yes when win95 came out there were registry problems but I haven't had any problems since 1996. It's a great idea, it's like having a database to store all your settings.
Linus didn't design Linux for the kind of work which NT is excelling at in these benchmarks.
When Dave Cutler sat down and designed NT, this was the kind of things they were trying to do, fine grain kernel locks, high performance and scalability. The market place has unfortunately seen many of the good things about NT get forgotten (portability for example), but NT still stands there with the ability to scale MUCH MUCH better than Linux can at the present.
Yes you may feel like going out and burning a few MS cds or whatever, but at the end of the day it's true. Improvements ofcourse are being made to linux, and linux may catch up.
However, I'm actually a bit worried about the fundamental design of Linux itself - I'm not saying it's totally 30 year old technology - far from it - but having experience with linux and NT for quite a number of years now, to me, NT seems like it is better designed and had good goals.
I won't bother to argue about whether they were met or not here tho
Some fiddly things about Linux/Unix I don't like are:
-Threading. According to IBM, Linux native threads are mapped processes!??! which makes their JDK rather slow compared to NT.
-Mutexes/Semaphores/CriticalSections etc - why doesn't Linux use them? I mean for god sakes what the hell are linux applications writing *.pid files around for? And what about
-Componentisation - it's happening slowly but only in the past few months (maybe a year). I'm still waiting to see the Unix APIs wrapped up.
-Registry. I've said it before and I'll say it again
Now I don't really care whether the registry is one huge file or several files (user and system) like in NT, but I just want some STANDARD APIs for reading writing settings - fast APIs.
Ofcourse the registry has other uses too, like storing COM/CORBA UUIDs etc etc etc.
Being a database it'll definitely be faster than parsing text files, and even better it's much easier to programatically add/remove/change settings (trying to parse text files to do that sort of thing sucks).
Anyway, it seems everytime something about Linux comes up the response is "someone is working on it". When it comes up again the answer is the same, and then everyone ignores the strengths that NT does have because Linux will have it cause "someone is working on it".
Just give NT, MS and Dave credit, and move on.
Linux is not the solution to everything. It's a great free small-medium server & emerging desktop OS. Let's leave it at that for the next year or so.
Initially, I used NT because it was simple to create accounts and setup for web use. This was before I had to create a hundred accounts at a time. It was also before I needed to implement virtual domains from a single system.
With a bit of reading and knowledge of scripting (hopefully skills common to most system admins), it was a piece of cake to accomplish this on a linux box. Mind you, I did not start this quest on linux. First I attempted to do this on NT. After many days of reading, an surfing, I found the only way to accomplish these tasks under NT appeared to involve purchasing additional software.
Being I did not have money in a budget (they don't even give me a budget), I thought I had nothing to lose by attempting to set up a linux system with a friend's copy of RedHat 5.0. Within hours of the installation, I found the information I needed for both the user account problem and the web issues. This was about three years ago. Due to Microsoft "features" which relates to their virtual networking and security scheme, I have been forced to choose between peopling a helpdesk from my bloated budget or shifting my servers over to non-NT.
The truth is, MS support is a Secret Art. Even texts by the bigger named publishing companies don't shy away from stating this in their training tomes. The problem is they are not trying to be funny; they are talking about MS's testing relative to their stated objectives.
I don't know it all about MS or Linux, but at least with Linux there are some great resources out their in both printed and electronic form. Too often the NT answer is, "Oh you want one of those, well, we sell that for...". Take your pick. E-mail is a prime example. Better yet, consider the cost of NT based on a per client basis (especially under their new licensing scheme for 2000).
In a place beyond time and space, in a land far better than this, look for me there...
This white paper makes me feel somewhat better about Mindcraft's position, at least enough to believe that there are ways in which Linux needs to play catch-up.
I think it is funny that this paper mentions four times that Mindcraft is unbiased.
THE PROBLEM IS that all this says is that the test results are valid, but the test itself (as we know from previous discussions) is suspect. These graphs (which are a bit sketchy themselves... fitting a curve to one data point is only so informative) are measuring obscenely high access rates for static web pages, which is a rather contrived benchmark.
I do appreciate that there are one processor benchmarks added here, and not surprisingly we see that some of the NT/Linux disparity is in SMP scalability (and ultimately TCP/IP multithreading.) But we don't need to wait for the multithreaded TCP/IP stack in Linux to find tasks that Linux is perfect for.
I cannot say with any certainty whether Mindcraft deliberately chose conditions to favor NT. It certainly seems possible, but on the other hand they have perhaps received more criticism than is warranted already, so I am reluctant to rant about it. I am curious why we don't see open benchmarks on a wider variety of tasks, and including the *BSDs as well. Even knowing that benchmarks are inherently weak, I would personally like to see how everything would fare.
how is this an unbiased test? they started it using a quad-cpu machine. linux is know for having rather poor smp support. thats one of the fixes the 2.4.x kernel is supposed to fix. i find it rather hard to believe that microcraft had no idea of this. not only that, they were using a old version of linux (rh to make it worse ;-) ). thats like having nt 4.0 (no service packs) versus rh 6.1 with 2.2.13, it just isn't fair.
I had a machine that connected to the German stockmarket on 2 nics (in case one died), and the other 2 nices were to bridge up to our mainframe, again in case 1 died (emphasis mine)
Ummm... so you're using two nics, and two are serving as backup? If that counts as using 4, then I guess my computer uses 3 mouses: 1 which I use, and two sitting in a drawer in case it breaks...
Chris
San Francisco values: compassion, tolerance, respect, intelligence
It is always fabulous when someone points out deficiencies in a piece of software. It gives everyone a goal to start shooting for. These benchmarks seemed pretty good, having the numbers and the methodology should allow the developers to fix the problems that may have cause Linux to come in second.
In my experience once and issue is identified, developers can fix it pretty quickly.
timbu
I have no doubt that the numbers given are correct. The question is are the benchmark's constraints realistic.
For example, I can benchmark my speed on foot vs. a car. We can do that on the drag strip for 1/4 mile, New York city at rush hour, or on a test track in a 10 foot race. I will loose badly in the first test, the second is a coin toss, but I will certainly win the third, no matter which car you choose!
I am also reminded of another 'benchmark', a Chevy Nova ([poorly] modified street racer) vs. a beat up Honda. The Honda won because the Chevy boiled over.
So, yes, I believe that benchmark, but I don't think it means what MS would like to think it means.
There is definitely a problem with the 4 ethernet card configuration that the Mindcraft folk use. I personally have never used more than 2 ethernet cards on a Linux box, and that caused some problems for me at first. I can certainly imagine that 4 fast ethernet cards could make a 2.2.6 kernel sh*t a brick, which seems to be the case here. Especially see some of the absurd initial Mindcraft tests where performance falls almost to zero at one point. The link I posted above mentions the fact that the kernel starts spending all of its time in interrupt processing with high load on multiple ethernet cards.
Okay, so what do we gain from this survey? Simple, don't use a 4 ethernet card configuration on a Linux box. Period. Or do it with caution, use a 2.3.x series kernel. Stick with single 100bt configurations for 2.2.x kernels, DON'T use pre-2.2.7 kernels (TCP bug, serious performance problems may result, ahem). If you need more bandwidth than 100bt, go with gigabit ethernet, or don't use Linux (I'd be using Solaris if I were dealing with a system with that kind of load).
Better yet. I'd save my company a lot of money, and improve performance by getting 3 or 4 single processor systems, or SMP systems with single 100bt cards. I mean, all we're really testing here is static HTML or file serving. For the price of one of these whopper Dell servers running NT, I could have a FAR more efficient better performing setup. Anyway, just making some obvious points that occurred to me as I was reading through the Mindcraft drivel. They may not be lying, but they're definitely still not being honest about WHY NT appears to outperform Linux.
Alas though, I must concur that there *does* appear to be a performance advantage to NT under the criteria of the mindcraft survey. Perhaps Mr. Cox & Co. could give us a suite of testing tools as used by m.c. so that development can engage in earnest on those areas which the kernel lacks. Single-threading in the TCP/IP stack has been a long time issue AFAIK... could a TCP guts rewrite to be more SMP-friendly turn out to be the only difference between the Linux vs. NT performance issue?
--
rickf@transpect.SPAM-B-GONE.net (remove the SPAM-B-GONE bit)
"People will pay big bucks for the luxury of ignorance."
In a previous life circa 1996, I did some real intensive filesystem performance studies including various commercial Unices (not linux). I know quite a bit about how to do this (I authored the reasonably popular "Bonnie" filesystem benchmark).
Back then, for a mixture of directory traversal / file open / file read operations, NT was a lot faster (on reasonably equivalent hardware) than any commercial Unix we looked at.
In that case, we went and bought the unix boxes anyhow because other factors were more important; I think this would be the case in most web-server apps.
But if you look at the netcraft benchmarks, you see a *lot* of ordinary file opening and reading - in this context it doesn't surprise me that NT does well.
On the other hand, it wold be nice if Linux did better.
-TimI dont care about this mindless crap of benchmarks... I know what Linux (Redhat or otherwise) has done for me, and that is run, provide a solid server, and make me learn.
I also know what Mickeysoft has done for me also... sure I was into NT and 95 when it first came out... but in the long run, it has done nothing but waste my time redoing configurations, rebooting, and constant upgrading.
Screw Benchmarks, I know which one is dependable in a real world situation, and no benchmark will change my mind.
'NT is a just ok server with a cartoon face.'
"If you have done 6 impossible things this morning, why not round it off with breakfast at Milliways" -- hhgg
The numbers were better when the test was fairer. A still fairer (ie more realistic) test would be even further in our favour. That they untied the weight around *one* of our ankles does not make it a fair race.
Benchmarks run by those without an axe to grind (eg c't) consistently come out in Linux's favour. A lot of design work went into finding ones that would point the other way: for example, using four 100Mbit cards rather than one gigabit card. That the actual anti-tweaks for Linux were taken out doesn't mean the anti-Linux design wasn't still there.
That's why everyone remembers these benchmarks over all the other Linux vs. NT benchmarks. It wasn't because they were particularly well done: they are famous and remarkable because they're the only ones that NT doesn't lose like a dog.
--
Xenu loves you!
I run NT4 on my own system and know from experiece that it is quite fast and fairly reliable as a workstation OS. However, while running linux i discovered that once it was setup and doing what i want i could leave it running forever and it would still be running just fine. NT does not quite have that low maintence luxury. Do to being a microsoft product RAM is SUCKED by any application, and is quite famous for having maintence problems. Thus the theme of my thread. If you want to run NT becouse its faster great, but if you want to be on the net hosting things and have your server stay up...get linux. Besides its free.
There may be people out there that see Linux as nothing but ani-Windows. They want to see Linux beat windows into the ground. These people really fail to see the strengths of Linux itself.
It is fun. Yup, that's right. I enjoy running it.
I don't preach Linux to people because it is the fastest horse in town, but because I enjoy it, and I think likeminded people will too. And almost without fail, they agree. Out of all of the people who I've helped transition, I've only lost one of them back to Windows land. You do have to hold their hand in the beginning, esp if they aren't good with *nix in general, but after a short amount of time, they become comfortable and enjoy running it.
Because it is just fun.
This sig is false.
I installed Caldera on one of my extra machines. It was a pretty cool & easy install. The coolest thing was that it never rebooted, just killed all processes running and started running the init scrips! Another cool thing was playing tetris while it decompressed files :-)
But according to their graphs for Web and File server tests, Linux would saturate both a 10mbps and a 100mbps connection before NT4, and at 120 clients, it's still a long way from falling below the 100mbps saturation point, except when running as a file server with a single processor.
So all questions of cost, reliability and configurability aside, the test results seem to show Linux having better throughput than NT under light loads and equivalent throughput under heavier loads for anyone with a normal network connection.
Didn't c't already come to this conclusion months ago?
First, if we all followed what other people say on paper (albeit virtual paper) we would be using fucking windows. Another point these assholes keep missing is that if Linux, GNU/Hurd or FreeBSD can't scale up to something then chances are the person looking inot it will buy a UNIX varient. Why preytell? Or why investigate building a beowulf cluster to match the high end performance of nt? Mainly because NT is a leaky, sloppy fat piece of unusable shit. Granted, joe jizzlobber can use NT and maybe even administer it, the same can be said for MACs or whatever other dipshit systems are out there, but when you try to get NT to do something like run an oracle db or some freaking Front Page-IIS crap it is like pulling fucking teeth. These dickwads insist that you upgrade this that and the other and then the fucking upgrade breaks other software, e.g. Oracle and/or IIS. Not too mention the second NT starts pissing itself with memory leaks your only recourse is to reboot. No core files, no nothing. Anyone worth their salt, and I mean all those slaves at infoworld or ZDfuck or where-ever who say NT is better than Linux, FreeBSD or UNIX is a dumb ass. More usable? Sure for idiots. Nicer to look at? Maybe. But better - wrong answer. With OSS driving UNIX/Linux and FreeBSD the simple fact is they are more stable, more open and a hell of a lot easier to manage. Plus - they are fun :)
" -- ow my brain hurts again -- "
If the test it to determine relative stregths of the two OS's, then why not run the same software on both OS's? I think to accurately benchmark Linux against NT, one needs to use Apache on both.
I agree that an underlying understanding of the mechanics of the Operating System help a great deal. This is where Open Source OS's (Linux, FreeBSD, Hurd --- whatever) can help an administrator understand other UNIXes. For example, taking a look at kernel code (not neccessarily modifying it) and understanding say how the struct task_struct operates can really help out. While they may not be the same, analagies may be drawn between them that could help a sysadmin understand why UNIX flavor X does something (or doesn't as the case may be). Of course, fundamental programming knowledge is required, but for that level of programming it shouldn't take long for the average whiz-banger to come up to speed. and this line: "The notion that one does not require deep knowledge of systems to be a systems administrator is tenable only in a system with nothing ever happens that is outside the training materials. No such system exists." Should be put in the smithsonian somewhere.
" -- ow my brain hurts again -- "
This hits me in a sensitive spot - I want Linux to be the fastest in addition to being the most configurable, fun, and attractive OS. Oh well, time to pick up the old ego and get back to work. We will get to where we want to be eventually and then it will be M$ left to make excuses...
This is outdated information even based on the
kernel I am using (2.12-35), Samba 2.0.5. Even if
they were right at that moment in time, we keep
moving at a pace faster than NT. And I have
an upgrade to Samba 2.0.6 due today.
The hardware they use is outlandishly beyond the budget of most of us poor mutts. I cant get NT
to even work any kind of fast on my P133/16M
as I can with Linux. Good to know that NT will
work great with a super-fast processor(s) with
lots of memory.
Less is more!
In _The Psychology of Computer Programming_(*), Gerald Weinberg wrote a story about a programmer who was flown to Detroit to help debug a program that was in trouble. The programmer worked with the team of programmers who had developed the program, and after several days he concluded that the situation was hopeless.
;)
On the flight home, he mulled over the last few days and realized the true problem. By the end of the flight, he had outlined the necessary code. He tested it for several days and was about to return to Detroit when he got a telegram saying the project had been cancelled because the program was impossible to write. He headed back to Detroit anyway and convinced the executives that the project could be completed.
He then had to convince the project's original programmers. They listened to his presentation, and when he'd finished, the creator of the old system asked,
"And how long does your program take?"
"That varies, but about ten seconds per input."
"Aha! But my program takes only one second per input." The veteran leaned back, satisfied that he'd stumped the upstart programmer. The other programmers seemed to agree, but the new programmer wasn't intimidated.
"Yes, but your program doesn't work. If mine doesn't have to work, I can make it run instantly and take up no memory. "
Moral of the story: correctness first, then speed.
How fast would NT be if they fixed it?
Saying that the TCP/IP stack is ``single threaded'' is completely misleading. The TCP/IP stack in Linux isn't threaded at all. It is not a process. It is just a bunch of code operating on data that is shared among processes/threads and interrupt service routines. Any process in the system may enter the TCP/IP code, as may any interrupt on any processor. The Mindcraft cluelessness about how the kernel works takes even more away from what little credibility they have left.
To say that the Linux stack is single-threaded implies that it has an internal thread which does all the work. Obviously, this is far from the case.
*note* slashdot is obviously overworked, this test was posted 4 months ago, only differnce is it was reformated. *sigh* This is OLD news.. These benchmarks exploited a flaw in the 2.2 kernel's IP stack. Everytime you add another network card you effectivly cut the performence in half. This was caused by the fact that 2.2.x locked the "whole" IP stack everytime one of the "other" network cards were in use. *duh* this is why mindcraft used 4 network cards instead of 1 100Mbit network card. If they needed more bandwidth they should of used a Gigabit network adaptor.
-- You can be a geeklord too
These benchmarks were released on the day of Gates' Comdex Keynote? Coincidence?
Everything we learned from the antitrust findings of fact would suggest that it's not coincidence. I therefore have to hand it to Microsoft, not for winning yet another questionable benchmark contest, but for maximizing the spin benefits thus obtained. That is true art.
This is an example of a Microsoft "spin" attack. There will be more to come - the battle has just begun. Let's admit it, Microsoft won this round in the spin battle - mainly because we weren't fighting, and took a punch below the belt. Referee - what referee? OK, so we learned something about the rules of the game.
Let's do two things:
1) Make Linux better so we win these high-end SMP contests as well. I don't know about you, but I'm treating myself to a 2-processor machine this Christmas, and I want it to kick ass running Linux. With 100,000's of geeks likely doing the same time, we can expect 2000 to be the breakthrough year for geek-SMP. We also need better file I/O. Not that the existing I/O isn't damm good, but it has to be the best, right? (Personally I'm putting my money where my mouth is - as a developer, I can make a difference and thanks, MS for getting me steamed enough to jump in.)
2) Master the PR game. Microsoft can, and will hurt us with PR. Some may say "so what, who cares what Microsoft says, it's how good Linux is that matters" and there's a lot of truth to that. Nonetheless, why don't we cover all the bases? We need an open-source think tank cum swat team whose only purpose is to anticipate, forestall and counter the PR moves that Microsoft makes. We have the collective intelligence to play that game well, and hey, it's a fun game when you win.
Life's a bitch but somebody's gotta do it.
*note* slashdot is obviously overworked, this test was posted 4 months ago, only differnce is it was reformated. *sigh* This is OLD news.. These benchmarks exploited a flaw in the 2.2 kernel's IP stack. Everytime you add another network card you effectivly cut the performence in half. This was caused by the fact that 2.2.x locked the *whole* IP stack everytime one of the "other" network cards were in use. *duh* this is why mindcraft used 4 network cards instead of 1 100Mbit network card. If they needed more bandwidth they should of used a Gigabit network adaptor. p.s. Run the exact same tests with 1 nic card and or wait for 2.4 to be released and you will see how f*cked these tests were.
-- You can be a geeklord too
Haha the real problem was that the 2.2.x linux kernel had "major performence" problems with TCP/IP ONLY WHEN USING MULTIPLE NIC CARDS. Which is why they USED multiple nic cards..
drink your coffe boys..
-- You can be a geeklord too
This type of test is GREAT! No, really! This shows exactly what kind of thing suits want in their enterprise. They don't care about performance, money, anything like that. They want a pretty little graph that they can show to their boss and he can show to his boss, and in the end, he who has the prettiest picture wins. Optimize for pictures!!!
LOAD "SIG",8,1
LOADING...
READY.
RUN
These benchmarks will always yield spurious results until they can test the same software on different operating systems. The test really compares the speed of Apache on Linux versus the speed of IIS on NT (which is tied into the OS's kernel, which may make a large performance difference). Apparently they don't feel Apache on Linux versus Apache on NT is a valid comparison because the Apache Group warns that the Win32 version is beta software.
As a rule, the "tests" or "studies" done by Mindcraft favor the sponsor of the tests. The recent Apache versus iPlanet benchmark they did is another fine example. When you read the actual reports they write up, you can find several places where the the scientific value of the benchmarks is corrupted by time constraints imposed on the losing side. Even in this June 30th Apache/IIS showdown, the RedHat team had to fly home in the middle of Phase 3, and they didn't have (sc., weren't allowed) time to implement and tune the latest build of RedHat.
Until _Consumer Reports_ or a similarly disinterested, research- or consumer-oriented venture (boy, there's a niche waiting to be filled) starts doing benchmarks, we are not going to see truly scientific benchmarks.
"Sometimes the light's all shinin' on me, other times I can barely see."--R. Hunter
It's called WOW (Windows On Windows). All Win16 apps run in the same address space (WOWEXEC) because many Win16 apps were designed to share data between processes. IPC was super-easy on Win16 because there is no boundary between processes. A Win16 app can simply send a pointer address (maybe via a Windows message) to another app. The other Win16 app can simply deference the pointer. cheap IPC.
cpeterso
To a certain extent, you are correct, however:
In the case of the NT tests they used a patch to assign each card to a processor (affinity)
This option, to my knowledge, was not available to the Linux boxes. I believe that the TCP/IP stack in the Linux kernel is bound to a single processor. I still see this as a SMP problem and not the fact that they used 4 NICs. It is still correct to say that Linux has serious SMP scalability problems, even Mindcraft admit however, that these issues are being worked on in the development kernel thread 2.3 and will hopefully become a standard performance improvement feature of 2.4 .
We can't blame NT for supporting a function that Linux doesn't, though it would be interesting to have a benchmark based on the following:
Using $x, contruct a system using NT and a system using Linux, tune the systems to their full ability.
With this type of benchmark, Linux can better compete, because of the following:
1. For low values of x NT will simply not run or cannot be purchased and hence is disqualified from the test.
2. The system can utilise a cluster topology, which although slightly more expensive for n systems, would at least provide a linear improvement per CPU. For n=4 and mid-range x, the recoup over costs of Enterprise NT Server, still make this viable for Linux to compete with NT.
It is also clear to note that many Internet websites have an aggregate bandwidth of 512kbps or less and seldom actually reach the loads that were reached in the benchmark tests. It's the case of the limiting factor again.
Let's get back to the real world: There are many webmasters and sysadmins who are perfectly happy running Linux on their Web Servers, they know the systems work, and they know that they don't have to monitor them as closely as one does with some other systems that are available on the market.
There are so many places that an operating system can win or lose... performance is just one of them. as well, there are reliability, cost, etc. if u need performance at high loads, then go buy some NT, but if you'd like to take fewer chances with reliability, go for linux. different OS's for different needs. im glad that there is some good competition goin on :)
Hmm, need a new server - should I use Linus or Gates? Go with the super-duper Red Hat 6.1 - It's supposed to have everything!
/usr like 6.0 did, heck, I'll just waste part of my fast disks for it... configure /etc/raidtab... mkraid... okay! hdparm -ft /dev/md0... 3.88 mb/s. Eww... gag...
/dev/hde... damn, doesn't work... Need better HPT366 support. Just search the web... Ah, here 's the link to the patch at http://ntucsu.csie.ntu.edu.tw/~b6506063/hpt366. Superb... I'll be up in no time...
/etc/lilo.conf, copy System.map, etc... reboot... damn, doesn't recognize the md RAID disk anymore! Ah, Red Hat must have installed the latest md patches to make it work... at least now I get 20.33mb/s with hdparm -ft /dev/hde... I'll just get the md patch for 2.0.13 from http://ftp.us.kernel.org/pub/linux/daemons/raid and... urk, last update was for 2.2.11. Uhh... S'ok, the linux-raid mail list archives say it works fine, just ignore the errors patching the PPC and Sparc archetecture parts. repatch... Now we're in business! It sees the MD disk! hdparm -ft /dev/md0...
Hmm, wait a minute - need rather better than the max 20Mb/sec that the two 7200 RPM IDE disks (new ones) give (of course! Any of the newer ones can pass data under their disks at near that speed). Hey, that's okay! I'll just use the Linux software RAID, that's supposed to be really really fast!
Hmm, waitaminnit. The mainboard used the HPT366 UDMA/66 controller. Well, that's been out quite a while, shouldn't be a prob...
Yup, just add "ide2=0,, ide3=0,," and it recognizes them. Install Red Hat 6.1, drat, it doesn't recognize the Future Domain 16C30 SCSI controller I was going to use to hold
Ah, DMA must not be enabled... hdparm -d1
Ah, the patch is against 2.2.13. That's okay, I'll just recompile the kernel. Download 2.2.13... patch... make {mrproper, menuconfig, dep, bzImage}... edit
drive hde lost interrupt
drive hdg lost interrupt
drive hdg lost interrupt
drive hde lost interrupt
.
.
.
[until system hangs, ignoring Alt-SysReq]
Ah, the hell with it... Just install NT 4.0
server with striping and the manufacturer's official drivers...
Ok, this is likely an old point, but since I have a policy against saying anything new here goes:
Summary: I don't believe medium+ companies CARE about performance. They see hardware as a capital investment. It's ok to buy $.5M worth of hardware, if that's what is needed for performance, as long as the software is stable.
So I hear you cry, why are they using NT?I don't know to what extent they are, but if it is, the fact that linux has so many new versions so often might be part of the reason.
Linux moves too damn fast for corporate adoption. I'm not even talking about 2.3, there are too many tweaks going on just in 2.2 for conscientious admins to install it. If I were to install it for my (ficticious) company, I'd like to give it a thorough going over before installing it. And that means that all the arguments of "this problem fixed in newest version" are immaterial. That's not the version I'm reviewing. Since so many changes are included in each version, I'd be very hesitant to just install the new one.
No company in its right mind would install beta software as production software, yet that is exactly what is suggested when people say "just install the newest version".
So I think linux is suffering slightly because of this; it is beta software, but not labeled as such. Perhaps what we need is an even stabler branch of the tree; one that only gets bug fixes, not new features.
Scenario:
linux 3 comes out. It incorporates all of the stable and well tested features of 2.2. We get three branches:
3a : corporate. No new features. ever. only
well tested bugfixes, backported from
3.0
3.0 : home user. New stable features and bugfixes
backported from 3.1
3.1 : bleedin' . Anything goes (down, that is).
may reformat your harddrive occasionally.
Or something like that. The idea is to create a stable enough platform for corporate use so that 1) they can have confidence in each new version because there is a longish debugging cycle and 2) we get a good baseline for comparisons to other os's.
The point is to separate out features (bug causes) from bug fixes, so that paranoid coorporations can rely on the software. And make each update small enough to be given an overview before being patched in.
Whaddaya think, eh?
If you look at the charts, Linux seems to have a slight edge over NT for systems on a T1 or 10baseT LAN. I don't know about you, but that matches most of the configurations I work with.
It is nice to see that Mindcraft has quantified my own observations, except that uptime is noticably missing, as is normal.
Thanks Mindcraft! Now I can show my boss numbers on why NT is overpriced for our needs.
BwaaaHaaaHaaa... Apology.. for which?? 1. testing Linux to NT but not NT to linux file system performance. 2. using an older kernel 3. being PAID by Microsoft to do the 'test' 4. allowing you to 'cuwstomize' the test environment such that microsoft would look better (I am sorry that you got tasked with that.. after all, it must have been very VERY difficult to do)
=-=-=-=-=-=-=-= - The Celtic - =-=-=-=-=-=-=-=
Table 1: File Server Speed Advantage of Windows NT Server over Linux
__________________________________________
|......# of................Client............Windows NT is faster.|
|Processors............Type.................tha
|----------------------------------------------
|.......1..............Windows 95................1.5 times..........|
|.......4..............Windows NT...............1.9 times..........|
|.......4..............Windows 95................2.7 times..........|
|_________________________________________|
"And NT? The F-4 Phantom. The gun used to ship separately, and it is living proof that, with a big enough engine, even a rock can fly."
Actually, I would have to compare NT to an F-102 Starfighter. Yeah, it did Mach 2+, but it also took half a state to turn around. The F-18 doesn't go quite as fast, but it's exponentially nimbler...
censorship is a form of noise, which actively seeks to drown out content with silence - Crash Culligan
Funny you should mention MicroSoft using a NT kernel that is beta, a developmental kernel.
i nuxMyths.asp
check out http://www.microsoft.com/ntserver/nts/news/msnw/L
"Linux does not provide support for the broad range of hardware in use today; Windows NT 4.0 currently supports over 39,000 systems and devices on the Hardware Compatibility List. Linux does not support important ease-of-use technologies such as Plug and Play, USB, and Power Management"
See NT4.0 doesnt support USB or Power Management, and the newer production kernels do, sort of. But Windows 1900 (formerly known as NT5) is going to so its just as good right? Maybe in the next benchmark we can show that the reliability of NT is approximately 3% of Linux, or maybe the Linux guys can use the Reiserfs since its in release now too.
I don't run any real servers personally either. So my direct everyday experience is at the power-home-user workstation level.
In my house network (with 5 other housemates' computers + a small 486 NAT FreeBSD server for cable internet) with various printers, scanners, etc., I'm the lone Linux user. The reliability of my Linux system is causing me to develop a lower tolerance threshold to crashes. I get pretty irritated nowadays whenever one of my housemates insists I go over so he can show me something and then I have to wait for reboots or sudden lockups in applications.
They tweak and fixup their systems as best as documented, so that argument isn't going to hold.
It's just that at the workstation level, the added variability causes drastic decreases in reliability of the NT machines. Performance differences aren't going to be really noticed between all our systems (P2's or P3's) since we don't usually use our computing power to maximum potential but if you want to factor in all the rebooting or waiting for tasks to be killed to resume operation, then that would be significant.
This is based on observation of what happens in my house for the past three years.
i click the link to see what you guys think about
- --------------------------------
linux losing what seems to be a fair webserver benchmark test and get this
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, malda@slashdot.org and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
-----------------------------------------------
Apache/1.3.6 Server at linux360.dn.net Port 80
i love linux but thought this extremely ironic
Sometimes for the usual reasons, e.g. I need to be able to run something like Quick Books for my business.
Sometimes for an unusual reason: I run a gateway/NAT on NT because I need to connect to my clients' offices through their Windows-based VPNs.
Sometimes for the usual reasons, e.g. I need to be able to run something like Quick Books for my business.
Sometimes for an unusual reason: I run a gateway/NAT on NT because I need to connect to my clients' offices through their Windows-based VPNs. The VPN client software doesn't exist for Linux.
PovBench Results
You will see that the fastest system listed running NT is an Athlon 600mhz which took 47 seconds to complete the scene. A 600mhz athlon running RedHat 6.0 was reported as finishing the same scene in 37 seconds, which is roughly a 21-22% speed increase. Granted, I don't trust the povbench page, as many of the entries seem questionable. Regardless, most of the entries at the top seem to be running linux.
Nite_Hawk
As far as I know, there isn't such a thing, and I wouldn't envy the poor souls that would undertake such a task. Nevertheless it might be just the
ticket. Just as BSD got the fine tooth comb
treatment, so perhaps should Linux.
An optimization team could identify and correct bottlenecks, and inform the maintainers of the offending code, and they could either heed or ignore the advice.
I wonder if one of the current bottlenecks is GCC and the associated libraries...
What is the point of comparing the speed of Apache which is a MultiTASK server with IIS which is a MultiTHREAD server?
Because they're comparing the performance of various webservers. Each implements it's functionality differently, but the end result in both cases is a client (browser) requests a page from the server. The server then sends that page to the client.
Whatever the server does during that makes no difference (to me, at least) so long as it gives the right page back to the client in a reasonable timeframe.
I think we're in violent agreement, to a certain extent. But:
;-)
> Heck, for the sake of fairness, spot them the OS, so that both teams have the same hardware budget.
I still don't believe that this is "fair." Benchmarks by nature try to convince you that their arbitrary imaginary scenario is fair. What I'm trying to propose is that context _matters_: one should start from a _real_ problem. For example, "the boss gave me a budget of $X, a set of requirements and told me that it's my ass on the line if the solution I concoct doesn't deliver" (or goes over budget - e.g., due to unforseen training needs).
Note that I'm not proposing that one or the other would win: I don't have enough personal information to make a reasoned hypothesis (I'd place a bet but not a bid). All I'm asking for is for some of the serious architects out there in Slashdot land to do the math and show how things would really stand up if we were playing for real money.
All that being said, I still get pumped when SGI posts a huge SPECfp-rate number (http://www.spec.org) - still my OS of choice for Big Problems(TM) $-).
And I hear my Octane is in the building now....
-- Doctor Bob
After the 2.4 kernel comes out, and Redhat releases a distribution based on it, Redhat should get Mindcraft to do another set of benchmarks of Windows 2000 (which may be out by then) vs. the latest Redhat distro.
The hardware and benchmark configuration could be specified to something more realistic, but still high end - other posters here have described many ways this could be done. (Gigabet ethernet, dynamic content, stability testing, etc, etc.)
I presume Linux could win that handily. Now, as many people here are pointing out, benchmarks are kind of meaningless relative to day-to-day real life situations... but I'd still like to see the headline:
June 4, 2000 Redhat Press Release: "Mindcraft Benchmarks show Redhat Linux 7 faster, more stable than Windows 2000"
"In Mindcrafts recent testing of the latest releases of Windows 2000 and Redhat Linux, Linux was the fastest for for web serving and file serving... Redhat CEO thanks Linux developers world-wide... We're the best for servers, the desktop is next... "
Ahhhh yes. Sell MSFT now, beat the rush.
Torrey Hoffman (Azog)
Torrey Hoffman (Azog)
"HTML needs a rant tag" - Alan Cox
If you want one example, read the Mindcraft benchmark. For "NetBench Enterprise Mix" benchmark, old version of RedHat (there was at least one version newer at the time of the test) outperformed latest NT (at the time of the test) at speeds under 127.7Mbit.
Unless I am really mistaken, this is beyond the speeds of FastEthernet (100 Mbit nominal, less in reality). For all practical reasons, unless you are a MAJOR web provider, chances are you will not have the net bandwidth for your server and clients to exceed 127.7Mbit. And besides, MAJOR web providers probably won't use NT anyway,just ask Microsoft why they are using BSD and Apache instead of NT on some of their sites.( See www.hotmail.com for one example )
As for uptime, I've run Linux and NT servers for several years at my last place of employment. I have NEVER had a production Linux box crash on me. The only time Linux boxes went down is when we needed to move it or swap out a UPS. NT servers on the other hand required a re-boot for one reason or another (a good deal of this is crashes and freeze ups)at about 30 day average interval. (some crashed almost daily)
The post above explains why the Mindcraft's bechmarks do not reflect real-world performance. Moderate it up!
___
If you think big enough, you'll never have to do it.
Then go to http://www.toysrus.com to see a huge, balanced cluster of top of the line NT boxes in a real world situation.
We all know what the result will be, but it's fun anyway.
Foamy
Tomorrow's headlines in the basketball press:
Big Fat Guy outperforms Lean Scrappy Dude in Basketball Competition
-- $SIGNATURE
You guys are trying to compare apples and oranges. A Sun box is not at all like and Intel. Why? Suns are made for one purpose only: moving data from the hard-drive to and from the network card really fast. This is why they make great file servers. Intel is talented at CPUs. They make great processors but they are not in the business of making buses. Sun is.
There are other CPUs out there too: Digital Alphas are cleaning up on Seti@home. This makes them better at things like CGI scripts. G4's are supposed to have incredible floating point processors. This makes them better at things like render farms and engineering workstations.
If you want my advise, shop around before you spend piles of money on hardware. Don't buy and F18 if you need to take out enemy radar.
Ozwald
The story is five months old, and it appears that people have forgotten a couple of things.
1. MindCraft chose the only hardware configuration where NT beat Linux:
Around the same time, a more comprehensive set of tests were run by c't:
http://www.heise.de/ct/english//99/13/186-1/
The c't tests showed that Linux beat NT under most conditions. It was only in a specific situation (serving multiple saturated high speed lines from a single box) where NT was substantially faster than Linux. Non-coincidentally, it was this specific situation that MindCraft chose to benchmark.
Thus, MindCraft may have run the test fairly, but they rigged it ahead of time when they chose the hardware configuration.
2. The results are ancient history:
The Linux problem that caused it to be slower (but not slow) in the MindCraft configuration was debugged and fixed within weeks of the test (thePC Week rerun, that is).
If you want a valid comparison today, the test would have to be rerun using the new Linux code.
Why is Slashdot running a repeat of 5 month old story anyway?
For multitasking, running several services on the same machine, and for reliability, NT is junk.
They can prove that NT is purely faster 100 times, or until the cows come home, I still wouldn't use NT Server to even host my homepage.
Pure, raw, speed was never the issue in the NT vs. Linux debate in my sphere of thinking. I need a Network OS that can do many things at one time, and not crash ... ever. Linux us much closer to this than is NT.
The Sony Playstation OS is faster than NT in many situations, and could probably be made to run a webserver pretty damn fast, but who would run a business on it?
Fast, but horribly unreliable. Who would possibly see this as an acceptable alternative to *nix?
I guess using your defination all software that ever releases a service pack is beta then, right? Linux must never be complete since there are nearly constant kernel updates right? The hole 'beta' concept has been twisted it's virtually meaningless these days.
It is a matter of considerable widespread opinion that NT was released WAY before it was ready as is the way of Microsoft. In many ways YES... it's still in beta. And will likely continue to be for some time.
IMO, Kernel 2.2.x is also still somewhat beta, and that's an opinion that seems to be shared amongst some linux kernel mailing list occupants. I can understand Linus' motivations for moving it into the 'stable' category, but lets face it, when 2.2.0 hit the scene it needed a lot of work, and in some areas still does. Doesn't mean it isn't GOOD, just means it hasn't become a fixed (as in, never modified) code base. So no, linux is not, nor (hopefully!) will it ever be, complete.
The whole idea of alpha->beta->'complete' are completely arbitrary lines of demarkation anyways. It's hard to twist something where everyone has a different opinion. If you like NT and think it's not-beta, fine. You'd be in the minority among those 'in-the-know', but noone's gonna stop you.
so you're saying that a Linux kernel custom built for the Mindcraft tests would make a fair comparison? That's about what I expected from /. posters, it's not a fair test unless it's biased in your favor.
A fair comparison would take into account all variables, including TCO, support, et al. If you're ranting on about a purely TECHNICAL comparison, then yes, a custom kernel built to better our standing in the mindcraft survey would indeed be A Good Thing. There's obviously shortcomings in the realm of threaded TCP/IP and multiprocessor support, one of the areas NT shines and where, alas, Linux lags behind for now. Thus, when the kernel can hammer those stats, that area of the kernel will have to have been improved and I'll be able to reap the benefit.
And where did I say the tests weren't fair, trollboy? If anything, we should be thanking the mindcraft people for pointing out areas needing work. I fully expect solutions to these issues will be along in due course.
<sarcasm variety=oozing>"That's about what I expected from lame /. trolls. It's not a fair post unless it's biased in MS's favor."</sarcasm>
Shouldn't you be in school anyways? Run along before you get detention.
--
rickf@transpect.SPAM-B-GONE.net (remove the SPAM-B-GONE bit)
"People will pay big bucks for the luxury of ignorance."
I don't know why people get upset about these benchmarks. If Linux has some shortcomings, isn't it best for us to know about them so they can be fixed? Besides, to me benchmark tests are irrelevant. I run both NT and Linux at work everyday. All that matters to me is that my Linux machine has been up ever since I started working here (running Caldera's Open Linux). My NT machine must be rebooted several times each day. I am doing the same work on both machines. Bottom Line -- What good is speed to me if it's not reliable?
Actually I'm in the UK, all I'm trying to do is see if I can stop the damaging kneejerk reactions that go on after every benchmark *grin*
Not at all, all 4 were running anyway, whilst they were in there for redundancy, we used them for speed as well. 2 nics meant a lot faster connection to the stock market, and speed is always good!
Please note that the significant results, the ones which actually tell one what happened, are buried two-thirds of the way down the body, in Figure 3.
Linux on a single-processor system was honestly outperformed by NT, with NT turning in 152% of the Linux performance on a single processor, server.
From this, Mindcraft conclude that their original comparison on 4-processor servers was fair, and that there is nothing wrong with claiming that NT will give you 200% of the performace of Linux.
The Linux (and Samba) team wished to reperat the test on a uniprocessor because Linux isn't particularly fast on a multiprocessor, and that was one of the concerns the team had with the first test.
This issue isn't addressed by the article. And why not? Well, I'll suggest that it was one of the major ways in which the benchmark varied from fair. Since the paper on the retest was such a paen to Mindcraft's fairness, it wouldn't do to admit to any remaining unfairness.
Despite being able to beat Linux in a fair fight, Mindcraft implemented the benchmark in a manner that suggested they feared the very opposite. Sigh...
davecb@spamcop.net
So, Mindcraft, to jog your memory, your credibility was in question because of you tested an unrealistic situation that is *known* to have its bottlenecks in different places than the more typical situation, and then you extrapolated the conclusion "NT is faster" from that unrealistic scenario. Your credibility was not in doubt because of the performance achieved in that unrealistic scenario.
Shame on RedHat for helping Mindcraft run another instance of the unrealistic scenario and letting them shift the topic again to the wrong points.
But, thanks to Mindcraft's tests, one admitted weakness of Linux was found (TCP/IP performance does not double when you add a second NIC). Thank you for helping debug the linux kernel, Mindcraft. It's being fixed. Luckily the fix will be in place long before your unrealistic scenario actually ever enters common usage, assuming it ever does.
Don't label something "offtopic" unless you know the topic well enough to tell what's on topic.
http://www.zdnet.com/sr/stories/issue 0,4537,2387282,00.html
These boys did a slightly different kind of test with NT vs. Linux.
Conclusions are up to you.
Knut ------------------------------- Crazy is as crazy does
My guess is at the time of the test their quad Xeon machine ran about $50,000, now down to $30,000 or so.
For $1000 to $2000 per box, I can build a Linux eddieware system that has redundancy, reliability, performance, and REAL scalability. Let's all just sit back and imagine a web server farm with 20 times the horsepower of slashdot taking on that poor little quad Xeon machine.
There, doesn't that feel better?
--- It is not the things we do which we regret the most, but the things which we don't do.
But they still didn't help Linux any.
# procs, client, NT > Linux by:
"1 Windows 95 1.5 times
4 Windows NT 1.9 times
4 Windows 95 2.7 times"
On 1 proc, Linux should at least be the same as NT for serving NT clients -- but they didn't show that one. Instead, they showed the NT clients when Linux had the 4Proc config (2.2.x SMP is not as good as NT's SMP). This is sneaky -- omitting the best possibly situation.
Besides that, quad-proc systems are, well, rare.
Could we get some submissions from real world IT shops that serve ~200 to 2,000 clients, and see what their loads are. Then, test various configurations of NT and Linux to see which serves best?
---
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
Ok people I can't understand this. Why this Mindcraft crap has put everybody in the wheels???
Windows NT better than Linux? That's questionable in detail but the general result is that Linux IS better than Windows. Sorry Windows fans, but I had some serious troubles with Windows stuff to say good words from it. Both on workstation and server side. And in the same hardware Linux outperformed NT in every detail, except beauty. However Linux is not a solution for all. Frankly a good professional should measure what OS is better for a specific task. In fact a lot of high-performance servers are better done on FreeBSD. If you need an Abrams-class server then it is better to use Solaris. If your server will look much like an autoban of data with a lot of warehouses, than choose Novell. If you have a lot of interface work and one-tasked server than Windows has a good chance to do the job. And Linux is a hybrid of a rocker/viloncelist capable of playing 7 instruments at once.
Oh! And don't forget about DOS. They are a Hell on small server systems. Easy, fast and good preformance...
Did you READ the Part III? The RH engineers chose to use RH 6.0 and Linux 2.2. They specifically said that they thought 2.3 was too unstable to get setup in time for the benchmark.
I'm much more of a unix-head than an NT person. However, I can still look at things objectively. This test seems pretty fair to me.
Most Linux servers in the enterprise would be running Samba, not exporting filesystems over NFS, since Windows is certainly the dominant desktop OS, like it or not. I think it's clear that NT does fileserve better than Linux at least for now.
The biggest problem with the test (IMO) is that Mindcraft didn't use a mix of static and dynamic pages. However, I don't think that serving dynamic pages would make such a dramatic difference.
Another suggestion that I've seen is to cluster Linux servers. Although I admit I'm not too familiar with clustering software, isn't Beowulf still in a very beta or unstable state? I didn't think that it was anywhere near a useful solution of yet. What would making a 4 server cluster do?
And the third point is TCO. Come on, these are people who will spend $20k on a server. How much do you think the $700 to buy NT server and maybe the extra $500 for licenses is going to affect them? IIRC, a unix sysadmin's average salary is higher than his NT counterpart, so you don't gain anything there. You have to watch Bugtraq for both OSs, and competently administrating a Unix server is just as hard as administrating an NT one.
The fact is, that right now, Linux, in my opinion, is not enterprise capable. I think Linux is extremely well suited to other network services (DNS, mail, possibly FTP depending on popularity of the site), and it is also very suitable for non-enterprise environments, but for those places where performance is vital, Linux doesn't work.
You need to take things in perspective; Linux was originally developed by hobbyists, and for the most part it still is. How many of us routine access to the kind of servers that Mindcraft was using?
Linux's original intent was for low-end and middle-end servers, and it fulfills those purposes well. Will it develop to be a capable enterprise platform? Probably, but it will take time. IMO, none of the free Unices can compete at the very high-end (FreeBSD has a better TCP/IP stack, but both it and Linux lack a JFS, etc), no matter how much we wish they could.
Redhat 5.2 is old. It didnt even come stock with kernel 2.2.6. If you just drop a new kernel into an old distro you run into alot of problems. You have to upgrade the rest of the software/drivers to support it. If you drop a 2.2 kernel into a 2,0 system it will boot and run. Just not right...and there will be error messages all over the place.
I have to return some videotapes...
Check out:
- faq.html#a14
http://www.mindcraft.com/whitepapers/openbench1
I don't know, but I definately don't like what they're saying.
As if every linux user were a 15 year old geek, anarchist, using swear words whenever he hears "NT", "Windows" and so on...
I definately don't like it...
-- The day Microsoft makes things that don't suck, it's the day they start making vacuum cleaners.
If they used the newest NT then why not the newest RH? And why use RH at all? Its slow next to Slackware and a few other lesser known Distro's
I have to return some videotapes...
Thank you for your input. You may rest assured that your comments will be taken into consideration in the development of the next version of the OS.
I remember doing a search on my company's intranet on Linux and coming to a page with links to the Linux sources. The page has a Warning telling OS developers not to look at Linux sources due to potential copyright/IP/etc problems.
Are MS (or any other company for that matter) allowed to peruse the Linux sources for good hacks and then incorporate those into their own OSes? Even to fix a benchmark? It would be cool if someone could clarify that. Perhaps it's an Ask Slashdot question?
my blog: good times, man, good times
Umm... the fact that Red Hat engineers participated doesn't mean they "did their job correctly", boy-o. Remember that Red Hat's latest releases seem to be competing with MS to see how big the HD and RAM requirements for a base install can go... these guys AIN'T known for efficiency, quality or what-have-you. They ARE known for making Linux a household name by commercializing it. And nothing more. Red Hat, their engineers and their software are hardly known as the pinnacle of technical quality. They're known as the guys who turned Linux into a business.
With spending like this, exactly what are "conservatives" conserving?
Linux Journal had an interesting article recently on Linux vs. Windows performance. As I recall, the same C code compiled on Linux was pretty consistently about 30% faster on Linux than on NT. What this tells me is that skewed scenarios like Mindcraft's are like my challenging my (genetically gifted) younger brother to an athletic competition. There may be times when I can beat him in certain situations of my choosing, but that is all. Similarly, carefully staged performances where NT can beat Linux only postpones the inevitable, when Linux's inherent superiority sweeps NT off the field.
Has there been a Linux vs. *BSD vs. NT shootout at a PC show?
If not, why the hell not?
I'd like to see a single SMP, multi UltraII SCSI disk'ed server tested with various types of clients at one of these shows. How would NT fare against Mandrake 6.1 plus latest Mandrake kernel and the modified Apache 1.3.9 as far as web and file serving speed goes. Or OpenBSD as far as security goes.
I've never seen an NT server stay up for more than 7 weeks. No stinking Mindjob tests are going to change my experiences with NT. Experiences with ultra expensive DEC Alpha and x86 hardware that is brought to its knees by NT. Whilst the same model of Alpha sitting next to the NT box, but running VMS non-stop.
What's more, if you spend the time looking, you can find ~30 other tests that show the very opposite of Mindshaft's tests. Interestingly enough, none of which are Mickey$loth paid for.
Get in the ring Bill.
War crimes, torture, lies, illegal spying... Would someone give Bush a blowjob, already, so he can be impeached?
their name is shit...
It doesnt matter if NT outperforms Linux by a factor of 10 to 1, Mindcraft's name is shit.
It doesnt matter how much they 'vindicate' themselves, or produce nice little graphs, their name is shit.
It doesnt matter if they were right the first time, and we have been completely unfair to them, their name is shit.
Its about time a bunch of M$ buddies felt what FUD is like. Their company name is shit, their reputation is shit, and fair or not, they will never vindicate themselves.
Such is life in a big FUD war. Get over it. Linux is slower than NT. Get over it. NT crashes 100x more than Linux. Get over it. I personally run *only* Linux, and even have a tattoo of Tux on my arm, and couldnt give a crap what Mindcraft or M$ have to say... I really couldnt care less. I dont care that MS is found to be a monopoly... it doesnt matter to me one bit....
Stop competing with M$, they are irrelevant. Stop worrying about Mindcraft tests, they are irrelevant and after the first round no-one will ever take them seriously again. Concentrate on what's important, developing free, open, stable and funky software. I dont care if it takes 10 years to get Linux good enough for the desktop. *STOP COMPETING* , its dangerous, and can only lead to crappy software. Take your time, make it stable, make it fast, and enjoy what you've created.
Bugger M$ and bugger Mindcraft
Simon
The real linux_penguin has Slashdot ID 101961. Anyone else is an impostor. Including Bruce Perens.
MicroSquish made it that way on purpose. Its a pain in the ass to administer.
:)
Get this, we wanted to run a cgi-script. Fine, look through the dumb-ass gui for IIS. Figure something that might work. Try it and no joy.
O.k, check the winhelp, ah-hah, must twiggle the thing-a-ma-doodle on the dongle dialog. No problem. We twiggle the thinga-a-ma-doodle on the dongle dialog, smoke a ciggie. But no joy.
Get on Microsofts web site, check their buglist. It says go muck around in the registry. O.k, we're fearless. We muck around in the registry. No joy.
We switch to Netscape (Apache not company aproved, so flame-off please, not my choice either). Administration o.k. cgi runs.
We switch to servlets in the end. Such is life.
But what kind of crap assed web server won't run a cgi-script out of the box? Simple, one that Microsquish wants you to run ASP on.
I've finally found the off by one erro
Of course, what is left unstated by both your concession and the Mindcraft tests is that there exist alternate configurations (e.g., a server farm) that will outperform either system in the Mindcraft tests, at much less cost.
Many people reading the Mindcraft results don't realize this fact, and thus don't realize exactly why the test is unrealistic.
cool, care to expand on "lightly souped"?
Using phrases such as "ungainly Solaris box". What the hell do you mean by "ungainly"?
multiple Pentiums are much more economical and also advantageous from the point of view of scaling
That's right, those Pentiums (pentium II?) scale really well. Just look at all those super cheap, scalable 8 and 16 way Pentium II machines available.
Correct me if I'm wrong, but Pentium II only supports glueless SMP to 4 CPUs, right? After that, we're back to custom ASICs (expensive,) losing the benefits of the economies of scale of available chipsets.
BTW, why are SCSI devices cheaper on Intel boxes? SCSI is SCSI is SCSI. I've used SUN CD-ROMs on my PC, and generic CD writers on both SUN and PC hardware.
There is no question that in the future we will see more and more SMP Intel machines (especially with Merced) taking over the work of ungainly and expensive Sun, HP and Alphas.
That "ungainly" word again! Can't wait for those cheap Merced processors to come out.
I was going to moderate your post to flaimbait.
But it was much more fun taking the bait.
Get a clue.
"TummyX" writes:
I don't know how to say this kindly, so...
In my considered -- and, I hope, reasonably well-informed -- opinion, you are quite simply wrong.
There have been standard APIs in Windows, ever since at least version 3, to do exactly that with .INI files. (Sure, just for strings, but any half-way decent class or function library wraps "GetProfileString", etc, in more user-friendly syntax that handles type conversions for you; e.g, Delphi's "TIniFile.ReadInteger".)
Speed matters a lot less than stability and maintainability in this case -- do you need to update your settings a zillion times inside a tight loop, or do you need to be able to back-up, transfer, and maintain them? Does it matter more if users get to wait a millisecond more on starting your application, than if some other application (or a random Windows crash) clobbers the settings for your app for some unlucky users?
Christian R. Conrad
MY opinions, not my employer's - Hedengren, Finland.
Christian R. Conrad
mail me at iki.fi ; same user ID as here