Slashdot Mirror

← Back to Stories (view on slashdot.org)

NT Beats Linux in Round 2

Posted by ryuzaki0 on from the here-it-is dept.

strat writes "PC Week ran it's own benchmarks, with Mindcraft, Microsoft and Red Hat. The margins were tighter this time, but NT still fared better. They specifically mentioned the lack of multithreading in the Linux IP stack as the main bottleneck. I wonder how 4.4lite would have fared? "

23 of 327 comments (clear)

  1. C't did their own benchmarks. by Gleef · · Score: 2

    The German computer mag C't decided to do their own, more detailed benchmarks. Their conclusion was that the PC Labs/Mindcraft results sound plausible, but under more realistic scenarios, Linux & NT are neck and neck, or Linux beats NT soundly. Find the English version of the article here

    --

    ----
    Open mind, insert foot.
    1. Re:C't did their own benchmarks. by PrinceOfChaos · · Score: 2

      They were not completely objective when testing CGI. ASP would be a better choice and ASP scripts could be created to match functionality of CGI scripts.

  2. Linux Vs NT by gavinhall · · Score: 2

    Posted by The Technical Revolutionary:

    Linux hasn't lost anything. Last time I checked I wasn't running Linux because it had the fastest web server, and I could care less how fast Samba is. We need to stop chasing Microsofts tail here! Microsoft is just trying to pull linux into their overbloated Marketing/benchmark/it don't matter if it works as long as it looks good on paper world! Linux is a good, free, stable and open minded/open source operating system. Its OK if they clean up some problems in the code, but not without forgetting what Linux is all about!!

  3. Re:Gotta start winning these.. c'mon, coders by gavinhall · · Score: 2

    Posted by saberk:

    I agree... This whole benchmarking shit is very annoying... The only thing Linux can claim now is stability. And can someone tell me who let the world know that Linux is faster than NT before testing and benchmarking it. Our (Linux) ass is kicked just because we claimed to be faster and now M$ is "proving" that it's not the case... Maybe it's time to focus more on tuning.

  4. Time to own up and be adults by Kostya · · Score: 3

    Ok, the results are in. The benchmarks, when administered properly, show that NT outperforms Linux in every category.

    We now need to prove the open source model instead of confusing the issue by nursing our bruised egos. I'm already seeing the "denial" posts that hark back to OS/2 days. "Well, they may have gotten better numbers in this, but that doesn't really matter." Yadda, yadda, yadda. What is wrong with you people? Can't you accept the results? Quit playing games and start making progress.

    Some would love to argue with me until they are blue in the face about how this doesn't mean NT's better, etc. Fine. That is NOT what I'm talking about. The benchmark has exposed an architectural flaw/oversight. We need to fix this in order to reach the performance numbers we need to be a server operating system. So let's do it. Let's fix it. Start downloading and start coding.

    I have linux installed on every one of my file servers. I will be able to fend off the criticism for now. I have faith in out community and in our talent. However, if we continue pointless arguements and pulpit pounding about how linux really is better, instead of making sure that it IS better, we're sunk!

    And if you can't code, you can always test. Get involved. This is your operating system.


    "Doubt your doubts and believe your beliefs."
    --
    "Doubt your doubts and believe your beliefs." -- Switchfoot, Ode to Chin
  5. Okay, so NT smokes Linux with several NICs... by Ralph · · Score: 2
    ...but you might want to read an "objective" Test done by the german magazine "c't".

    The (yep!) english article can be found here.

    Sorry, there is no (I wonder why) german page available, the results can be found in c't 13/99 on pages 186 and following.

    Short summary: This test shows, that Linux+Apache outperforms NT as a webserver if there is only one NIC to handle. It also shows, that the number of processors daoesn't matter that much, when using Linux as the host sytsem for apache.
    NT runs much better than Apache on Linux, when more then one NIC has to be served - which might be the result of not having a multi-threaded IP-stack under Linux.

    Read ;-)

    Ralph

  6. Something to consider. by cthonious · · Score: 2

    It seems that Micros~1 inspected the linux code and found a weakness. They then publicized a benchmark that exploited that weakness. Good strategy.

    Will linux ever win benchmarks against NT if Micros~1 just benchmarks NT against linux's weaknesses? They will just find another one.

    The good side is that Micros~1 is actually funding linux development by finding weaknesses. Perhaps linux will one day be perfect and there won't be any weaknesses left :-).
    The bad part is that linux may never win any benchmarks, since only Micros~1 can afford to test them.

    Eg., why haven't we seen a gigabit ethernet benchmark?

    --

    support gun control: take guns from cops
  7. Re:4 net cards realistic? by edgy · · Score: 2


    This is a very important issue.

    A German Computer magazine c't did benchmarks with Linux against Windows NT and found that Linux trounced NT in every case except when Linux had multiple ethernet cards.

    If you're using Linux as a workgroup or departmental server, there's no point in having multiple ethernet cards in it. In those cases, Linux is still faster.

    The smart reseller benchmarks show just how much faster Linux is when you're dealing with situations that most businesses will have to deal with.

  8. Reality check: by malraux · · Score: 5

    http://cs.alfred.edu/~lansdoct/mstest.ht ml

    Kinda puts the whole shebang into perspective. Watch out for the twist of irony at the end.



    Regards,

    --


    Regards,
    -scott
    1. Re:Reality check: by igjeff · · Score: 2

      You kind of hit on a good point here...lemme elaborate on it a bit more...you also kind of missed a point. :)

      You say a web server serving external clients won't bog the system...that's not exactly true...but it bogs the system for a different reason.

      The benchmarks were run with directly connected networks, no routing, no wan links, etc. What does this mean to how the typical web hit is processed? The typical web hit on a benchmark is (I would guess...don't have real numbers here...educated guesses) about 1 to 2 seconds long...the connection is made...the data is passed very rapidly because you're dealing with 100Mbps pipes all the way through, no routing, etc., and the connection is shut down.

      In a typical Internet web serving situation, each hit will be much longer lived. How long does it take, start to finish, to load a typical web page on a typical v.90 modem connection? 30 seconds? a minute? I may be high on these numbers, but the idea stands...its not going to be (for the most part) a 1 or 2 second connection duration.

      With the longer duration connections, you're going to have more apache processes running (remember, you have one apache process per hit...and apache processes aren't typically small), you're going to have kernel structures allocated, etc.

      The meaning of all this being...that 1 processor 256 MB RAM box, is likely *not* going to even get close to serving 6 t1's worth of data to typical Internet clients before it falls over.

      To generalize that further...while this benchmark has exposed some limitations of Linux in serving large amounts of data...and this guy's article does give some good perspective on it...don't take his conclusions on how much a linux box will serve without a grain of salt...yes, he does acknowledge that dynamic content has a huge affect on the serving...but that's not all that affects it.

      Yet another posting to point out the goofiness of benchmarks...yes, they have their uses, but they're pretty limited.

      Jeff

  9. Linux over MS by eponymous+cohort · · Score: 3

    When people choose to dump their MS OS and go with Linux, I've never heard "because Linux serves more pages per second than NT" given as a reason.

    The reasons are usually one of the following:

    Better Stability
    Tired of being locked into MS-only solutions
    No license fees
    Doesn't need an expensive hardware upgrade
    No annoying features like animated paperclips
    Linux doesn't pretend to be smarter than you, and then fail at it.
    No registry which causes more pain and grief than it actually solves.

    It doesn't really matter than NT beats Linux in this test. Sure we should fix the problems, but NT is still NT with all the liabilities which it has become famous for.

    --

    Of all the comments I've ever posted, this is definately one of them

  10. Linux performance analysis by crow · · Score: 4

    http://www.kegel.com/mindcraft_redux.html

    (Found at LWN.) This page is full of technical analysis of the Linux kernel and Apache, explaining a number of performance problems that the benchmarks brought to light, as well as solutions for many of them.

    It sounds like a repeat test in another month or two would bring things even closer. With khttpd, we might even win.

  11. NT smoked Linux - who cares... by mw · · Score: 3

    One of the most important points why more and more companies choose Linux/Apache
    has nothing to do with speed or price of the OS or the webserver. The really
    expensive part when running webservices are administrative costs. NT & IIS may win here
    for *very* small sites, and UNIX wins when it comes to housing more than a few dozen
    sites - enough to make NT and especially IIS unmanageable.

    * NT has no useful scripting, Linux has everything you can ever need

    * you cannot remote administer NT (Im not talking about fast connections here,
    (where you could use VNC), try to administer NT over a modem line. Good luck)

    * once youve made your decision to use IIS, youre completly stuck when it comes
    to changing to another type of webserver, or sometimes even when you want to
    transfer sites from on IIS to another. Microsoft has a tool to do that job, but -
    guess what - it crashes on even the smallest problems. Apaches configuration
    files can easily converted to another text-based config - use sed, awk or whatever
    youre use experienced with.

    * if something goes wrong with IIS, the event log will contain such useful error
    messages "could not bind instance XXX. The data is the error code. 43 00 00 6c".

    * IIS is a hell when it comes to logging. All logging is done asynchronously, so your
    only chance to see whats going on is to wait a few minutes for IIS to sync() the
    logs. Really a pain when you want to study the logs...

    * Sometimes under NT, the MMC console simply is stuck. Then your only chance to
    get it running again is to restart the system, simply logging in as a different
    user does not help. Very annoying.

    * Any finally, those beloved situations where those windows popup:
    "Your system is running low on virtual memory...". When you check the taskmanager,
    it will show you that neither applications nor the system itself seems to use
    that much memory. Again, your only chance is to restart the system.

  12. C'T magazine did their own tests... by kieran · · Score: 2

    ... and Linux came out marginally on top. Conditions were a good bit more real-world, and the article is pretty well-written. Read it in English at http://www.heise.de/ct/english//99/13/ 186-1/

    (Info found at http://www.lwn.net/daily/)

  13. Why Linux really lost. by iapetus · · Score: 3
    1. They cheated.
    2. It's not fair to use more than one processor.
    3. A zero got missed off the end of the Linux performance results.
    4. NT only outperforms Linux in non-real-world situations, like when both machines are turned on.
    5. It doesn't matter that NT was faster, because Linux is the right speed. Anything that goes faster is just dangerous.
    6. Once you reach a certain threshold (almost exactly where you get faster than Linux, in fact) reliability becomes more important.
    7. They used NT 4.0, and we only used Linux 2.2, so all the Linux scores should be doubled to make up for the version advantage.
    8. NT smells funny.
    9. Who cares about benchmarks anyway?
    --
    ++ Say to Elrond "Hello.".
    Elrond says "No.". Elrond gives you some lunch.
  14. It's finally nice to see .... by opencode · · Score: 2

    .... that the press is interpreting benchmarks as what they really are: a process implemented to improve results. In other words: OK, the benchmark has been done, and now the Linux community knows what needs to be done. I applaud the press for this interpretation, instead of something along the lines of, "looks like the Linux Community is 'pulling a Microsoft' by burying the results in a dainty little PR statement, and not actually addressing the issues..."

    Now if only the Linux Community can learn to implement this interpretation of benchmarks ....

    That is to say: folks, benchmarks are our friends, EVEN IF they're biased !! So what if the NT box uses an Apache server ... I'd like to see WINE use IIS, just to say that we've tried it -- wouldn't THAT be a hoot !!

    --
    "He who questions training trains himself at asking questions." - The Sphinx, Mystery Men (1999)
  15. I have a T3 line you can use! by DonkPunch · · Score: 2

    Oh, you wanted one that was actually connected to something? :)

    --

    Save the whales. Feed the hungry. Free the mallocs.
  16. Re:Not FUD, Fuel by remande · · Score: 2
    Micro$oft is now convinced that they are #1, and Linux will never be able to compete.

    Don't you believe it.

    If Microsoft thought that Linux would never be able to compete, they wouldn't have bothered to publish the test results. They think that Linux is so popular that bad press is better than no press; that it is no longer under the radar. Never believe that a company's marketing reflects the internal realities; that goes double for Microsoft.

    As near as I can figure, Microsoft sees Linux as the new platform threat, and will deal with Linux accordingly. If they're smart, they will realize that FUD cannot destroy Linux, but can certainly slow it down. When MS markets a commercial product out of existence, the war has a limited duration and is over when the product's vendor pulls it. Since nobody can pull Linux, marketing and FUD wars could last for decades. But in the meantime, MS may find it useful to slow Linux growth until they can organize a better defense.

    If we're lucky, MS will be stupid and try to FUD us to death. Linux can beat any FUD, because it has more long-term viability than any proprietary software; we have forever to make Linux kick ass. I'm not going to count on that, however; MS shipped all their stupid people to federal court.

    They may be able to embrace and extend popular protocols (like TCP). They can put their proprietary ware on top of the open source Linux kernel, though they can't do much to the kernel itself. One interesting strategy might be to port Win32 to Linux as Microsoft payware. Thus, they get to collect their tax as you install MS-Office onto your corporate Linux desktop. I don't know if this approach would be beneficial or harmful to them.

    Microsoft understands that Linux is a threat. They are allocating resources to deal with it. Don't turn your back on them.

    --

    --The basis of all love is respect

  17. Actually, That's Linux => 1, Mindcraft => 0 by Clark+Kent · · Score: 3

    Here's what the second benchmark showed:

    1. The Linux Advocates Were Right

    Mindcraft *did* seriously mis-configure Linux in the first test.

    The Linux/Apache peak performance in the second test was approximately 50% higher than in the first test.

    More important, the disasterous collapse of Linux/Apache at higher loads, that occurred in MindCraft's first test, was nowhere to be seen in the second test - Linux/Apache performance remained high as the load increased. The performance drop-off in the first test was caused by MindCraft's mis-configuration.

    2. The Anti-Linux Zealots Were Wrong

    Linux advocates did *not* oppose the first test simply because NT beat Linux. The opposition was based on valid concerns about how the test was run - concerns that have been born out by the second test.

    There is little serious opposition to the second test, which is generally considered fair (within the limits of the benchmark). In fact, the knowledge gained from the second test has been welcomed by the Linux community, who look forward to the performance gains that will result.

    3. NT/IIS Beat Linux/Apache - Not That It Matters

    IIS on NT *did* achieve a higher benchmark result than Apache (or Samba) on Linux. But, as many have pointed out, the conditions of the benchmark are highly artificial. In the real world, where there is a greater mix of activity on the server, Linux's virtues in the areas of stability, task management, and I/O performance would play a greater role.

    As some have pointed out, when you're shopping for a reliable delivery van, the fact that it can be beat by a dragster is of little consequence. Or, to use another car analogy, by pouring on the nitrous, you can beat any other car on the track - for one lap (before burning out your engine).

    4. Linux/Apache Performance Was Excellent - Not That It Matters

    As others have pointed out, the Linux/Apache performance on one CPU was enough to handle the load of *twenty* T1 lines. But again, the test is too artificial for that to have much meaning.

    5. That MindCraft Guy is a Whiner

    There are always jerks on every side of every issue. For him to pick out the most obnoxious things said by some Linux supporters, and suggest that they mean something, is childish. Those mouth-offs don't represent the Linux community, anymore than Ballmer represents the...NT...oh...wait... :^)

  18. Interesting spread of comments by eof · · Score: 2

    This is a really interesting spread of comments on this matter. We have on the one hand the Linux advocates (which I will happily count myself a member of). Opposite us are the NT advocates. Both groups make good (and bad) points, reflecting a range of reasons from excellent and well-spoken, to extremely close-minded. In short, it doesn't matter _why_ NT beat Linux so much as what it means to have this kind of attention. Yes, the hardware was more conducive to NT than Linux. Is this unfair to Linux? Nope. Does it mean that NT is better than Linux? Nope. It only means that NT outperforms Linux within the limited parameters of the test. Additionally, some limitations within Linux were uncovered that can be improved on. This is excellent news! All of this states quite firmly (to those who choose to look at the big picture instead of a few numbers, however captivating) that Linux and NT can be seen on a comparable scale. Not bad for an OS that scales well on a 386 or higher. Keep in mind just how flexible Linux can be, as well as how committed its development and support is. It's never shameful to be beaten in a contest. It's only shameful to refuse to improve yourself for the next contest. Peace.

    --

  19. Gotta start winning these.. c'mon, coders by drougie · · Score: 2

    Microsoft would never have pushed for these tests if they weren't 100% sure they would win. They surely anticipated our response to sketchy testing procedures, and sure enough, we continue to lose. Yeah, yeah, this isn't a drag race, but it would be nice if we could program software that wasn't this much slower than Windows when one of the aspects of Linux we've defended is speed and stability. So, let's do it.

    --
    Calling out bogus battery capacity claims.
  20. exact same URL as "NT beats Linux: again" by TheDeal · · Score: 3

    we had this story, a couple days ago: here... it has the exact same url. this is the same story. doh!

  21. Maybe there is another principal a work by JediLuke · · Score: 2

    Uh, maybe there is the fact that you can get Linux for free, as many users as you want for free, and run lots of free program, without big brother looking over your shoulder. Anyone know what NT Server w/25 clients costs...like $3500 i think?
    $3500 != free
    i think not...viva linux!

    --

    JediLuke
    -Do or Do Not, There is no Try