Slashdot Mirror
High Performance Network Applications
An Anonymous Coward sent in this: "An article over at SysAdmin magazine seeks the truth while comparing network application performance under RH Linux, Solaris x86, FreeBSD 4.2, and Windows 2000. I'm a little suspicious of the writer's results, but you be the judge."
The flaw here was that the tests relied on 'simple C++ programs' to 'evenly' benchmark the different OSs. The problem is, in the real world, this is not how serious large-scale web applications are written and the sorts of results that this study comes up with are effectively meaningless. Show me a transaction server (or object broker). Show me how the systems scale with thousands of simultaneous users. Show me web performance based on code that people are actually likely to write in real life, not the TCP/IP equivalent of "hello, world" and you may have something that may be of interest outside of the context of an assignment for an undergraduate CS course in networking.
I'm just sick and tired of these so-called "studies" which proclaim that they are, once and for all, going to end some religious battle. These studies do nothing for professionals or the industry, so why do people still bother?
As any professional will tell you, "it depends". Performance always depends on your needs, capabilities, money, skills, software, and hardware. Someone claiming that there is a simple answer by running some simple tests is just trying to either (1) sell consulting services, or (2) sell advertising space.
And nothing else.
So someone, tell me, please oh please, why I should pay attention to salesmen who claim to hold "answers". And tell me which CIOs really bite at these numbers. This is just for hit generation. Page views. These are not for me. They are not for the community. They are not for making good decisions.
They say:
/etc/system:
n ms 415/patch1/TuningGuide.html )
/etc/init.d/inetinit:
/dev/tcp tcp_keepalive_interval 30000
/dev/tcp tcp_time_wait_interval 15000
/dev/tcp tcp_conn_req_max_q 1024
/dev/tcp tcp_conn_req_max_q0 1024
/dev/tcp tcp_xmit_hiwat 32768
/dev/tcp tcp_recv_hiwat 32768
/etc/vfstab, like this:
/dev/rdsk/c0t1d0s7 /opt ufs 2 yes logging,noatime
/etc/sysctl.conf:
> At Lyris Technologies, we write high-performance, cross-platform,
> email-based server applications. Better application performance is
> a competitive advantage, so we spend a great deal of time tuning all
> aspects of an application's performance profile (software, hardware,
> and operating system). Our customers frequently ask us which operating
> system is best for running our software. Or, if they have already chosen
> an OS, they ask how to make their system run our applications faster.
> Additionally, we run a hosting (outsourcing) division and want to reduce
> our hardware cost while providing the best performance for our hosting
> customers.
What a crap! They're claiming to be experts! Ha!
They just don't know how to tune Solaris or FreeBSD properly.
Results will be completely different if they've tuned it well.
Solaris Tuning Guide.
1) Apply latest recommended patches from http://sunsolve.sun.com
2) Add the following to the end of
* Raise TCP connection buffer size
set tcp:tcp_conn_hash_size=262144
* Increase various kernel buffers
set maxusers=2048
* Set hard limit on file descriptors
set rlim_fd_max=1024
* Set soft limit on file descriptors
set rlim_fd_cur=1024
* Increase directory name lookup cache
set ncsize=100000
* Should be the same as setting above
set ufs_ninode=100000
* Enable priority paging
set priority_paging=1
(These settings are based on information taken from:
http://docs.iplanet.com/docs/manuals/messaging/
3) The following should be at the bottom of
# TCP stack tuning
# default is 7200000
ndd -set
# default is 240000
# change to "tcp_close_wait_interval" on Solaris 2.6
ndd -set
# default is 128
ndd -set
# default is 1024
ndd -set
# default is 8192
ndd -set
# default is 8192
ndd -set
4) Speed up filesystem access under Solaris 2.7 and later.
Add logging to filesystem mount options in
/dev/dsk/c0t1d0s7
I have added noatime - this is another setting that might help
on very busy filesystem, but not that much as logging.
FreeBSD Tuning Guide
Recompile kernel with increased number of MAXUSERS (good number
to start is 256) and NMBCLUSTERS (I use 10000, see netstat -m
under load to get number that good for you).
You might want to play with "options HZ=1000".
Add this to
kern.maxfiles=65536
kern.maxfilesperproc=32768
net.inet.tcp.delayed_ack=0
net.local.stream.recvspace=65535
net.local.stream.sendspace=65535
net.inet.tcp.sendspace=65535
net.inet.tcp.recvspace=65535
Turn on softupdates on all filesystems
using tunefs -n enable (noatime might help as well).
Vadim Mikhailov
So you're saying that if you want good performance from Linux, you just code it normally - but if you want good performance from windows, you have to use all the platform dependent nonportable operating system extensions.
It might not be a valid benchmark, but perhaps there is a point to be learned from it after all...
-dentin
Alter Aeon Multiclass MUD - http://www.alteraeon.com
You said:
3. They only tuned the Linux, FreeBSD and Solaris setups -- they should have tuned Win2k server as well.
Well, that's not a fair assersion. They did exactly 1 modification to each unix kernel: Change the number of file handles. They set each of them to use 65536. IIRC, windows2k doesn't need this tweak due to it's internal way of record keeping.
The greatest problems with benchmarks is what tweaking to do. Out of box tests fail because "Any competent admin will use tweak foo", and tweaked tests fail because "tweak foo on os1 is vastly more potent than tweak bar on os3." (think the first mindcraft test).
Zapman
You're absolutely right. Their "benchmark" is perfectly valid, for their product running on a naively tuned operating system. But only a neophyte would put an out-of-the-box OS -- whether Linux, Solaris, Windows, or BSD -- into production as a high-performance network server. All the complaining boils down to two things:
The FreeBSD folks are especially upset because the article states that the OS was logging resource failures but the testers still didn't perform any tuning. That's an amazing level of incompetence to display in a magazine which is supposed to inform system administrators.
Now do you see what all the noise is about?
Agreed -- it's been a long time since I've seen a "benchmark" as poor as this one. But I don't think Windows was treated any more poorly than the other OSes. It wasn't a fair test of any of them.
The "tuning" for the Unix systems consisted in bumping up the maximum number of file descriptors. That's it. The FreeBSD system in particular was left completely mistuned and clearly running out of socket resources -- they report that it was logging errors but seem entirely ignorant of what those errors were (beyond their being load-related) and how to correct them.
Polling is hardly the best system interface for multiplexing TCP connections on either Windows or FreeBSD. As you mention, completion ports are best for Windows. Kqueue is best for FreeBSD. It just happens that polling is used in the crappy commercial SPAM program they "benchmarked". (All the OSes support scatter/gather, BTW, so you can't claim Windows was treated unfairly by its omission.)
None of the systems were testing in a way that shows their actual capabilities. The article is just a thinly disguised commercial for a (barely-)cross-platform "bulk email" product.
The architecture they say performs the fastest, One-thread-many-tasks (asynchronous), is exactly the one encouraged and supported by my StreaModule system. I knew that things worked out this way, but I'm quite surprised to find such clear agreement by a third party. This idea doesn't really seem to crop up in many places.
Need a Python, C++, Unix, Linux develop
You're an idiot. They're using the 'poll' system call. If you bothered to read anything, you'd realize that 'poll' is the way to do asynchronous event based I/O under Unix. It's close to what 'WaitForMultipleObjects' does under NT.
They may use the sockets API, but as far as I know, that's the way to do TCP/IP under Windows. There are a few special calls to get NT 'handles' for your sockets so you can then do WaitForMultipleObjects based event based I/O handling. I'm betting this is exactly what they did.
As for scatter gather buffering, that depends a lot on your internal application architecture. I would agree that, in general, it's a good idea. I don't think their code would do scatter gather under Unix, and not under NT. Scatter gather is implemented nearly identically under both platforms.
Your comment shows a great deal of ignorance. It's a travesty that you were moderated to +5. *sigh*
Need a Python, C++, Unix, Linux develop
You misunderstand 'poll' completely. poll asks the OS to suspend your process until one of the indicated events happens, then you get to go respond to it. It's essentially the same thing.
Say, for example, that your dumping data into a socket. Under Unix, you write to the socket until the OS tells you that the socket buffer is full by setting the socket to non-blocking and writing until write returns EAGAIN as an error. Then you put the ability to write to that socket on the list of OS events you're interested in. Then, you go do whatever else it is you have to do. After you get done servicing everything you can service, you call poll and it blocks your process (possibly running others) until one of the indicated events happens and there's something else to service. Same basic paradigm.
Need a Python, C++, Unix, Linux develop
Also, VirtualAlloc there sounds and awful like like 'mmap'. Again, same basic idea, and Microsoft does it completely differently.
I know a fair amount about the insides of NT, and most design choices they made that are different than Unix's are worse.
Here are just two:
Need a Python, C++, Unix, Linux develop
To say an OS's network or disk performance is poor, without considering the drivers used for your hardware, is kinda irresponsible.
It's clear, as your comment shows as well, they did not make any effort to properly tune and configure the overall system for each OS tested.
I AM, therefore I THINK!
I'm sure Linux will talk just fine to Linux, but other platforms might not be tuned the same. (2.4 kernels were having trouble because of this recently. Linux implemented some feature that lots of routers didn't, and performance was hosed somtimes.)
/. page? ECN is supposed to help avoid that.)
You don't seem to understand ECN. ECN is now (as of June 12) an internet standard. It will improve the performance of the Internet by allowing ECN-aware stacks to note congestion and respond appropriately instead of waiting for packets to fail to be acked and backing off one the transmission speeds. (Ever got a 'stalled' message loading a
Buggy routers responded incorrectly to ECN packets by terminating the connection. It appears as if the other computer isn't even on the net. Cisco has released bug fixes to correct this bug. They have not been applied by all of the admins.
Yes, Linux 2.4 shipped with ECN enabled. The distribution packagers generally (all?) included a command in the start-up scripts to disable the feature.
Because TCP/IP is a standard, there should not be performance differences between stacks whereas a stack performs better speaking to another stack of the same design. TCP/IP should be completely interoperable.
I have discovered a truly marvelous sig, unfortunately the sig limit is too small to contain i
An IO Completion Port is just a thread pool blocked on a counted semaphore to call select() or WaitForMultipleObjects(). If you set the initial semaphore count to the number of processors, then the OS scheduler should efficiently pin each thread processing a select/WFMO event to its own processor.
cpeterso
They used all three of these systems "out-of-the-box". So look at what these systems are pre-tuned out of the box for: Redhat Linux for speed and FreeBSD for stability (and Windows Y2K for media benchmarking). Think about it.
A Government Is a Body of People, Usually Notably Ungoverned
So, based on using 1 of the dozen or so filesystems linux supports, you determine it's
crap?
Try reiserfs....
I bet that 18 gigs takes *forever* to fsck if you reboot...
crappy benchmark, to say the least.
IF the question is 'which network stack is fastest' there are ways to sort that out. 'which is better under high load'.
There are so many questions that can be asked...
And any of the systems tested are capable of blindingly fast network operations if the programmer takes into account the best way to do things on that particular machine.
Compiling the same code on 4 machines and testing the output is more of a compiler/libarary benchmark than a system benchmark.
While your point that this benchmark is somewhat flawed is correct, you also point out a large problem with Windows:
You are forced to use proprietary MS-only extentions rather than straight, standardized POSIX calls to achieve the best performance. That means you have to suffer proprietary lock-in if you want to code high performance network applications for Windows.
I think is deliberate: there is no reason why calls like malloc, creat, mmap, poll, whatever, couldn't have been tuned to get similar performance to the Windows specific VirtualAlloc, CreateFile, etc. Microsoft wants you to trade off portability for speed.
It wasn't tuning per-se, just the raising of maxfiles because Unix defaults to lower settings. They point out that it wasn't necessayr under Windows, presumably because the equivalent is uncessary.
--
--
My comments and opinions completely reflect those of anyone and anything I am remotely associated with.
If there were a Geek Speek generator on the net similar to the Mission Statement generators, that's what it would sound like.
How embarrased you must be.
--
--
My comments and opinions completely reflect those of anyone and anything I am remotely associated with.
That doesn't mean that if_fxp is a poor driver currently. Everything can be improved however which is what the mii rewrite is doing. if_fxp is already a very excellent driver and is the best card/driver combo under FreeBSD (and probably most OS's).
--
--
My comments and opinions completely reflect those of anyone and anything I am remotely associated with.
Ok, so that's what it looks like. However, I did an awful lot of benchmarking at my last job to get our performance up on our hardware. So what I benchmarked was our software. It was a modified version of the apache webserver. So I had extensive results from the use of our product, and virtually NO results for SQL servers, spreadsheets, 3d-games or email applications.
I just think these guys did their job (optimizing their software), and ended up publishing their benchmark results to enlighten other people. I wish I had gathered up everything and put it out there.
Basically we tested a version of apache on BSDI 4.01, redhat linux 6.2 and solaris 7. The systems were compaq 1850r p2 450x2 boxen. BSDI needed a LOT of tweaks, but ended up being the most efficient. Solaris was pretty stable, but a little slower. Linux was about the same performance as BSDI... sometimes. Sometimes it would flake out at high loads. I'm sure it's much better now, especially with tux.
In reading the top-moderated comments, one thought came to mind: Slashdot readers, who are accused of being rabid Linux supporters, are bashing a benchmark that came out pro-Linux.
Kudos to the Slashdot community for being objective, despite your theoretical biases.
So you're saying that if you want good performance from Linux, you just code it normally - but if you want good performance from windows, you have to use all the platform dependent nonportable operating system extensions.
If that were the case for Linux, the Tux guys wouldn't be trying to put an http daemon in the kernel. They'd just keep it in user-mode and 'just code it normally'
Simon
Coming soon - pyrogyra
I think is deliberate: there is no reason why calls like malloc, creat, mmap, poll, whatever, couldn't have been tuned to get similar performance to the Windows specific VirtualAlloc, CreateFile, etc.
... apart from the fact that they expose different paradigms entirely?
Malloc - heap based allocation
VirtualAlloc - allocates entire pages from the VMM. Allows you to reserve or commit pages when and as you need them.
fopen - opens a file handle
CreateFile - Allows you to open a file handle, specifying buffers to use, etc etc etc.
poll - you sit there waiting and doing nothing most of the time because you're asking all your connections "are we there yet?"
CompletionPorts - the OS comes back to you when it's done, and tells you that it's finished. You can now use those spare cycles doing something else - like another 1000 network connections.
Simon
Coming soon - pyrogyra
Nice! So in other words, they used straight BSD sockets for their
implementation - which is NOT the way to get performance from Windows. You
need to use:
1. Asynchronous, Event based socket handling.
2. Completion ports.
3. Scatter/Gather buffering.
Polling is lousy no matter what way you do it. You'll lose most of your
performance spent going round a small loop.
Similarly you can infer that they used straight malloc() for their memory
handling, and most likely file handling - again very lousy
performance-wise on windows compared to the alternatives, such as
VirtualAlloc, CreateFile(), scatter-gather file handling and more.
As for the second test, we can guess (from their comments) that they're
using straight C++/C file operations under windows instead of tuning them to
the architecture, so of course performance is going to be lousy -- they're
benchmarking Microsoft's C runtime implementation, nothing more, nothing
less.
Also note that:
1. They don't provide details of which compiler they're using.
2. They don't provide details of the actual benchmark code for test 2.
3. They only tuned the Linux, FreeBSD and Solaris setups -- they should have
tuned Win2k server as well.
Sheesh. Talk about a crappy way to benchmark.
Simon
Coming soon - pyrogyra
Anyone else notice the heavy concentration in that article about the efficiency of mailing out large numbers of email messages. Now, I'm certain there are many MANY legitimate reasons why someone would have a "test list" of 200,000 email addresses, its just that I can't seem to think of any at the moment.
-Restil
Play with my webcams and lights here
In case you didn't notice, NT is not Unix, never has been Unix and never will be Unix. There are so many design differences in the underlying system that it is hard to believe you are even suggesting it's a good idea to use a single code base.
I've seen plenty of code that has a single source for Unix and NT, but NONE of it is high performance and most of it behaves very strangely on NT when you compare it to a properly written NT service or application.
If you are writing high performance code then you are almost certainly writing for a particular system and have to write the code for that system. Writing for NT is different to writing for Unix (I prefer writing for NT personally but that's another issue) and trying to say that it's lock-in is just stating the obvious.
By your argument Linux is deliberately encouraging the use of non-portable code through applications like Tux which only work on Linux boxes and not Windows, or even other Unixes.
It's just daft.
Fear: When you see B8 00 4C CD 21 and know what it means
select() is a pig of a system call (IMHO). Replacing it is probably not a good idea though - you'll break compatibility all over the place. I believe this is discussed every so often in the Linux kernel and improvements are slowly being made here and there with single thread wakeups and the like.
:-(
There's plenty of evidence that Unix is every bit as fast as NT with it's API. My point is that just because NT supports the BSD sockets API doesn't mean you should use it in high performance code. Personally I prefer using ReadFileEx() and WriteFileEx() on NT for socket (and file, named pipe and everything else) I/O. If you do it properly you don't have to wait for any events at all - the system just calls your completion routine when it's finished all by itself (kinda like a signal).
Winsock 2.x is good now that socket handles are full blooded file handles. Damn shame you still can't pass them as stdin or stdout to a child process though.
Fear: When you see B8 00 4C CD 21 and know what it means
I'm not sure. It looks like they've tried to use the same methods on 4 different operating systems. This is something that is doomed to failure in a benchmark situation as there are different programming paradigms for the different systems.
A much better benchmark would have been simply comparing IIS to Apache or Tux. Oh yeah. That's been done. Tux won. Hehe.
Fear: When you see B8 00 4C CD 21 and know what it means
True. There's another way that's also very fast in NT that would be really difficult to emulate on Unix (probably because it wouldn't be fast on Unix):
To set this up you treat the sockets as file handles and use ReadFileEx() and WriteFileEx() with the lpCompletionRoutine parameter set to point to a function that the OS should call directly when the I/O is done. When you are blocked waiting for activity, put the thread in an alertable wait state using *WaitForXXXObjectEx() function and the completion routine you specified will be called by magic (actually via an Asynchronous Procedure Call or APC, but close enough to magic) when the I/O has finished.
This works very quickly on NT because it mirrors the way the underlying kernel and device driver stack works. Basically the I/O completion can come straight up from the driver routine into user space with a minimal delay and minimal number of context switches. The second advantage is you don't have to open event handles for every I/O you have outstanding, and so you don't run into the limit of waiting on 64 objects at a time.
The only drawback to this method (if you can call it a drawback) is that I/O that is initiated on one thread is always sent back to that thread so you have to run one thread per CPU and round robin them
The closest thing on Unix to this sort of behaviour is signals, but signals and multithreaded code tend not to mix very well.
Just a FYI really, not saying it's good or bad compared to Unix - just another thing to have in your bag of tricks.
Fear: When you see B8 00 4C CD 21 and know what it means
The method used here for programming Windows 2000 is almost certain to guarantee slow results. Assuming he's written his code to use select() or even WaitForSingleObject() then he's signifiantly slowing down the system.
If you want to write high performance socket applications on Windows you MUST use I/O completion ports (something this article failed to mention at all). Most high load applications I've written using sockets have shown a 50% to 100% improvement in throughput for the same CPU load when switching to I/O Completion ports from a tradition (Unix style) asyncronous I/O model.
I'm not saying in this case that Win2k would beat Linux, just that the tests were skewed by the author's inadequate knowledge of writing high performance code on Windows 2000.
Fear: When you see B8 00 4C CD 21 and know what it means
Thank you for agreeing with my point, but coming to a diffrent conclusion. As far as developing for all systems... Does it really matter? can you really tell me 'all coders are created equal'? Isn't it more logical to conclude the majority of development is done on one system, and then 'ported' to the others (why else do we have portable code?). Most coders would probably aggree a native version is always going to take more advantage of that systems abilities than a simple port.
It doesn't really matter in this case, however.
(shrug) As I tried to point out, the results are only relevent if you are using Lyris' software, hence my conclusion that this is a comercial, not a test. They even say, right in the article, that this is a result of testing customers have asked for so they know which platform is best suited for THIER apps. The results are wrapped up in something that is guarenteed to cause controversy at the 'religous' level. I say, well done to the marketing weenies.
I read this a couple of weeks back when a linux-centric friend sent it to me... my main observation: This is Obviously a comercial masquerading as a 'test'. When the 'device' being used to do this so called 'benchmark' is a software application written by the testers for something else, there is nothing else to call it. Maybe the title of the article is a bit misleading, the meat clearly says all they are doing is showing which OS they have optimized thier application for. They then use that as the FLAWED basis for determining which OS is 'best'? Give me a break.
My own complimentary subscription for presenting at LISA '99 just ran out, but as anyone who's read this journal before can tell you, this article was just written by Joe Admin, and was about on par for the magazine. Even if you haven't read the journal before, you could click on the big "Write For Us" link at the top of the page, and see that "all of our articles are written by readers."
Now, I'm not slamming the magazine! It's a decent piece of work, and actually has some good articles about tricks and tools that help sys admins get their day to day jobs done. But at the same time, it's also subject to some one-sided reviews and some articles take a lot of flak for their controversial positions. Just look at who wrote the article (the original developer of the mail engine) and take it with a grain of salt.
And if you really disagree write them a counter piece, or at least a letter to the editor pointing out the flaws.
The reasoning behind I/O completion ports is that it permits you to do something on completion of I/O, such as initiate yet another I/O (a "feedme" signal that is delivered reliably as an event, unlike a UNIX signal, which is merely a persistant
condition).
It's clear from their comments that they did not turn on Softupdates on the filesystems when they set up their FreeBSD machine for the testing. It's no wonder that they found disk I/O to be slower on FreeBSD, therefore.
Traditionally, Linux has traded speed for safety in filesystem meta data handling. FreeBSD has always refused to do so, insisting that metadata be updated synchronously. With softupdates, the metadata is cached, but the cache is flushed in the right order. The upshot is that you get the speed and the safety.
In short (too late), I am sure that their opinion of FreeBSD would improve markedly if they would set it up properly.
From what I see, just about every other OS represented has a defender saying exactly the same thing. That doesn't speak well for the thoroughness of the testing. I'll leave it at that.
I was surprised that FreeBSD didn't show better in the "benchmark", too.
While I'm a Linux user, I've always admired the real world performance of ftp.cdrom.com, a FreeBSD based site, IIRC. It would handle legendarily humongous loads of network connections and file transfer (bytes/day) on cheapo x86 hardware.
Comparing to the other OS on the above review, I got the impression that FreeBSD meant not only getting the free{beer,speech} advantage over W2K and Solaris/x86, but reliability (which Linux has), but, as of several years ago, a significantly better network and VM performance than Linux.
I was impressed.
"Provided by the management for your protection."
Obviously this "test" is a crock, as FreeBSD would not be that far behind, what with it's surperior network stack and thread handling.
This sounds very much to me like "I found this benchmark's results surprising, therefore I rejected them." You've given absolutely no evidence to support your claim that FreeBSD's network stack and thread handling are superior. You've said nothing about what mistakes may have caused their benchmarks to be skewed. If you are going to reject results you don't expect, what's the point of running the test?
I am not biased. I have all of these operating systems installed on my machines. The desktop I am at now is running Linux 2.4 and also has a copy of Windows 2000 installed. My colocated server is running FreeBSD. The old SPARCstation in the basement has Solaris (though, admittedly, I don't really use it).
It may be true that this is a very bad benchmark...but don't reject the results simply because they surprise you. Look into it...what was wrong with their test procedure? Without an answer to that, you have no credibility.
That's true. Also keep in mind that Solaris is ass-slow on IDE systems.
If they were testing on an Ultra-5 or 10, that would make Solaris look lousy.
Conformity is the jailer of freedom and enemy of growth. -JFK
I was going to read this article and make an informed comment about it. But, because of my laziness to wait forever for it to load, I'm just going to post this summary of comments to come:
Linux users: Linux is better, Windows is unstable.
Win users: Windows is better, Linux is hard.
BSD users: You're both wrong.
Mac users: Hey, look at us. We are pretty.
Top 3: Mac, shut up.
BeOS users: We're better but y'all will never know it.
Bill Gates: All your $$ is belong to me.
---
Everyone here seems to be stating over and over the blatantly obvious fact that they didn't use code that was optimized for each target platform.
Well, I think the test was fucked for a completely different reason: They used a live internet connection for the test. Don't they know that the latency between those connections will change from execution to execution? For example, sometimes I hit slashdot, and on my nice slick t1 here at work, it loads instantly. Other times, during heavy loads, it takes up to thirty or fourty-five seconds. My point is that they are connecting to mail-servers which they have no knowledge of the current load of each server. This probably skewed their results wildly. In order for this test to be fair, they should have set up some boxen on their own intranet, and tested the connections with no other traffic around to mess up the results.
Well, your fingers weave quick minarets; Speak in secret alphabets;
std::disclaimer<std::legalese> sig=new std::disclaimer; sig->dump(); delete sig;
It means nothing if "A" is fastest, if it runs on a bad OS, cheap commodity hardware or isn't supported. You go with "B" becuase it DOES.
Fast != correct all the time.
The only part that I will have to agree with is that EXT2 fs is very fast. Although my one mail server that runs linux has an 18G mail partition that has just reached 50% fragmentation. The other 3 servers run FreeBSD with 1% fragmentation. That server will be switched to FreeBSD on Tuesday. After seeing that I am having trouble convincing anyone that Linux is a good OS.. especially myself.
Sudo Chop!
In today's cheap iron world, I don't think narrow margins in performance matter that much.
Everything is about reliability. I would rather have an OS that works is up >99% of the time, and shell out a few more bucks for hardware, than an OS that runs itself into a blue screen like a freakin' rocket.
Whatcha doooo with those rollin' papers?
Make doooooobieees?
Obviously this "test" is a crock, as FreeBSD would not be that far behind, what with it's surperior network stack and thread handling
Thanks AC for the nice plug. I'm a frontline developer for FreeBSD and I've been tasked with doing a complete overhaul for the tcp/ip stack, as well as doing a POSIX implementaion of threads, something it sorely needed.
It's quite possible this test was done without the BSD_booey_stack extensions compiled in, that would account for the less than stellar results. It's almost impossible for the tcp/ip stack to be the bottleneck for any problems anymore since I introduced double pass ID caching into the CVS. Basically it uses user space memory to reduce the overhead on the kernel, while performing table lookup translations on a stack hashtable. It's getting still better times if you enable my optimized dynamic MTU settings, even though that's still in the experimental stages.
I don't let this stuff get me down though, when we release the next point version at comdex next year the industry will watch with awe, it's going to be a smoking little OS by then.
Wagner LLC Consulting Co. - Getting it right the first time
Solaris is much more finely grained in its locking than any of the other OSes mentioned. Because of that, comparisons with other OSes running on one or two CPUs (usually on PCs) do not do Solaris its due justice. Sure, Linux or FreeBSD, which aren't very finely grained in their locking (but are working towards changing that) spend less overhead in locking calls, so they run faster.
But how fast can they run on a 32-cpu machine? Or a 64-cpu machine? According to some public documents I saw, Sun will release a 72-cpu machine this summer. They currently support 64 cpus on their E10000 machines. Solaris is a highly scalable OS. Linux is not. FreeBSD is most certainly not. Windows2000 may like to style itself scalable, but come on, we all know they are dreaming. Maybe scalable to 4 CPUs (if you own Pentium Xeons), and maybe in someone's wet dream it could scale to 16 CPUs or so, but none, I repeat none, of these OSes can scale like Solaris.
Solaris' strength isn't the fact that it's blazing fast on a single CPU, because a lot of tests can show Linux is faster. But Solaris *is* blazing fast on massively parallel machines. Solaris shows time and again an amazing ability to scale performance with the addition of more CPUs. The overhead required to build that scalability into the OS penalizes Solaris on single or dual-cpu machines, and that *must* be taken into account by people.
And don't even talk about 64-bit. Sure, Solaris for Intel is limited to 32-bit address spaces due to the constraints of the CPU architecture on which it runs, but Solaris the OS is built through and through as a 64-bit OS, and Solaris running on UltraSparc hardware supports zillions of bytes of RAM. The new SunFire 6800s can support in the hundreds of gigabytes of RAM.
Can Windows2000 do that? Can Linux do that? Can FreeBSD do that? Really we are talking about different markets here, that's all. You really need to test the OSes in the areas they are designed to operate, and then you'll see who the real champ is.
Why isn't Novell Netware included here. It seems kind of strange that THE best Network OS is being left out here. -- Jim
-- Jim
Obviously this "test" is a crock, as FreeBSD would not be that far behind, what with it's surperior network stack and thread handling. Windows 2000 may be a fast system, but it certainly isn't as fast as FreeBSD, despite the fact that MS largely ripped their TCP/IP stack off the FreeBSD project. The jury, however, is still out on where Linux should have placed. It may not be the fastest system in the world, but it certainly rates at least above Windows 2K.
"I don't know that atheists should be considered citizens, nor should they be considered patriots." - George Bush
I have to say that I am really tired of seeing tons of people jump at the chance to trash an OS comparison. I have been reading posts like this for years, and this is the FIRST time that I have ever posted any comments.
Did anyone actually read the article. Under the heading "Real-World Test" the author said in very clear terms "The operating systems were the latest version available from a commercial distribution and were not recompiled (i.e., everything was tested right out of the box)." The author only made a 1 change to the amount of file descriptors that were available. There is always someone that has to say that if they tweaked a little more then "" would blow the doors of all others. This logic brings us back to MindCraft study where Microsoft installed an eperimental patch that allowed the admin to bind CPUs to NICs. You have to remember that most commercial distros cater to about 75% - 90% of their users. Not to the 5% that worry about things like 2 millions emails per hour or 10 billion hit per day. Also the more that you tweak the less stable the system may become.
There will NEVER be an end-all-be-all benchmark between all OSes. They are too different.
Something that everyone ALWAYS seems to forget is that certain applications are better suited for different OSes.
Also consider the distribution of this article, SysAdmin Magazine. They have limited space for their articles in their magazine. If the author included all of the data, code, graphs, it would probably fill the entire magazine. This was not meant to be a white paper or a doctoral thesis. If you want the entire thing including all 12 graphs that comprise the data of Figure 3, then email the author. I'm sure he would be happy to send it to you.
There is a moral to my rant. Benchmarks are only good for the person/group doing the benckmarks. Anyone reading those benchmarks outside the authors environment should only use that information as a guide, not an absolute. What works in one environment might not work in another. If you have an application that is supported on multiple platforms, then test it yourself in YOUR environment. MOST people do not do that, and usually end up spending way more time and money than they would have if they had tested it in the first place.
-GH