Slashdot Mirror
Benchmarking the Scalability of BSD and Linux
Fefe writes "I recently did some benchmarks for a talk about scalable network programming I held at Linux Kongress 2003. The benchmark results turned out to be surprising enough to present them on their own. This ought to end those pesky flame wars about whose IP stack or memory management scales better. Or maybe not."
Can anyone explain the discontinuities in the FreeBSD plots? Intuitively I would guess that something is breaking at high load, rather than getting miraculously faster. The author suggests that a clever optimization is kicking in, but I wonder if his tests were actually ensuring that the calls succeed.
Also watch out as you read the graphs - just to keep you on your toes, he changes the colors in every one!
Who's going to publish benchmark about this webserver's scalability?
Apparently the webserver with the results didn't scale so well. It's /.ed already.
That there is a story posted on scalability, but the gents web server does not respond?!? Just a thought...
His experiments noted a few crashes. HELLOOOOO. That's a clue. Check out this line from the study: Reluctantly? Why reluctant? He's already installed the non-stable branch. Why not go whole hog and run current? It's like he was trying to make freebsd fail by installing an experimental non-stable branch.
Sheeeez. Just for those who don't know, it's important in freebsd to not just look at the version numbers, but also look at what the branch is -RELEASE, -CURRENT or -STABLE. Right now, only the 4.x branch is -STABLE. Just because the version number is higher does not mean you should upgrade. Stick with -STABLE, unless you know what you're doing
Now, I'm not saying linux won't do a great job compared to freebsd. It's just that this study did everything it could to slant against freebsd. What kind of reaction would the community have it this had been released by Microsoft? I suspect in this case, it was just a beginner's mistake.
Benchmarking is hard work. He should redo his study with proper installations from a -STABLE tree, and report back in.
Note to self: Never run benchmarking tests on live webserver
slashdot, news for crazed liberal socialist zealots
...his web server is not very scaleable in response to Slashdotting!
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
The winner in this case is Open Source software.
The article is very fair and very well thought out. It is almost like reading a research paper. It looks like he is inviting criticism, insight, and corrections, rather than trying to force the experiments into a pre-determined outcome.
Such a thing is not possible in the proprietary world. Any study done on proprietary software has to be tainted with opinion and the experiments must be skewed. Read the EULAs. Some EULAs won't even allow you to publish the results of such tests.
Open Source software, of the BSD kind and the GPL kind, has totally changed the way we think about and work with software. One day, we will be able to scientifically determine what software we need to suit our needs. We will know ahead of time exactly what limits and what capabilities each piece of software has. IT managers will be able to sort through real facts based on real research, rather than a bunch of shallow articles and biased reports. Software will survive on its merits alone.
The whole industry is going to benefit by this, in a large, large way. The question one day will no longer be "Microsoft or Linux?" but "Which Open Source software should we use, and why?"
The radical sect of Islam would either see you dead or "reverted" to Islam.
I have a mirror (minus four graphs) :( workin on it...
members.shaw.ca/infonick
You are confusing me with someone who cares.
Dass ist doch wahnsinn, du Sau! Die leute hier koennen das gar nicht lesen! Kannst du das uebersetzen?
Its all dutch (well, german) to me
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
it appears, fefe's website does not scale well to a slashdotting. :)
Loose: Linux can't obviously not contain any SCO IP, because it's not business ready. SCO's lawsuit will collapse.
The only problem is that the BSD benchmarking might count as disturbing the rest of the dead and will have severe legal consequences.
Owner of a Mensa membership card.
Topic of this paper is
"Scalable Network Programming
Or: The Quest For A Good Web Server (That Survives Slashdot)"
What a coincidence!
By the way, fnord web server has at least once survived one slashdotting-event. 4 seconds of googleing result in this comment which should have let to a stream of visitors.
I hope fefe will publish the numbers of visitors and the behavior of its web server as soon as possible.
Horray for Haxalot! Oh, and the PDF is quite interesting, too.
Who wants to start taking bets on when Theo takes the bait.
Slashdot's scalability is quite amazing. It seems as a sites ability to resist a slashdotting goes up, slashdot's ability to slashdot a website also goes up. Usually it's at a higher rate but sometimes the sever makes it out alive. Although most of the time it ends up as a smoking pile of slag.
I have a complete copy (graphs and all) here.
Although the results compared with BSD are fairly interesting, the results difference between Linux 2.4 & 2.6 are truely astounding. For those who can't see the graphs or can't be bothered to RTFA...
bind() to port 0
n = number of processes:
2.4: O(n) rise number of processes
2.6: around 0 even with 10000 ports
nmap every other page in a 200MB file:
n = latency in microseconds:
2.4: O(n) rise
2.6: constant (basically no latency)
The kernel guys have certainly done some fantastic work in this last year. I can't wait for 2.6.0!
I wasn't able to see the benchmark results, but I was able to see that the server its hosted on is not very scalable.
I hope RMS reads the slides. They're in German at the link I used, so here's a translation of slide 13 which is page 14 of the PDF file:
"The memory required for an empty process is shockingly large on current Linux systems. However, this is not the fault of Linux, it's the fault of GNU libc.
GNU libc leads all libc implementations by a large margin in bloat and waste of memory. One day it got so painful that I wrote my own libc. With this, a static binary of 'Hello world' took only 300 bytes..."
I've long suspected that FSF stands for Fat Software Foundation.
(He doesn't say, but I assume his home-brew libc was a subset, otherwise we'd all want it).
I took three computers out in my rowboat, a Windows system, a Linux system, and a BSD system, and threw them overboard to see what would happen.
The Windows system sank like a rock, the Linux system bobbed back to the surface, and the BSD system rose to the sky, to be greated by a chorus of angels.
Then I woke up, so I don't know what the angels were singing.
Sheesh, evil *and* a jerk. -- Jade
So this guy grabs a beta version of a new tree in freebsd, and runs it against stable netbsd, openbsd and linux? Eh? Did he even compile his own kernel and take out all the debugging information in the released kernel? Did he turn off the debug info in the kernel config, on by default? I kinda doubt it somehow... If he knew enough to do that, he'd know that 5.1 has NOT been tagged stable.
Most freebsd users will look at this study and laugh. But people who don't know anything about freebsd (perhaps that includes the author?) might get the wrong idea. The study needs to be redone using the -stable tree, not the debug-riddled 5.1-RELEASE.
The SIGIO comparison is somewhat fair, but keep in mind that implementing a program with SIGIO requires multiplexing I/O asynchronously, which radically alters program design. Linux 2.4 lacks a synchronous, stateful interface for I/O mulitplexing. Programs that implement synchronous I/O multiplexing can easily transition to a library for their main loop to immediately take advantage of high performance multiplexing/events mechanisms. One such library is libevent.
Sabotage? Someone able to bring me up to speed on this?
As for the installation routine, it is something out of the dark ages, considering even redhat 4.0 - my first Linux installation - had a curses based install. However, if you ignore that, it still seems a decent enough platform - I use it on an Ultra5 at work and as my firewall/NAT gateway at home.
What I'd now like to see, which is what often doesn't occur, is for each OS camp to provide tuning suggestions to the researcher, and have him re-perform the tests. This is because no production webserver is likely to be untuned from the default installation.
Oh and the read-only sysctl problem for FreeBSD that he mentions was probably due to securelevel's being on (meaning you can't modify kernel variables).
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
Using an unstable development version and then complaing about instability, peppering the results with emotive commentary and clueless rhetoric. (btw the 1024-cylinder boot restriction he complains so much about has been fixed for a while) Especially funny was this idiotic statement:
Someone should hit him with a cluestick on this issue. Yeah, like itojun is despicable and unworthy...OTOH, the results are of concern and should be verified by someone less obviously biased. I haven't noticed them in practice on moderately loaded servers though (but I'm biased in the opposite direction).
Not sabotage, security. In case you don't know: itojun is the guy between all the BSD's IPv6 support, and has been very active in the standarisation process.
I'm wondering, if he was going to be doing a scalability test, why didn't he test the version of FreeBSD that is actually reccomended for production (4.8)?
He had the time to test the stable and devel versions of the linux kernel, but only the new technology version of freebsd?
Evidently his site doesn't handle traffic very well, especially on the Slashdot Scale.
Your hair look like poop, Bob! - Wanker.
It's convinced me 2.6 is a *lot* better than 2.4. I'm not really surprised at the results. FreeBSD has never seemed faster than Linux, while Open has seemed noticeably slower.
Does anyone know why Open underperformed so dramatically? Can the OpenBSD development team comment on this benchmark ?
Well, it didn't. ;-)
Cheers, Ulli
Simple things should be simple, complex things should be possible.
Alternatively: buy a Mac! Pretend your beloved operating system is now one of the most widespread consumer *nixes! Because we all know that Aqua has nothing to do with it, Mac is BSD!
Slashdot commentors: Please realize that many people who visit Slashdot are from countries where English is not the native language. It is VERY likely that they won't understand a subtle joke. Don't become hostile because someone doesn't understand. Just explain.
From /usr/src/UPDATING:
NOTE TO PEOPLE WHO THINK THAT 5.0-CURRENT IS SLOW:
FreeBSD 5.0-CURRENT has many debugging features turned on, in
both the kernel and userland. These features attempt to detect
incorrect use of system primitives, and encourage loud failure
through extra sanity checking and fail stop semantics. They
also substantially impact system performance. If you want to
do performance measurement, benchmarking, and optimization,
you'll want to turn them off. This includes various WITNESS-
related kernel options, INVARIANTS, malloc debugging flags
in userland, and various verbose features in the kernel. Many
developers choose to disable these features on build machines
to maximize performance.
Poul-Henning Kamp -- FreeBSD since before it was called that...
I knew the Linux 2.6 kernel was supposed to be faster and more scalable, but, damn, that's awesome.
There's no need for such a very technical benchmark.
Regular usage of various operating systems on the same host makes it obvious.
When it comes to speed and features (or bloat), Linux is more efficient than FreeBSD, NetBSD and OpenBSD. This is especially significant in SMP environments.
Linux users are always talking about the just-released experimental patches that will help their system to get 0.1% faster, or the most aggressive flags to optimize their Gentoo system.
BSD users just advocate their system with the generic word "robust".
Nowadays, stability is not really the key. Every Linux or BSD free operating system has basically the same stability. The software is the same, with the same bugs. The package system have equivalents (Debian works on NetBSD, Gentoo works a lot like BSD ports, etc) and support for common hardware is almost identical.
The reason to choose one OS over another is often more political than technical. People tend to use FreeBSD just to try "something else". People tend to use Linux because the Mandrake/RedHat/Conectiva/SuSE installers are beautiful or because Gentoo is fashion and a good way to learn what Unices are made of.
But if this is just to use common software like Apache and Qmail there's no real difference except speed. If this is what you need, Linux is definitely the best choice nowadays, especially since 2.6 kernels are almost ready for production use.
For other needs, your mileage may vary.
For instance I love OpenBSD for development. The compiler and the libc have very handy features to automatically detect bogus code. And the man pages are also excellent, with helpful hints.
For firewalls and trafic shaping, I wouldn't use anything but *BSD because of PF. PF is really the best thing in *BSD systems IMHO. The firewall is very easy to configure yet extremely powerful and fast. And I was fond of Iptables before.
For bridging and transparent firewalls, I would also use BSD because it seems to work better than Linux in this area.
In fact it's just like the girl of your dreams. Everyone's always looking for the perfect operating system that will perfectly fit all needs, but it just doesn't exist.
{{.sig}}
I think Theo should get off the rags. Hey THEO! Y0u r teh scr3w.
Your scientific method isn't one, and these results are invalid as a result.
You took in-progress development release of OpenBSD compared to stable releases of other operating systems (even your NetBSD was a RELEASE rather than a CURRENT). CURRENT is always going to be an in progress development and not entirely stable.
Computer benchmarking data are (usually) quite noisy. As such plain graphs, like those shown are not very useful. Simple statistical graphics like lowesss plots are much easier to interpret. Take a look at the for some ideas.
This stuff is easy to use, at least if you have a computer background, and would produce far better graphs.
-- Anthony Staines
Well, I'm not entirely sure whether these results demonstrate that Theo is an even bigger dickhead than we all already thought he was, or if this guy is a perfect candidate for a position on Apple's G4 benchmarking and marketing team. Maybe both? Do you have to be able to read docs to market G4s?
I find your ideas intriguing and I wish to subscribe to your newsletter.
Hostility is bad manners, no matter who does it.
Also watch out as you read the graphs - just to keep you on your toes, he changes the colors in every one!
The author should use a graphing technique Edward Tufte calls "small multiples". In that you lay out a series of thumbmails of the each graph series. The eye can quickly scan down the thumbmails and get a rough feeling for how each series compares.
This would avoid the problem where graphs overlay each other and with the inconsistency between graphs in the plotting conventions.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
I read many comments claiming that linux 2.6 is _much_ better than FBSD 5.1 - re-read the article please. I know its hard for a linux zealot not to exaggerate the results, just as it is difficult for a FreeBSD zealot not to get over-defensive.
;-)
As to the quirks. As someone pointed out, the author should have read the handbook.
Also, the FreeBSD kernel (well, the OS, as the kernel isn't developed separately from the rest of the system) development is more in its earlier stages than linux kernel development. There is no code freeze yet (I think) and I very much doubt that the 5.x branch will be stabilized until we see a 5.3 release. Thats not going to happen until spring (5.2 is due in December). In other words, while 2.6 underwent some stages of optimization, this is yet to happen to the 5.x branch. Yes folks, the results you see from FBSD is still that of an unoptimized kernel - and that's not as bad as zealotry (from either side) would have it,no?
It's this type of information that does OSS a disservice - making it look amateur and unprofessional.
You used an pseudo-scientific method, i.e. your graphs are nice, your data points are nice, but you forgot the fundamentals of testing: you didn't clarify the exact projects you were testing (you should have asked the project leads for advice on which versions of the projects to test against), nor did you asked for feedback and determine the reasons for the anomalous conditions (e.g. the FreeBSD maxprocs could have been resolved by a quick question to a FreeBSD expert). This is a lowbrow technical approach.
Not only that, you presented the pseudo-science results to a large Linux congress, so that while Linux does perform well (undisputed) and everyone can pat themselves on the back, you make the congress look amateur and unprofessional.
The result is that many of us that do know a thing or two about methodology, experimentation, engineering, etc, will simply interpret your results as providing a "rough general guideline", but "leaving too many questions unanswered" and continue to think that "a lot of the linux community is still amateur and unprofessional".
I cannot argue on scalability. Benchmarking should be more, though.
I had been running a SOHO-server on P75, 128MB, Dial-up/DSL, DHCP, Print-server, Apache, SSH, hylafax, Firewall on Linux for some years. Then I tried OpenBSD for a change. It runs with about 1/3 of the memory (Linux around 100 MB, OpenBSD about 35 MB).
Since I installed it, there were just 8 patches to be added (that is about 1 per month); installed pretty easily. I felt quite safe and the system has been pretty stable.
Though I don't see too much of a future in it, I appreciate its safety considerations and - in my case - stability.
My desktops run Linux, but currently I see no contender as replacement for the OpenBSD-box. I wouldn't mind, though, to go back to Linux if there was anything equivalent.
Let's simply assume that OpenBSD will remain in the niche for small (it will probably never be SMP) servers of security-concious admins with low hardware requirements and low throughput.
I am not pretending to produce the be-all end-all benchmark here. That would at least include having an SMP server for the operating systems that support it, and it would have used Gigabit Ethernet. Frankly, I don't have the hardware to do that.
;-)
But I do have these benchmark programs, and I was hoping to spark enough interest in doing some real comparative benchmarks by posting the results here. The tools are now there, and they do use the platform specific API hacks to get the best performance on each operating system, so the biggest area of concern for comparative benchmarks should be out of the way -- that the benchmark program is skewed in one direction or the other.
If you are in possession of a real benchmarking lab with enough free disks and spare time on your hand to complete these benchmarks, please do so!
I'm not from the Linux mafia. If my goal was to discredit BSD, I would not have added support for kqueue in the first place. If my goal was to pat myself on the back, I would not have used the BSD sysctl to increase the number of files in the system (on Linux 2.6, you basically don't have to do that, just use ulimit), and that would have been fair, if only in the Netcraft/Microsoft sense.
But neither am I from the Intel scalability labs with 1000 identical PCs and multiple gigabit ethernet connections to the server. I'm just a guy with a notebook who happened to have a little spare space on his hard disk and decided to include the BSDs in the benchmarks.
In The article, The author mentions that he finds "unacceptable", or embarassing, a few things about openbsd, mainly concerning it's general speed and scalability.
ALthought not beeing an expert in bsd nor in linux, maybe i'm rong, but isn't OpenBSD made to be secure, and not the fastest operatin system ( additionally, comparing oragnges to apples, by testing release/current/stable, wasn't the best way of comparing those OSes) ????
By implementing a few feature in the OpenBSD stack and Kernell, I guess that the devellopers are delibetatelly sacrifying performance in order to get an Os less vulnerable to DOS and other vulnerabilities
It owuld be interesting to compare the same OSes in a security test to find out which OS is more secure, if speed and scalability isn't the only issue in a OS...
Oh, wait, but we are comparing those dissimilar products when they're all attempting to do the same thing. Why else would car makers boast about engine horsepower, acceleration, traction and braking, for example?
I guess you didn't actually read the article, where the guy states that he's doing the same damn thing on each OS, using the same hardware, so the only variable is the OS/distro being used. His choice of OS/distro may be suspect, but the results still stand. He's comparing oranges to oranges and telling you which is sweeter, which is juicier, etc.
Sure, the comparison is valid, you can compare those elements, but are the results relevant? Not many would use OpenBSD in a performance (as in speed) critical role, and not many would expect Linux to have the security track record of OpenBSD.
If I'm going to war, I'll take the tank. If I'm going to race, I'll use the sports car. And if I'm taking 40 friends to the mountains for some skiing, count me in on that bus.
Does everything include nothing?
Executive Pope (small) Kallisti Engineering
Mac OSX server is based on Free Bsd5
other web services running out of the box
Apache 1.3 and 2.0
Tomcat 4.1 and Axis 1.1
JBoss 3.2
MySQL 4
JBoss Deployment Tools
WebObjects on JBoss
Perl 5.8 and PHP 4.3
Ldap, samba
Some drink at the fountain of knowledge. Others just gargle.
Linux end users on average tend to be the scum of the earth with median level IQ's. Ever witness the various linux ghettos on irc or usenet ? You cannot miss them. So it doesn't surprise me that this person was unable to conduct an "Empirical scientific double blind benchmark". Hence the oddball subjective extraneous derivatives of his so called "research" on these benchmarks are pointless. It serves only to satiate the need for lust in the lackluster minds of the mass linux user base --- that is the sole purpose of this propaganda.
.
.. total lack of consistency from a linux E-tard. He says at other times he was "surprised" that NetBSD outperformed and even beat the others clearly on benchmark to do with sockets. That would have come as no surprise to a serious engineer who had the integrity and quality of character to study the code changes to each OS by reading the mailing lists, developer change notes and even reading source code. The NetBSD socket performance win was clearly because of Jason Thorpes hack here :
/03/0011.html
In order for this study to be worth a damn he would have to:
1.) Use two computers with the same hardware from two different manufacturers
2.) Have the common decency to do thorough research which I will prove he does not do before benchmarking.
3.) Email head developers from each operating systems on tips on tweaking each OS and what versions of each OS to use in each benchmark. If he still decided to use bleeding edge from both linux and the *BSD's-- he must do so consistently across the board. Obviously he didn't because he didn't use NetBSD 1.6ZD (current) from fresh cvs diffs. Strangely enough he went with -current for both OpenBSD and FreeBSD though. The only phrase that comes to mind is "highly subjective propaganda produced by a plebeian simian who was not even clever enough to mask his propaganda with at least the nomenclature of the scientific method. How hard would it have been for him to email Luke Mewburne, Miod, and someone from FreeBSD ?
He is obviously experienced with linux and semi literate with FreeBSD (although not really because he was too stupid to set certain tweaks). While totally clueless in the NetBSD and OpenBSD environments.
"In the mean time I heard that the -CURRENT version of NetBSD has over two years worth of improvements in it, so it probably is even better than NetBSD 1.6.1, which already surprised me with its good performance (although it is clearly outperformed by FreeBSD)." -- fefe.
Then you have no excuse for using the bleeding edge of Linux and FreeBSD
http://mail-index.netbsd.org/current-users/2002/ 07
"The clear loser of this benchmark is NetBSD, because they simply don't offer a better API than poll. As I wrote in the introduction, I only benchmarked the stable NetBSD 1.6.1 kernel here,
and I assume they have included kqueue in their -CURRENT kernel. I will try to update my NetBSD installation and rerun the benchmarks on it." -fefe
Gee, you think fefe ? You assume they have included a native kqueue ? Ah you mean you are a complete moron who couldn't be bothered to conduct real "research" and "benchmarking". Clearly it wouldn't have taken long to research and be able to say with authority "NetBSD has native kqueue it was added to the code on such and such a date by developer X". Just for the record NetBSD current does have native kqueue and native pthreads.
I bet you didn't even bother to tweak NetBSD at all as you barely tried with FreeBSD. Note that it would have been a good idea to tweak
Kern.maxfiles, kernmaxvnodes and vm.anonmax with sysctl on NetBSD.
Also, why mention itojun on ipv6 but leave out other developers names where NetBSD and FreeBSD win on the benchmark ? He only goes on extravagant explainations on why OpenBSD loses in one scenario. His need to be verbose only in such a situation shows his linux bias or guilt for skewing the benchmark so bad. Oth
Felix is more than opinionated about Linux vs. BSD. Check the qmail mailing list archives for his rants with BSD users about filesystem and other performance issues. In particular, he's had a number of run-ins with those on the list who use OpenBSD--the one he labels a `stinker'.
He doesn't give details about the installation of each OS. Did he benchmark them one by one after wiping out the previous OS? Or did he just partition the disk and put each OS in its separate partition? My impression is that he did the latter though I may be mistaken.
Any hardware junkie knows that data reads and writes can be upto 2x as fast at the start cylinders of a disk than at the end. If he put FreeBSD in the beginning and OpenBSD at the end, it would really skew the results.
Why would anyone ask a rhetorical question?
Conclusion: NetBSD outperforms all other operating systems in this benchmark. However, all contestants scale equally well, there are no clear losers. And the overall latency in this benchmark is so low that the results are interchangeable in practice.
The first graph, which this quote is under, does not support this. It clearly shows Linux 2.6 using HALF the CPU cycles of NetBSD above 4000 connections. How did he get this conclusion? 2.6 is the clear winner in his tests.
JON
The interesting point is that all of these operating systems seem to be getting faster. It seems to come down to how many recent developments have been integrated into the version being tested, not any inherent differences between operating systems. This is, of course, as it should be: the source for all of these operating systems is available, and there are even frequently papers describing the techniques. If a technique is, in fact, better, it should eventually be adopted by all of them, and so your results will depend on how much has been adopted in the version you're testing.
It is encouraging to see that all of these developers are competing with the real opponent, which is not each other or even Microsoft, but the slashdot effect. After all, the goal should not be simply to be better than the others, but to be sufficient for the user's purpose, which is not hampered but rather assisted by sharing all of your tricks. It can sometimes seem like there are endless wars between Linux and BSD, but, behind the scenes, the sides actually share information. Never as much as they'd like, but always more than people think.
In reference to a particular metric (not the entire suite of tests) he says "Even Windows would probably outperform OpenBSD"
Talk about taking the kids gloves off!
-Adam
Comment removed based on user account deletion
I know this benchmark is about open source and all, but I would love to see Solaris thrown into the mix. With all its vaunted scalability and stability, I'd love to see what it actually does better. I guess it would have to be the Intel version, but I would think their kernel algorithms should be the same across architectures.
He speaks truth!!!!
clifgriffin > blog
You read too much marketing literature and misinterpret it. To shatter some illusions, Cheer detergent will not make your relationships better. Drinking lots of beer will not attract 5'10", 100lb models with large breasts and bikinis.
Further:
NeXTStep was a Mach kernel with BSD4.3 userland tools running a proprietary windowing system that uses Display PostScript (making their printer cheap) coded with Objective C.
Mac OS 10.0-10.2 is a Mach Kernel with a userland MOSTLY from FreeBSD 3.x running a proprietary windowing system that uses Display PDF (making printing to PDF trivial) coded with Objective C
Note a similiarity?
Panther finally uses FreeBSD 4.x userland tools.
Plus much tuning to their proprietary windowing system.
Still a mach kernel. Note that the word mach contains neither a "B" an "S" nor a "D".
Panther (NeXTStep 7.0.0 according to uname) would still crawl on my NeXT Cube that runs NeXTStep 3.3 just fine at 33MHz.
RE: Your list of stuff taht comes with it. You get 3 for 10 right. perl is
% perl -v
This is perl, v5.8.1-RC3 built for darwin-thread-multi-2level
Apache 1.3.28, no apache 2.x
No sql server (postgres would be a better fit for their completely free software needs).
I don't find jboss nor webobjects nor tomcat on my Panther box.
LDAP *clients* (everyone has ldap clients these days)
Samba, yes.
PHP, yes.
Please try again with some correct facts.
I for one had a terribly hard time reading the graphs. Three problems in particular:
1)
I (like about 20% of the male population) am partially color deficient.
The graphs' colors where very hard for me to distinguish in the ledgend; what with the 1 pixel wide font and sample line/symbols. The colors chosen were all of about the same intensity/saturation also, so if you are color blind the entire graph would look all about the same level of gray. This problem is further compounded by all the graphs having different color assigments (Linux 2.6
2)
crowding. On a few of the graphs, there were differences in order of magnitude between groupings. Since this dataset was supposed to show the fastest, that would naturally be the most interesting data. You don't take photo finishes of the 7th and 8th finishers in an 8 horse race. Using a logarithmic scale for those charts would make the data easier to read, showing detail in the lower(faster) end and allowing the less significant higher(slower) data to "blend" in the curve.
3)
Point size. The symbols are too large for that many data points. This may be personal preference, but single points should be used for scatter graphs like these, not symbols. Symbols may mask other more subtle changes in the results because they overflow other points nearby. Similarly, make your legend samples substantial, a block of color is much more apreciated than a single pixel line, and bold your identifier text.
If you make graphs for the web, please take them in to Photoshop, GIMP, or something similar and reduce or remove the color data (convert to grey scale). The chart should still be at least fairly reable in greyscale.
Choose a wide range of Hue, Saturation and Lightness
If you are going to use light/desturated colors, use a black background. If you will use dark/heavily saturated colors, use a light background.
If you can't/won't do that, then please try to make the raw data available in CSV or some other "universal" format so we can create our own graphs.
Article X: The powers not delegated... by the Constitution...are reserved...to the people
Ha, well, i don't know too much about the internals of kernels and OSs and stuff, but hey, why doesn't somebody take the relevant code of the speediest winner of each of these "tests" and smack it all together into one "Performance-enhanced OS"... ha, plz no flames! i understand this would not be a light undertaking, but it is an idea i suppose.
This is great if all your reads and writes are like this.
I deal with huge database servers that get 10 or more times throughput using dd(1) than they do in acual use with that pesky real data.
Also wondering why I'd care about the CPU on a copy.
Give me a laptop disk on a 3GHz i386 and I'll beat it with a sparc 10 (125MHz) attached to a nice Baydel RAID box (sustained real world writes of > 40-50MB/s with RDBMS systems).
Ah, so you choose to assume *BSD hackers and users are unemployed. Could it just be that not all featherless bipeds are mentally equal ? I think the latter is more likely. You choose to skirt around most of my points and just take up the specific sysctl tweaks I mentioned. Sorry , but life doesn't work that way. You don't get to choose what points you want to respond to and win the whole argument. What is this candy ass land ?
OSX 10.2 Server. I'd imagine it comes with some RDBMS, maybe jboss/tomcat, etc. OpenDirectory is OpenLDAP with a custom backend and slightly different schema I belive.
- RustyTaco
I just wanted to point out that it most likely wasn't the OS that was his problem.
And yes neatly stacked data, and I wouldn't get even close to such write speed, I barely reach it with reads.
On the other hand, this system would cost 500 USD if built today
But 20+ minutes for 17 Megs? I'd say something is badly wrong.
And I don't think it's the OS.
My 486 DX2/66 copies 17 Megs in less than 30 seconds (and it's running OpenBSD).
Executive Pope (small) Kallisti Engineering
He specifically didn't test 4.8 because he was told 5.1 would scale better. He knows 4.8 is more stable, that wasn't really the point of the article. 5.1 did an excellent job.
Fuck Beta. Fuck Dice
That made me laugh our loud. You hit it on the head. Thanks...
Who is this Anonymous Coward character, how does he post so much, and why is he always such a whore?
I read the tests and many of the comments on them.
/if/ the tester happened to be more familiar or friendly with one of the OS's tested. That would really help improve the value of the test. Unprejudiced==scientifical==professional==a virtue.
I was very curious about the test, but very disappointed with everything after I read it.
The person who performed the tests was very naive when he said this ought to be the end of flamewars about network performance/scalability among the Linux/*BSD users. His test was just not good enough to be meaningful to anyone in the respect of offering acceptable conclusions.
A proper test would be
- Performed on proper hardware. I mean, doing a network performance test on an old laptop with inadequate harddisk size, with four very much server-oriented operating systems, one of which (OpenBSD) couldn't even be installed on the same part of the harddisk as the others! That's like organising a race between a Lamborghini, a Lotus, a Porsche and a Ferrari in downtown New York during rush hour, and disqualifying the Ferrari on the grounds that it is difficult to park backwards.
- Unbiased. This one was very much biased against OpenBSD, and in favour of Linux/'Leanux', as follows from many of the comments made by the tester.
- Performed by somebody who knew enough about installing and running all of the OS's involved to run tests on them (the tester seems knowledgeable about Linux, but is totally clueless as far as for instance OpenBSD is concerned.)
- Performed with a test programme that wasn't developed with a bias toward one of the OS's in the test (the test programme involved was developed on Linux, later 'ported to' *BSD.)
- Described plain fact, by someone who would be objective and who would avoid showing emotions about the subject, much less a general favour or disgust toward the OS's tested. This is especially important because of the sensitive nature of the test subject. There are so many flamewars already!
- Described withoud prejudice even
- Carefully giving minute details about the test conditions (hardware, software, test programme details, OS installation details..) This test wasn't remotely accurate, look how it even fails to mention at what time OpenBSD-CURRENT was downloaded. That's crucial information.
- Compare equally. The test is already invalid because it compares an ancient -STABLE NetBSD with a -CURRENT FreeBSD.
- Be clear about its subject. The test focuses exclusively on network performance/scalability, and then goes on to praise or totally disqualify the OS's tested solely on this ground. The test doesn't look at crucial aspects like security, maintainability, documentation, correctness of design, etcetera, but will still not hesitate to draw very un-subtle conclusions.
- Ask those who are in the know for comments on the test conclusions before making the results public.
In this precise case, it would have prevented several stupid factual mistakes from being published (the OpenBSD installation problem and IPv6 idiosyncrasies for instance.)
- Call into the test all relevant players, or at least represent different groups properly.
Much as I dislike it - I'm an open source adept as well - Windows is used as a server system on the public internet by many organisations. It should be compared with the other systems in this test, and be given a fair chance. I wouldn't have been surprised if it performed very well, seeing that the TCP/IP stack of modern Windows versions has been largely copied from FreeBSD..
Also I would have liked it if there were at least one proprietary Unix system (such as Solaris) in the test. Just for the sake of the comparison.
- Look closer into the reasons, backgrounds, pros and cons of faults that the tests find. Again, if the tester had done this, he would have found that some of the badness he found wasn't a design mistake, but a design decision based on healthily made trade-offs (security trade-offs in the case of, not surprisingly, OpenBSD.)
where his data is incorrect or his conclusions are wrong.
remember to include your own data supporting your assertions.
Some Google searching turned up a thread discussing this, and as suggested there I replaced -lpthread with -pthread in the Makefile. It built fine after that.
Enjoy your job, make lots of money, work within the law. Choose any two.
Hmm, perhaps I should have been more obvious about that :) I figured you have to try hard to mess up the spelling of Mensa, especially in a post about spelling. On the Internet, spelling 'lose' incorrectly is somewhat of a phenomenon, a mistake that's easy to make because the phonetics come out 'oo'. I admit that I have thought about comitting that mistake once or twice :)
A month or so ago someone on Slashdot posted an inspiring flame about how people very often spell 'lose' incorrectly, so I thought I'd take my turn at carrying the torch, especially when I saw the Mensa member bit. If people in the top two percentile of intelligence engage in persistent spelling mistake, what hope is there for the rest of the world? Card carrying Mensa people should be ones to set a standard in things like spelling, learning from others' horendously common mistakes, and humility.
For most people, I make allowances, since not everyone has an eye for spelling. But my faith in humanity is pretty low and I have to find hope somewhere. Admitdetly, I don't even have a great deal of confidence even in 98th percentile IQ testees, but that is another matter.
The 2nd most common spelling mistake I can think of is one of ignorance, and not accidental: that of spelling 'a lot' as one word instead of two. But if you inadvertantly spell Mensa with to e's, you probably have peanut butter stuck in your keyboard or something. That makes me think... if Star Trek style teleportation existed, I wonder if there would be mischievous people who teleported bits of peanut butter into people's brains at the exact right spot so that they always spell 'lose' incorrectly. Or maybe there already are! Hmm I must go think now.....
Yep. Welcome to slashdot, where anything that shows linux as not perfect OS gets torn apart.
It is disappointing to see many of the comments here. Some are good and beneficial comments, others are not.
/. .
Where is the spirit of Sharing Ideas here?
#Define Flame On
I suggest that some here Grow Up! Zip your pants up, and quit the p*ss*ng contest. Your 15 seconds of fame are over! Now start acting like team players in this world!
#Define Flame Off
How about several of the people here that apparently seem that they know their stuff the best (and they may), smash heads together to come up with some benchmarks that test appropriately each of the different Operating Systems. I do mean all of the different OSs, such as OS/2, HPUX, Solaris 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, Windows (3.1, 95, 98, 2000, NT, etc....) Linux 2.1, 2.2, 2.4, etc... FreeBSD 4.8, 4.9, OpenBSD, NetBSD, etc.....
I would suggest the following:
1. Get some people together to come up with a general test that measures specific areas that the kernel is programmed for. These would be tests that would be subject to each OS developers approval (ie, that they test appropriately the area specified). Also, acquire each OS tuning parameters for each specific test.
2. Use these tests to benchmark where an OS has issues.
3. Forward these tests to developers who care about the OS they program, to use these as starting points where code may be improved.
4. Post the tests for each OS. Both before and after fixes.
Do I know what I am talking about? Probably not, but I was a test engineer for several years and we did perform quite a few test just like Fefe did to see what would happen. As time went on, yes we came up with better and better tests. Accept this fellows test as for what it is, a start of people getting together to put together better and better tests. By the way, he did a fairly good job for one that has gone out of their way to help the nerds out there think about things, some apparently think, while others just like to
Which OS do I use? Which ever is handy at the moment! Sometimes we don't get a choice!
HGTTG: "I knew that there was something fundementally wrong with the Universe."
Mach is a microkernel. There are a lot more pieces that compose whole kernel.
From the The Unofficial Darwin FAQ:
# What is Darwin's lineage? Justin Walker's description of Darwin's heritage is: Mac OS X began life as a child of OpenStep 4.x. The first stage in the evolution was the move from OpenStep 4.x to Rhapsody, which was based on BSD Lite2, with a batch of NeXT-instigated changes. When we shifted to Mac OS X from Rhapsody/Mac OS X Server, we incorporated FreeBSD 3.2 changes for the networking piece. The rest of the BSD portion of the kernel remained more or less as it was. At the same time, we (i.e., Fred, with your [Darwin's] help) pulled in command and library updates. Most of these are from FreeBSD, although I'm not positive about the heritage of the pieces that are now in the system.
Please try again with some correct facts.
Yes, please ;))
Redoing all those benchmarks without the strange libraries that the author uses and instead using plain syscalls shows that what the benchmarks measure is how the benchmark code performs, not the actual syscalls.
A real bind benchmarks has a O(1) time on OpenBSD and NetBSD.
The connect latency benchmark seems seriously confused. And since I don't find any source for it, I must conclude (based on the rest of the benchmarks) that it must be flawed in some way.
The "read one byte from each page" benchmark on OpenBSD performs that badly because the file used doesn't fit in the buffer cache.OpenBSD doesn't use all the memory for the file cache. When the file fits in the buffer cache OpenBSD has a performance comparable to linux 2.4. Anyone with half a clue about how a computer works would have listened to the disk and noticed that it's making noise.
And the rants about ipv6 only show that he doesn't know how to write proper ipv6 code and instead relies on a compatibility interfaces that were created for lazy programmers to allow for quick conversion of old ipv4 programs into ipv6.
The author is clueless and the benchmarks more often measure the performance of the benchmark itself than the operating system.
Loose still holds second place to viola and walla being used in place of "voila". Though I have given up all hope as I've recently seen "anyways" and "that's a whole nother thing" used in would-be serious posts.