Slashdot Mirror
Benchmarking the Scalability of BSD and Linux
Fefe writes "I recently did some benchmarks for a talk about scalable network programming I held at Linux Kongress 2003. The benchmark results turned out to be surprising enough to present them on their own. This ought to end those pesky flame wars about whose IP stack or memory management scales better. Or maybe not."
Can anyone explain the discontinuities in the FreeBSD plots? Intuitively I would guess that something is breaking at high load, rather than getting miraculously faster. The author suggests that a clever optimization is kicking in, but I wonder if his tests were actually ensuring that the calls succeed.
Also watch out as you read the graphs - just to keep you on your toes, he changes the colors in every one!
Who's going to publish benchmark about this webserver's scalability?
linux 2.6 is generally as good as, or worse than 2.4
great job guys
Apparently the webserver with the results didn't scale so well. It's /.ed already.
That there is a story posted on scalability, but the gents web server does not respond?!? Just a thought...
His experiments noted a few crashes. HELLOOOOO. That's a clue. Check out this line from the study: Reluctantly? Why reluctant? He's already installed the non-stable branch. Why not go whole hog and run current? It's like he was trying to make freebsd fail by installing an experimental non-stable branch.
Sheeeez. Just for those who don't know, it's important in freebsd to not just look at the version numbers, but also look at what the branch is -RELEASE, -CURRENT or -STABLE. Right now, only the 4.x branch is -STABLE. Just because the version number is higher does not mean you should upgrade. Stick with -STABLE, unless you know what you're doing
Now, I'm not saying linux won't do a great job compared to freebsd. It's just that this study did everything it could to slant against freebsd. What kind of reaction would the community have it this had been released by Microsoft? I suspect in this case, it was just a beginner's mistake.
Benchmarking is hard work. He should redo his study with proper installations from a -STABLE tree, and report back in.
So what if FreeBSD is fast? So's MS-DOS.
I have over 70 freaks, do you?
here
I have over 70 freaks, do you?
Note to self: Never run benchmarking tests on live webserver
slashdot, news for crazed liberal socialist zealots
...his web server is not very scaleable in response to Slashdotting!
Donte Alistair Anderson Roberts - hi son!
Karma: Chameleon
The winner in this case is Open Source software.
The article is very fair and very well thought out. It is almost like reading a research paper. It looks like he is inviting criticism, insight, and corrections, rather than trying to force the experiments into a pre-determined outcome.
Such a thing is not possible in the proprietary world. Any study done on proprietary software has to be tainted with opinion and the experiments must be skewed. Read the EULAs. Some EULAs won't even allow you to publish the results of such tests.
Open Source software, of the BSD kind and the GPL kind, has totally changed the way we think about and work with software. One day, we will be able to scientifically determine what software we need to suit our needs. We will know ahead of time exactly what limits and what capabilities each piece of software has. IT managers will be able to sort through real facts based on real research, rather than a bunch of shallow articles and biased reports. Software will survive on its merits alone.
The whole industry is going to benefit by this, in a large, large way. The question one day will no longer be "Microsoft or Linux?" but "Which Open Source software should we use, and why?"
The radical sect of Islam would either see you dead or "reverted" to Islam.
I have a mirror (minus four graphs) :( workin on it...
members.shaw.ca/infonick
You are confusing me with someone who cares.
it appears, fefe's website does not scale well to a slashdotting. :)
Loose: Linux can't obviously not contain any SCO IP, because it's not business ready. SCO's lawsuit will collapse.
The only problem is that the BSD benchmarking might count as disturbing the rest of the dead and will have severe legal consequences.
Owner of a Mensa membership card.
... but I might as well state it too.
He used a development version of freebsd:
version 5.1-CURRENT. That's hardly a fair
comparison, since only the 4.x (and older)
trees have gone stable.
It would be interesting to see the experiment
run after he installs FreeBSD 4.9, now in
RC2 status. (Yes, the 4.9 tree is the
latest stable, even though 5.0 and 5.1 are
out! You don't go by numbers, but by *TAGS*.)
Topic of this paper is
"Scalable Network Programming
Or: The Quest For A Good Web Server (That Survives Slashdot)"
What a coincidence!
By the way, fnord web server has at least once survived one slashdotting-event. 4 seconds of googleing result in this comment which should have let to a stream of visitors.
I hope fefe will publish the numbers of visitors and the behavior of its web server as soon as possible.
Who wants to start taking bets on when Theo takes the bait.
Slashdot's scalability is quite amazing. It seems as a sites ability to resist a slashdotting goes up, slashdot's ability to slashdot a website also goes up. Usually it's at a higher rate but sometimes the sever makes it out alive. Although most of the time it ends up as a smoking pile of slag.
Forget to click 'Post Anonymously', Sir Fagsalot?
No...
I have a complete copy (graphs and all) here.
OpenBSD 3.4 was a real stinker in these tests. The installation routine sucks, the disk performance sucks, the kernel was unstable, and in the network scalability department it was even outperformed by it's father, NetBSD. OpenBSD also gets points deducted for the sabotage they did to their IPv6 stack. If you are using OpenBSD, you should move away now.
Although the results compared with BSD are fairly interesting, the results difference between Linux 2.4 & 2.6 are truely astounding. For those who can't see the graphs or can't be bothered to RTFA...
bind() to port 0
n = number of processes:
2.4: O(n) rise number of processes
2.6: around 0 even with 10000 ports
nmap every other page in a 200MB file:
n = latency in microseconds:
2.4: O(n) rise
2.6: constant (basically no latency)
The kernel guys have certainly done some fantastic work in this last year. I can't wait for 2.6.0!
I wasn't able to see the benchmark results, but I was able to see that the server its hosted on is not very scalable.
Or perhaps your IQ has floated so high that you have loosened perspective? ;)
Seriously, spelling lose with two o's on the internet should be a crime punishable by 300 baud modem. Then the increased upstream latency will make people think twice before typing extra letters.
-The Spelling Police
I hope RMS reads the slides. They're in German at the link I used, so here's a translation of slide 13 which is page 14 of the PDF file:
"The memory required for an empty process is shockingly large on current Linux systems. However, this is not the fault of Linux, it's the fault of GNU libc.
GNU libc leads all libc implementations by a large margin in bloat and waste of memory. One day it got so painful that I wrote my own libc. With this, a static binary of 'Hello world' took only 300 bytes..."
I've long suspected that FSF stands for Fat Software Foundation.
(He doesn't say, but I assume his home-brew libc was a subset, otherwise we'd all want it).
I took three computers out in my rowboat, a Windows system, a Linux system, and a BSD system, and threw them overboard to see what would happen.
The Windows system sank like a rock, the Linux system bobbed back to the surface, and the BSD system rose to the sky, to be greated by a chorus of angels.
Then I woke up, so I don't know what the angels were singing.
Sheesh, evil *and* a jerk. -- Jade
So this guy grabs a beta version of a new tree in freebsd, and runs it against stable netbsd, openbsd and linux? Eh? Did he even compile his own kernel and take out all the debugging information in the released kernel? Did he turn off the debug info in the kernel config, on by default? I kinda doubt it somehow... If he knew enough to do that, he'd know that 5.1 has NOT been tagged stable.
Most freebsd users will look at this study and laugh. But people who don't know anything about freebsd (perhaps that includes the author?) might get the wrong idea. The study needs to be redone using the -stable tree, not the debug-riddled 5.1-RELEASE.
How could a German PDF possibly be interesting?
Of course, its obvious that Haxalot is using multiple accounts to praise himself.
The SIGIO comparison is somewhat fair, but keep in mind that implementing a program with SIGIO requires multiplexing I/O asynchronously, which radically alters program design. Linux 2.4 lacks a synchronous, stateful interface for I/O mulitplexing. Programs that implement synchronous I/O multiplexing can easily transition to a library for their main loop to immediately take advantage of high performance multiplexing/events mechanisms. One such library is libevent.
Sabotage? Someone able to bring me up to speed on this?
As for the installation routine, it is something out of the dark ages, considering even redhat 4.0 - my first Linux installation - had a curses based install. However, if you ignore that, it still seems a decent enough platform - I use it on an Ultra5 at work and as my firewall/NAT gateway at home.
What I'd now like to see, which is what often doesn't occur, is for each OS camp to provide tuning suggestions to the researcher, and have him re-perform the tests. This is because no production webserver is likely to be untuned from the default installation.
Oh and the read-only sysctl problem for FreeBSD that he mentions was probably due to securelevel's being on (meaning you can't modify kernel variables).
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
Using an unstable development version and then complaing about instability, peppering the results with emotive commentary and clueless rhetoric. (btw the 1024-cylinder boot restriction he complains so much about has been fixed for a while) Especially funny was this idiotic statement:
Someone should hit him with a cluestick on this issue. Yeah, like itojun is despicable and unworthy...OTOH, the results are of concern and should be verified by someone less obviously biased. I haven't noticed them in practice on moderately loaded servers though (but I'm biased in the opposite direction).
Not sabotage, security. In case you don't know: itojun is the guy between all the BSD's IPv6 support, and has been very active in the standarisation process.
No. You must be American, as I see you don't understand irony.
I'm wondering, if he was going to be doing a scalability test, why didn't he test the version of FreeBSD that is actually reccomended for production (4.8)?
He had the time to test the stable and devel versions of the linux kernel, but only the new technology version of freebsd?
Evidently his site doesn't handle traffic very well, especially on the Slashdot Scale.
Your hair look like poop, Bob! - Wanker.
you sir are quite astute at locating irrational conjectures - the hallmark of Mac zealots. As a former zealot (Mac free since '98), I thank you for alerting us so that we might retain our grip on reality.
[ed. note: in the following text, former FreeBSD developer Mike Smith gives his reasons for abandoning FreeBSD]
When I stood for election to the FreeBSD core team nearly two years ago, many of you will recall that it was after a long series of debates during which I maintained that too much organisation, too many rules and too much formality would be a bad thing for the project.
Today, as I read the latest discussions on the future of the FreeBSD project, I see the same problem; a few new faces and many of the old going over the same tired arguments and suggesting variations on the same worthless schemes. Frankly I'm sick of it.
FreeBSD used to be fun. It used to be about doing things the right way. It used to be something that you could sink your teeth into when the mundane chores of programming for a living got you down. It was something cool and exciting; a way to spend your spare time on an endeavour you loved that was at the same time wholesome and worthwhile.
It's not anymore. It's about bylaws and committees and reports and milestones, telling others what to do and doing what you're told. It's about who can rant the longest or shout the loudest or mislead the most people into a bloc in order to legitimise doing what they think is best. Individuals notwithstanding, the project as a whole has lost track of where it's going, and has instead become obsessed with process and mechanics.
So I'm leaving core. I don't want to feel like I should be "doing something" about a project that has lost interest in having something done for it. I don't have the energy to fight what has clearly become a losing battle; I have a life to live and a job to keep, and I won't achieve any of the goals I personally consider worthwhile if I remain obligated to care for the project.
Discussion
I'm sure that I've offended some people already; I'm sure that by the time I'm done here, I'll have offended more. If you feel a need to play to the crowd in your replies rather than make a sincere effort to address the problems I'm discussing here, please do us the courtesy of playing your politics openly.
From a technical perspective, the project faces a set of challenges that significantly outstrips our ability to deliver. Some of the resources that we need to address these challenges are tied up in the fruitless metadiscussions that have raged since we made the mistake of electing officers. Others have left in disgust, or been driven out by the culture of abuse and distraction that has grown up since then. More may well remain available to recruitment, but while the project is busy infighting our chances for successful outreach are sorely diminished.
There's no simple solution to this. For the project to move forward, one or the other of the warring philosophies must win out; either the project returns to its laid-back roots and gets on with the work, or it transforms into a super-organised engineering project and executes a brilliant plan to deliver what, ultimately, we all know we want.
Whatever path is chosen, whatever balance is struck, the choosing and the striking are the important parts. The current indecision and endless conflict are incompatible with any sort of progress.
Trying to dissect the above is far beyond the scope of any parting shot, no matter how distended. All I can really ask of you all is to let go of the minutiae for a moment and take a look at the big picture. What is the ultimate goal here? How can we get there with as little overhead as possible? How would you like to be treated by your fellow travellers?
Shouts
To the Slashdot "BSD is dying" crowd - big deal. Death is part of the cycle; take a look at your soft, pallid bodies and consider that right this very moment, parts of you are dying. See? It's not so bad.
To the bulk of the FreeBSD committerbase and the developer community at large - keep your eyes on the real goals. I
It's convinced me 2.6 is a *lot* better than 2.4. I'm not really surprised at the results. FreeBSD has never seemed faster than Linux, while Open has seemed noticeably slower.
Does anyone know why Open underperformed so dramatically? Can the OpenBSD development team comment on this benchmark ?
Well, it didn't. ;-)
Cheers, Ulli
Simple things should be simple, complex things should be possible.
Then its slashdoted :)
well, Down on your knees, for beyond you is the great Slashdotter......
The lunatic is in my head
What We Can Learn From BSD
By Chinese Karma Whore, Version 1.0
Everyone knows about BSD's failure and imminent demise. As we pore over the history of BSD, we'll uncover a story of fatal mistakes, poor priorities, and personal rivalry, and we'll learn what mistakes to avoid so as to save Linux from a similarly grisly fate.
Let's not be overly morbid and give BSD credit for its early successes. In the 1970s, Ken Thompson and Bill Joy both made significant contributions to the computing world on the BSD platform. In the 80s, DARPA saw BSD as the premiere open platform, and, after initial successes with the 4.1BSD product, gave the BSD company a 2 year contract.
These early triumphs would soon be forgotten in a series of internal conflicts that would mar BSD's progress. In 1992, AT&T filed suit against Berkeley Software, claiming that proprietary code agreements had been haphazardly violated. In the same year, BSD filed countersuit, reciprocating bad intentions and fueling internal rivalry. While AT&T and Berkeley Software lawyers battled in court, lead developers of various BSD distributions quarreled on Usenet. In 1995, Theo de Raadt, one of the founders of the NetBSD project, formed his own rival distribution, OpenBSD, as the result of a quarrel that he documents on his website. Mr. de Raadt's stubborn arrogance was later seen in his clash with Darren Reed, which resulted in the expulsion of IPF from the OpenBSD distribution.
As personal rivalries took precedence over a quality product, BSD's codebase became worse and worse. As we all know, incompatibilities between each BSD distribution make code sharing an arduous task. Research conducted at MIT found BSD's filesystem implementation to be "very poorly performing." Even BSD's acclaimed TCP/IP stack has lagged behind, according to this study.
Problems with BSD's codebase were compounded by fundamental flaws in the BSD design approach. As argued by Eric Raymond in his watershed essay, The Cathedral and the Bazaar, rapid, decentralized development models are inherently superior to slow, centralized ones in software development. BSD developers never heeded Mr. Raymond's lesson and insisted that centralized models lead to 'cleaner code.' Don't believe their hype - BSD's development model has significantly impaired its progress. Any achievements that BSD managed to make were nullified by the BSD license, which allows corporations and coders alike to reap profits without reciprocating the goodwill of open-source. Fortunately, Linux is not prone to this exploitation, as it is licensed under the GPL.
The failure of BSD culminated in the resignation of Jordan Hubbard and Michael Smith from the FreeBSD core team. They both believed that FreeBSD had long lost its earlier vitality. Like an empire in decline, BSD had become bureaucratic and stagnant. As Linux gains market share and as BSD sinks deeper into the mire of decay, their parting addresses will resound as fitting eulogies to BSD's demise.
Slashdot commentors: Please realize that many people who visit Slashdot are from countries where English is not the native language. It is VERY likely that they won't understand a subtle joke. Don't become hostile because someone doesn't understand. Just explain.
Open Source software may have changed the way we think about computing, but BSD is not so good. Windows is better right now, at least for x86 processors. They may not like the iPod, but then, the iPod doesn't work on BSD, now does it? I hope every man, woman and child will try BSD at least once so that the userbase will grow. Once that happens, the source code will mature and Windows will go bye-bye.
dd
Even if his webserver didn't scale so well, it doesn't say much about benchmarks, or BSD. The only thing that matters is if BSD can grow it's userbase so that the source code will mature. If it doesn't mature, BSD will be in trouble. Then we will always have to use Windows.
dd
How can he say that with a straight face? Through the results, all he did was bash BSD.
From /usr/src/UPDATING:
NOTE TO PEOPLE WHO THINK THAT 5.0-CURRENT IS SLOW:
FreeBSD 5.0-CURRENT has many debugging features turned on, in
both the kernel and userland. These features attempt to detect
incorrect use of system primitives, and encourage loud failure
through extra sanity checking and fail stop semantics. They
also substantially impact system performance. If you want to
do performance measurement, benchmarking, and optimization,
you'll want to turn them off. This includes various WITNESS-
related kernel options, INVARIANTS, malloc debugging flags
in userland, and various verbose features in the kernel. Many
developers choose to disable these features on build machines
to maximize performance.
Poul-Henning Kamp -- FreeBSD since before it was called that...
You probably mean Canadian
...so how come this wasn't posted to that topic, in addition to Linux, Unix, Software, and Operating Systems?
I knew the Linux 2.6 kernel was supposed to be faster and more scalable, but, damn, that's awesome.
There's no need for such a very technical benchmark.
Regular usage of various operating systems on the same host makes it obvious.
When it comes to speed and features (or bloat), Linux is more efficient than FreeBSD, NetBSD and OpenBSD. This is especially significant in SMP environments.
Linux users are always talking about the just-released experimental patches that will help their system to get 0.1% faster, or the most aggressive flags to optimize their Gentoo system.
BSD users just advocate their system with the generic word "robust".
Nowadays, stability is not really the key. Every Linux or BSD free operating system has basically the same stability. The software is the same, with the same bugs. The package system have equivalents (Debian works on NetBSD, Gentoo works a lot like BSD ports, etc) and support for common hardware is almost identical.
The reason to choose one OS over another is often more political than technical. People tend to use FreeBSD just to try "something else". People tend to use Linux because the Mandrake/RedHat/Conectiva/SuSE installers are beautiful or because Gentoo is fashion and a good way to learn what Unices are made of.
But if this is just to use common software like Apache and Qmail there's no real difference except speed. If this is what you need, Linux is definitely the best choice nowadays, especially since 2.6 kernels are almost ready for production use.
For other needs, your mileage may vary.
For instance I love OpenBSD for development. The compiler and the libc have very handy features to automatically detect bogus code. And the man pages are also excellent, with helpful hints.
For firewalls and trafic shaping, I wouldn't use anything but *BSD because of PF. PF is really the best thing in *BSD systems IMHO. The firewall is very easy to configure yet extremely powerful and fast. And I was fond of Iptables before.
For bridging and transparent firewalls, I would also use BSD because it seems to work better than Linux in this area.
In fact it's just like the girl of your dreams. Everyone's always looking for the perfect operating system that will perfectly fit all needs, but it just doesn't exist.
{{.sig}}
Now, Apache uses a BSD style license but they have an open development model which allows them to take advantage of a very large developer pool in order to stay ahead of their competition. In fact although proprietary versions of Apache exist which perform better than the official releases, SGI has put out some open source patches which generate even larger performance boosts. This is the reason why they have such a strong showing in terms of market share.
BSD once had potential but the procedural problems they are experiencing hurt it when it comes to the market. I suspect that this is probably in part because the BSD teams are not interested in such things, and that is a shame... In fact, although I labeled it as an inferior OS, this is not due to lack of progress within BSD -- it has been progressing somewhat, but rather because all the improvements they make tend to be quickly copied by their competitors AND they lack the developer pool to stay ahead of this game (a problem which does not exist in the Linux or Apache communities, though for somewhat different reasons).
I don't think that there is enough widespread support for BSD to save the operating system. What must be done is an opening up of the development process OR a GPL-style restriction on redistribution. In many ways I favor the former.
Even in a worst case scenario, I don't see BSD completely dying. I think the developers are less into competition and more into a sort of idealized cooperation. As a result, even if BSD becomes more marginalized, I don't think that it will die outright. It will most likely outlive Netware, for example.
I like English humor. When I was in N.Z., I thought the Wizard was funny. (He's English, of course.) Recently he told me he is retiring. No more wizardry. No more British Conservative Army.
Military guy: "Get Linus Torvalds in here!"
"You told us that the new 2.6 kernel will be faster with more efficient web server!"
Linus Torvalds: "It is faster! Over five mill.." BOOM! Headshot!
I think Theo should get off the rags. Hey THEO! Y0u r teh scr3w.
The benchmarks themselves are quite interesting. The conclusions are revolting.
The author blatantly assumes that you only run a system because of performance, performance, performance. That is far from the case.
(I can't speak for Linux, as I'm not a Linux guy... but I seriously doubt recommending all Linux users switch to an unreleased kernel is wise...)
You use the right tool for the right job. So OpenBSD doesn't perform as well as Free or Net. It has a much better security track record. If you need a secure box, use it.
So NetBSD doesn't perform as well as FreeBSD, but suppose you have a hardware platform that FreeBSD doesn't support. Well, NetBSD is a great choice.
There are far too many factors that play into the choice of an operating system. You should pick the one that does the job you need to do best.
As I said, the benchmarks, without all the idiotic commentary, are very, very interesting, however the author really needs to get a clue.
Your scientific method isn't one, and these results are invalid as a result.
You took in-progress development release of OpenBSD compared to stable releases of other operating systems (even your NetBSD was a RELEASE rather than a CURRENT). CURRENT is always going to be an in progress development and not entirely stable.
Computer benchmarking data are (usually) quite noisy. As such plain graphs, like those shown are not very useful. Simple statistical graphics like lowesss plots are much easier to interpret. Take a look at the for some ideas.
This stuff is easy to use, at least if you have a computer background, and would produce far better graphs.
-- Anthony Staines
Well, I'm not entirely sure whether these results demonstrate that Theo is an even bigger dickhead than we all already thought he was, or if this guy is a perfect candidate for a position on Apple's G4 benchmarking and marketing team. Maybe both? Do you have to be able to read docs to market G4s?
I find your ideas intriguing and I wish to subscribe to your newsletter.
Hostility is bad manners, no matter who does it.
There is code to handle booting up there, mostly written by Tobias Weingarter.
However, the bootblock code was written by Michael Shalayeff, who views this as his sole property, and has some real trouble communicating with people at times, and writes completely shitty code because he's lazy.
So, this degenerated into a feud between those individuals, and Theo de Raadt, once again, showed pretty poor managements skills by choosing sides wrongly.
And so, once again, this code only exists in absentia. The situation may be fixed before OpenBSD 3.5. Then again, maybe not.
You *CANNOT* create benchmark tests for OS as disimilar as BSD and Linux,let alone BSD,LINUX and Windows and expect them to have to have any real meaning whatsoever.
They are just too different from each other.
PC Gamers like the idiot who "created" this lame-assed benchmark test really *NEED TO GET A LIFE*
Next time he'll benchmark GIMP 1.2 and GIMP 1.3. Once he finds that GIMP 1.3 crashes a lot and is a memory hog (GIMP 1.3 is a unstable release, by the way), he'll say that GTK 2 doesn't work and we should all use GTK 1.
He needs a big bag of reality.
Also watch out as you read the graphs - just to keep you on your toes, he changes the colors in every one!
The author should use a graphing technique Edward Tufte calls "small multiples". In that you lay out a series of thumbmails of the each graph series. The eye can quickly scan down the thumbmails and get a rough feeling for how each series compares.
This would avoid the problem where graphs overlay each other and with the inconsistency between graphs in the plotting conventions.
Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
From the article:
The OpenBSD and FreeBSD graphs stop early because OpenBSD crashed when I forked more processes, and I couldn't find out how to increase FreeBSD's system limit on the number of processes (sysctl said the value was read-only).
MOD THIS STORY DOWN AS A TROLL! BSD NOT DEAD!
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
I read many comments claiming that linux 2.6 is _much_ better than FBSD 5.1 - re-read the article please. I know its hard for a linux zealot not to exaggerate the results, just as it is difficult for a FreeBSD zealot not to get over-defensive.
;-)
As to the quirks. As someone pointed out, the author should have read the handbook.
Also, the FreeBSD kernel (well, the OS, as the kernel isn't developed separately from the rest of the system) development is more in its earlier stages than linux kernel development. There is no code freeze yet (I think) and I very much doubt that the 5.x branch will be stabilized until we see a 5.3 release. Thats not going to happen until spring (5.2 is due in December). In other words, while 2.6 underwent some stages of optimization, this is yet to happen to the 5.x branch. Yes folks, the results you see from FBSD is still that of an unoptimized kernel - and that's not as bad as zealotry (from either side) would have it,no?
It's this type of information that does OSS a disservice - making it look amateur and unprofessional.
You used an pseudo-scientific method, i.e. your graphs are nice, your data points are nice, but you forgot the fundamentals of testing: you didn't clarify the exact projects you were testing (you should have asked the project leads for advice on which versions of the projects to test against), nor did you asked for feedback and determine the reasons for the anomalous conditions (e.g. the FreeBSD maxprocs could have been resolved by a quick question to a FreeBSD expert). This is a lowbrow technical approach.
Not only that, you presented the pseudo-science results to a large Linux congress, so that while Linux does perform well (undisputed) and everyone can pat themselves on the back, you make the congress look amateur and unprofessional.
The result is that many of us that do know a thing or two about methodology, experimentation, engineering, etc, will simply interpret your results as providing a "rough general guideline", but "leaving too many questions unanswered" and continue to think that "a lot of the linux community is still amateur and unprofessional".
algorithms which would improve Linux 2.6 or FreeBSD in areas the benchmarks revealed to be weak. The pantent filers? Why MS, SCO and Sun, of course. They'll attempt anything to block Linux.
I cannot argue on scalability. Benchmarking should be more, though.
I had been running a SOHO-server on P75, 128MB, Dial-up/DSL, DHCP, Print-server, Apache, SSH, hylafax, Firewall on Linux for some years. Then I tried OpenBSD for a change. It runs with about 1/3 of the memory (Linux around 100 MB, OpenBSD about 35 MB).
Since I installed it, there were just 8 patches to be added (that is about 1 per month); installed pretty easily. I felt quite safe and the system has been pretty stable.
Though I don't see too much of a future in it, I appreciate its safety considerations and - in my case - stability.
My desktops run Linux, but currently I see no contender as replacement for the OpenBSD-box. I wouldn't mind, though, to go back to Linux if there was anything equivalent.
Let's simply assume that OpenBSD will remain in the niche for small (it will probably never be SMP) servers of security-concious admins with low hardware requirements and low throughput.
Only a Slashdotter would compare operating systems to girls...
If you're talking about "professional scaling" then ok, 2.6 will be great.
I did reinstall Mdk 9.1 with jfs and, while it works seamlessly, it's doesn't feel any better than my other reiserfs partition. Later I discovered that at my scale (desktop, somewhat lousy machine) neither jfs or xfs can't really show their abilities.
I can recommend Linux at my work now, many thanks for the benchmark; nonetheless, on the desktop, we don't have that many processes... we need some things to get snappier (Gnome/KDE and Oo.o come to mind).
In other words, this is for the extreme performance fanatics, don't believe this applies to "your" situation. But it surely is invaluable if you run big business.
I am not pretending to produce the be-all end-all benchmark here. That would at least include having an SMP server for the operating systems that support it, and it would have used Gigabit Ethernet. Frankly, I don't have the hardware to do that.
;-)
But I do have these benchmark programs, and I was hoping to spark enough interest in doing some real comparative benchmarks by posting the results here. The tools are now there, and they do use the platform specific API hacks to get the best performance on each operating system, so the biggest area of concern for comparative benchmarks should be out of the way -- that the benchmark program is skewed in one direction or the other.
If you are in possession of a real benchmarking lab with enough free disks and spare time on your hand to complete these benchmarks, please do so!
I'm not from the Linux mafia. If my goal was to discredit BSD, I would not have added support for kqueue in the first place. If my goal was to pat myself on the back, I would not have used the BSD sysctl to increase the number of files in the system (on Linux 2.6, you basically don't have to do that, just use ulimit), and that would have been fair, if only in the Netcraft/Microsoft sense.
But neither am I from the Intel scalability labs with 1000 identical PCs and multiple gigabit ethernet connections to the server. I'm just a guy with a notebook who happened to have a little spare space on his hard disk and decided to include the BSDs in the benchmarks.
I don't want to start a holy war here, but what is the deal with you BSD fanatics? I've been sitting here at my freelance gig in front of a BSD box (a PIII 800 w/512 Megs of RAM) for about 20 minutes now while it attempts to copy a 17 Meg file from one folder on the hard drive to another folder. 20 minutes. At home, on my Pentium Pro 200 running NT 4, which by all standards should be a lot slower than this BSD box, the same operation would take about 2 minutes. If that.
In addition, during this file transfer, Netscape will not work. And everything else has ground to a halt. Even Emacs Lite is straining to keep up as I type this.
I won't bore you with the laundry list of other problems that I've encountered while working on various BSD machines, but suffice it to say there have been many, not the least of which is I've never seen a BSD box that has run faster than its Windows counterpart, despite the BSD machines faster chip architecture. My 486/66 with 8 megs of ram runs faster than this 800 mhz machine at times. From a productivity standpoint, I don't get how people can claim that BSD is a "superior" machine.
BSD addicts, flame me if you'd like, but I'd rather hear some intelligent reasons why anyone would choose to use a BSD over other faster, cheaper, more stable systems.
In The article, The author mentions that he finds "unacceptable", or embarassing, a few things about openbsd, mainly concerning it's general speed and scalability.
ALthought not beeing an expert in bsd nor in linux, maybe i'm rong, but isn't OpenBSD made to be secure, and not the fastest operatin system ( additionally, comparing oragnges to apples, by testing release/current/stable, wasn't the best way of comparing those OSes) ????
By implementing a few feature in the OpenBSD stack and Kernell, I guess that the devellopers are delibetatelly sacrifying performance in order to get an Os less vulnerable to DOS and other vulnerabilities
It owuld be interesting to compare the same OSes in a security test to find out which OS is more secure, if speed and scalability isn't the only issue in a OS...
OpenBSD 3.4 was a real stinker in these tests. The installation routine sucks, the disk performance sucks, the kernel was unstable, and in the network scalability department it was even outperformed by it's father, NetBSD. OpenBSD also gets points deducted for the sabotage they did to their IPv6 stack. If you are using OpenBSD, you should move away now.
...this guy does some default installations of mostly experimental releases and then wonders why things are broken. Shows your intelligence...
For example: if you find the installation routine of OpenBSD difficult, stick with freakin' RedHat Linux and their wonderfull GUI.
For example: if you don't bother to even read the documention on the various OS's releases, stick with Microsoft Windows where you can click 'yes' or 'no'.
I'm surprised that someone with such credentials (he knows unices thoroughly - or so it seems) makes such obvious mistakes.
I wonder how much this guy got paid and by who...
it's an interesting benchmark but I think most people would like to see it compared with windows server.(on a "standard" server) I don't know if that would be easy to do as the api are not the same.
The autor seems to assume windows is a lot slower but they are not real benchmark about it. (I know a benchmark is only a benchmark and not real life situation)
Arbeit macht frei!
Fuck Beta. Fuck Dice
Oh, wait, but we are comparing those dissimilar products when they're all attempting to do the same thing. Why else would car makers boast about engine horsepower, acceleration, traction and braking, for example?
I guess you didn't actually read the article, where the guy states that he's doing the same damn thing on each OS, using the same hardware, so the only variable is the OS/distro being used. His choice of OS/distro may be suspect, but the results still stand. He's comparing oranges to oranges and telling you which is sweeter, which is juicier, etc.
Sure, the comparison is valid, you can compare those elements, but are the results relevant? Not many would use OpenBSD in a performance (as in speed) critical role, and not many would expect Linux to have the security track record of OpenBSD.
If I'm going to war, I'll take the tank. If I'm going to race, I'll use the sports car. And if I'm taking 40 friends to the mountains for some skiing, count me in on that bus.
Does everything include nothing?
Linux end users on average tend to be the scum of the earth with median level IQ's. Ever witness the various linux ghettos on irc or usenet ? You cannot miss them. So it doesn't surprise me that this person was unable to conduct an "Empirical scientific double blind benchmark". Hence the oddball subjective extraneous derivatives of his so called "research" on these benchmarks are pointless. It serves only to satiate the need for lust in the lackluster mind of the mass linux user base --- that is the sole purpose of this propaganda.
.
.. total lack of consistency from a linux E-tard. He says at other times he was "surprised" that NetBSD outperformed and even beat the others clearly on benchmark to do with sockets. That would have come as no surprise to a serious engineer who had the integrity and quality of character to study the code changes to each OS by reading the mailing lists, developer change notes and even reading source code. The NetBSD socket performance win was clearly because of Jason Thorpes hack here :
/ 07 /03/0011.html
In order for this study to be worth a damn he would have to:
1.) Use two computers with the same hardware from two different manufacturers
2.) Have the common decency to do thorough research which I will prove he does not do before benchmarking.
3.) Email head developers from each operating systems on tips on tweaking each OS and what versions of each OSto use in each benchmark. If he still decided to use bleeding edge from both linux and the *BSD's-- he must do so consistently across the board. Obviously he didn't because he didn't use NetBSD 1.6ZD (current) from fresh cvs diffs. Strangely enough he went with -current for both OpenBSD and FreeBSD though. The only phrase that comes to mind is "highly subjective propaganda produced by a plebeian simian who was not even clever enough to mask his propaganda with at least the nomenclature of the scientific method. How hard would it have been for him to email Luke Mewburne, Miod, and someone from FreeBSD ?
He is obviously experienced with linux and semi with FreeBSD (although not really because he was too stupid to set certain tweaks). While totally clueless in the NetBSD and OpenBSD environments.
"In the mean time I heard that the -CURRENT version of NetBSD has over two years worth of improvements in it, so it probably is even better than NetBSD 1.6.1, which already surprised me with its good performance (although it is clearly outperformed by FreeBSD)." -- fefe.
Then you have no excuse for using the bleeding edge of Linux and FreeBSD
http://mail-index.netbsd.org/current-users/2002
"The clear loser of this benchmark is NetBSD, because they simply don't offer a better API than poll. As I wrote in the introduction, I only benchmarked the stable NetBSD 1.6.1 kernel here,
and I assume they have included kqueue in their -CURRENT kernel. I will try to update my NetBSD installation and rerun the benchmarks on it."
Gee, you think fefe ? You assume they have included a native kqueue ? Ah you mean you are a complete moron who couldn't be bothered to actually be bothered with real "research" and "benchmarking". Clearly it wouldn't have taken long to research and be able to say with authority "NetBSD has native kqueue it was added to the code on such and such a date by developer X. Just for the record NetBSD current does have native kqueue and native pthreads.
I bet you didn't even bother to tweak NetBSD at all as you barely tried with FreeBSD. Note that it would have been a good idea to tweak
Kern.maxfiles, kernmaxvnodes and vm.anonmax with sysctl...
This only confirms what we allready know : The vast majority of linux users are inexperienced clueless unprofessional zealots.
The benchmark looks pretty straightforward and valid to me.
Straightforward: Would you rather see a test of a current linux 2.6 against an in-a-half-year F-BSD? Or a half-year-old linux 2.6 against a current F-BSD? If so, come off it. Run the benchmark again in half a year with then-current kernels and maybe it'll be some BSD variant taking the gold.
Valid: IMHO a benchmarks validity is very closely related to how it compares to real-life scenarios. It's ok to tune all readily available parameters and apply a common patch or two (let's say up to ten). A benchmark of a system with 200 custom kernel hacks and installed with support from the development team are absolutely unrealistic and worthless to any lowly sysadmin the likes of mine. I'm not interested in what could be done but what can be done within reasonable time and cost limits.
You linux and F-BSD crusaders can have it out all among yourselves whether the systems were tuned optimally. For me, the system test is ok if the cost of time spent for tuning is equivalent to or outweighed by the cost of more performant hardware. This is not directly documented in this test, but the tuning steps described look realistic in this aspect.
Back to you, guys
Ho
Mac OSX server is based on Free Bsd5
other web services running out of the box
Apache 1.3 and 2.0
Tomcat 4.1 and Axis 1.1
JBoss 3.2
MySQL 4
JBoss Deployment Tools
WebObjects on JBoss
Perl 5.8 and PHP 4.3
Ldap, samba
Some drink at the fountain of knowledge. Others just gargle.
Linux end users on average tend to be the scum of the earth with median level IQ's. Ever witness the various linux ghettos on irc or usenet ? You cannot miss them. So it doesn't surprise me that this person was unable to conduct an "Empirical scientific double blind benchmark". Hence the oddball subjective extraneous derivatives of his so called "research" on these benchmarks are pointless. It serves only to satiate the need for lust in the lackluster minds of the mass linux user base --- that is the sole purpose of this propaganda.
.
.. total lack of consistency from a linux E-tard. He says at other times he was "surprised" that NetBSD outperformed and even beat the others clearly on benchmark to do with sockets. That would have come as no surprise to a serious engineer who had the integrity and quality of character to study the code changes to each OS by reading the mailing lists, developer change notes and even reading source code. The NetBSD socket performance win was clearly because of Jason Thorpes hack here :
/03/0011.html
In order for this study to be worth a damn he would have to:
1.) Use two computers with the same hardware from two different manufacturers
2.) Have the common decency to do thorough research which I will prove he does not do before benchmarking.
3.) Email head developers from each operating systems on tips on tweaking each OS and what versions of each OS to use in each benchmark. If he still decided to use bleeding edge from both linux and the *BSD's-- he must do so consistently across the board. Obviously he didn't because he didn't use NetBSD 1.6ZD (current) from fresh cvs diffs. Strangely enough he went with -current for both OpenBSD and FreeBSD though. The only phrase that comes to mind is "highly subjective propaganda produced by a plebeian simian who was not even clever enough to mask his propaganda with at least the nomenclature of the scientific method. How hard would it have been for him to email Luke Mewburne, Miod, and someone from FreeBSD ?
He is obviously experienced with linux and semi literate with FreeBSD (although not really because he was too stupid to set certain tweaks). While totally clueless in the NetBSD and OpenBSD environments.
"In the mean time I heard that the -CURRENT version of NetBSD has over two years worth of improvements in it, so it probably is even better than NetBSD 1.6.1, which already surprised me with its good performance (although it is clearly outperformed by FreeBSD)." -- fefe.
Then you have no excuse for using the bleeding edge of Linux and FreeBSD
http://mail-index.netbsd.org/current-users/2002/ 07
"The clear loser of this benchmark is NetBSD, because they simply don't offer a better API than poll. As I wrote in the introduction, I only benchmarked the stable NetBSD 1.6.1 kernel here,
and I assume they have included kqueue in their -CURRENT kernel. I will try to update my NetBSD installation and rerun the benchmarks on it." -fefe
Gee, you think fefe ? You assume they have included a native kqueue ? Ah you mean you are a complete moron who couldn't be bothered to conduct real "research" and "benchmarking". Clearly it wouldn't have taken long to research and be able to say with authority "NetBSD has native kqueue it was added to the code on such and such a date by developer X". Just for the record NetBSD current does have native kqueue and native pthreads.
I bet you didn't even bother to tweak NetBSD at all as you barely tried with FreeBSD. Note that it would have been a good idea to tweak
Kern.maxfiles, kernmaxvnodes and vm.anonmax with sysctl on NetBSD.
Also, why mention itojun on ipv6 but leave out other developers names where NetBSD and FreeBSD win on the benchmark ? He only goes on extravagant explainations on why OpenBSD loses in one scenario. His need to be verbose only in such a situation shows his linux bias or guilt for skewing the benchmark so bad. Oth
Felix is more than opinionated about Linux vs. BSD. Check the qmail mailing list archives for his rants with BSD users about filesystem and other performance issues. In particular, he's had a number of run-ins with those on the list who use OpenBSD--the one he labels a `stinker'.
He doesn't give details about the installation of each OS. Did he benchmark them one by one after wiping out the previous OS? Or did he just partition the disk and put each OS in its separate partition? My impression is that he did the latter though I may be mistaken.
Any hardware junkie knows that data reads and writes can be upto 2x as fast at the start cylinders of a disk than at the end. If he put FreeBSD in the beginning and OpenBSD at the end, it would really skew the results.
Why would anyone ask a rhetorical question?
Empirical professional double blind studies ELSE >> /dev/null !
I'm guessing....that you did not.
Conclusion: NetBSD outperforms all other operating systems in this benchmark. However, all contestants scale equally well, there are no clear losers. And the overall latency in this benchmark is so low that the results are interchangeable in practice.
The first graph, which this quote is under, does not support this. It clearly shows Linux 2.6 using HALF the CPU cycles of NetBSD above 4000 connections. How did he get this conclusion? 2.6 is the clear winner in his tests.
JON
Right, of course... and the fact that most code compiles out of the box for both operating systems with little to no modifications tells us what?
Oh, right.. That the OS' are just way too dissimilar and give no meaning whatsoever!
You need to learn a little about computers and logic, son.
So what about the comparison between FreeBSD and Linux? That was probably the main reason for this benchmark, with NetBSD and OpenBSD being thrown in there for reference sake.
Everyone's always arguing that FreeBSD is faster than Linux and now the results show a different story. Maybe it will change when the stable releases of each OS are available, and maybe it won't. But this is how it is NOW.
"The first graph, which this quote is under, does not support this. It clearly shows Linux 2.6 using HALF the CPU cycles of NetBSD above 4000 connections. How did he get this conclusion? 2.6 is the clear winner in his tests."
Yes, keyword being *HIS*. As in "his" neophyte idiosyncratic subjective unprofessional non empirical study with no double blind safeguards. In otherwords the benchmark is totally pointless to any real intellectual not bound up in emotional ideological hangups.
This turd is worse than Hanzo-San. Even the troll community hates him.
Aren't amatuer and unprofessional synonyms for "linux user" ?
You want to know why linux users don't do all that? We're EMPLOYED. We don't have time to spend 60 hours pouring over flame wars in every mailing list and developer note.
Despite that, this guy managed to put together some decent numbers. I note that you haven't. It's much easier to tell someone "You should have spent every waking moment on your life on these benchmarks!" when you haven't bothered to do so yourself. Fefe put the code up for download. Run your own test. Or are you afraid that even your supposed uber-tweaks from bowing at the feet of BSD developers aren't up to the task and will only prove the hollowness of your own existance?
Or perhaps you should be a reasonable person and take Fefe's benchmarks for what they are: A scalability test. He wasn't after whether BSD or Linux could be tweaked to have numbers 10% better than the other. He was after whether they scale at O(n) or O(1). That's something that usually pretty obvious, and it shows from the graphs. Tweaking the kernel a little bit won't change it.
Kern.maxfiles, kernmaxvnodes and vm.anonmax won't change a graph from angled to flat. They may lower the angle, but it doesn't matter. Scalability is all about O(n) or O(1). That's what Fefe was after, and that's what he recorded.
The interesting point is that all of these operating systems seem to be getting faster. It seems to come down to how many recent developments have been integrated into the version being tested, not any inherent differences between operating systems. This is, of course, as it should be: the source for all of these operating systems is available, and there are even frequently papers describing the techniques. If a technique is, in fact, better, it should eventually be adopted by all of them, and so your results will depend on how much has been adopted in the version you're testing.
It is encouraging to see that all of these developers are competing with the real opponent, which is not each other or even Microsoft, but the slashdot effect. After all, the goal should not be simply to be better than the others, but to be sufficient for the user's purpose, which is not hampered but rather assisted by sharing all of your tricks. It can sometimes seem like there are endless wars between Linux and BSD, but, behind the scenes, the sides actually share information. Never as much as they'd like, but always more than people think.
I'm the Bolshevik RMS getting clubbed in the head by a Waffen-SS soldier on the eastern front !
In reference to a particular metric (not the entire suite of tests) he says "Even Windows would probably outperform OpenBSD"
Talk about taking the kids gloves off!
-Adam
Comment removed based on user account deletion
I know this benchmark is about open source and all, but I would love to see Solaris thrown into the mix. With all its vaunted scalability and stability, I'd love to see what it actually does better. I guess it would have to be the Intel version, but I would think their kernel algorithms should be the same across architectures.
Agreed I could have added that into my points i.e
."
/usr /etc /var etc... are placed on the geometry of the HD effects performance to a great degree as well. The automatic option in the FreeBSD installer i.e [A] tries to take advantage of the points you make above for the purpose of speed. So if FreeBSD wasn't closest to the start cylinders then it would perform less well oveerall.
" 1.) Use two computers with the same hardware from two different manufacturers
I should have mentioned that each time the hard disk should have been reformatted and each OS should have the benefit of consisting of the whole hard disk. Also, where
Like I said this benchmark is pointless and unprofessional.
He speaks truth!!!!
clifgriffin > blog
When will Slashdot learn that I have taken over Clif Griffins mind and replaced it with a soy-based substitute?
You read too much marketing literature and misinterpret it. To shatter some illusions, Cheer detergent will not make your relationships better. Drinking lots of beer will not attract 5'10", 100lb models with large breasts and bikinis.
Further:
NeXTStep was a Mach kernel with BSD4.3 userland tools running a proprietary windowing system that uses Display PostScript (making their printer cheap) coded with Objective C.
Mac OS 10.0-10.2 is a Mach Kernel with a userland MOSTLY from FreeBSD 3.x running a proprietary windowing system that uses Display PDF (making printing to PDF trivial) coded with Objective C
Note a similiarity?
Panther finally uses FreeBSD 4.x userland tools.
Plus much tuning to their proprietary windowing system.
Still a mach kernel. Note that the word mach contains neither a "B" an "S" nor a "D".
Panther (NeXTStep 7.0.0 according to uname) would still crawl on my NeXT Cube that runs NeXTStep 3.3 just fine at 33MHz.
RE: Your list of stuff taht comes with it. You get 3 for 10 right. perl is
% perl -v
This is perl, v5.8.1-RC3 built for darwin-thread-multi-2level
Apache 1.3.28, no apache 2.x
No sql server (postgres would be a better fit for their completely free software needs).
I don't find jboss nor webobjects nor tomcat on my Panther box.
LDAP *clients* (everyone has ldap clients these days)
Samba, yes.
PHP, yes.
Please try again with some correct facts.
I for one had a terribly hard time reading the graphs. Three problems in particular:
1)
I (like about 20% of the male population) am partially color deficient.
The graphs' colors where very hard for me to distinguish in the ledgend; what with the 1 pixel wide font and sample line/symbols. The colors chosen were all of about the same intensity/saturation also, so if you are color blind the entire graph would look all about the same level of gray. This problem is further compounded by all the graphs having different color assigments (Linux 2.6
2)
crowding. On a few of the graphs, there were differences in order of magnitude between groupings. Since this dataset was supposed to show the fastest, that would naturally be the most interesting data. You don't take photo finishes of the 7th and 8th finishers in an 8 horse race. Using a logarithmic scale for those charts would make the data easier to read, showing detail in the lower(faster) end and allowing the less significant higher(slower) data to "blend" in the curve.
3)
Point size. The symbols are too large for that many data points. This may be personal preference, but single points should be used for scatter graphs like these, not symbols. Symbols may mask other more subtle changes in the results because they overflow other points nearby. Similarly, make your legend samples substantial, a block of color is much more apreciated than a single pixel line, and bold your identifier text.
If you make graphs for the web, please take them in to Photoshop, GIMP, or something similar and reduce or remove the color data (convert to grey scale). The chart should still be at least fairly reable in greyscale.
Choose a wide range of Hue, Saturation and Lightness
If you are going to use light/desturated colors, use a black background. If you will use dark/heavily saturated colors, use a light background.
If you can't/won't do that, then please try to make the raw data available in CSV or some other "universal" format so we can create our own graphs.
Article X: The powers not delegated... by the Constitution...are reserved...to the people
Ha, well, i don't know too much about the internals of kernels and OSs and stuff, but hey, why doesn't somebody take the relevant code of the speediest winner of each of these "tests" and smack it all together into one "Performance-enhanced OS"... ha, plz no flames! i understand this would not be a light undertaking, but it is an idea i suppose.
This is great if all your reads and writes are like this.
I deal with huge database servers that get 10 or more times throughput using dd(1) than they do in acual use with that pesky real data.
Also wondering why I'd care about the CPU on a copy.
Give me a laptop disk on a 3GHz i386 and I'll beat it with a sparc 10 (125MHz) attached to a nice Baydel RAID box (sustained real world writes of > 40-50MB/s with RDBMS systems).
Ah, so you choose to assume *BSD hackers and users are unemployed. Could it just be that not all featherless bipeds are mentally equal ? I think the latter is more likely. You choose to skirt around most of my points and just take up the specific sysctl tweaks I mentioned. Sorry , but life doesn't work that way. You don't get to choose what points you want to respond to and win the whole argument. What is this candy ass land ?
He benchmarked the changes-hourly, may-not-even-build-this-week development branch of OpenBSD
and it crashes? I'm surprised it compiled at all.
Fuck foreigners. Any pansy who can't use english, the language of the internet, should suffer. Not the vast majority of people. Until GNU learns this, their shit will always be slow. Linus knows this. Ulrich doesn't.
This type of benchmarking serves a great purpose for all involved. The Linux kernel folks get a good comparison of their successful work on the new kernel and the BSDers, particularly the OpenBSD folks, get a heads up on problem areas to focus on.
This guy is not a microsofter, trying to convince you to drop one for the other, he's just giving out basic benchmarks and opinions.
When he didn't configure a BSD kernel the right way (maybe he did, maybe he didn't) the BSDers should take the clue and improve the documentation for the more arcane areas of tweaking performance.
Don't flame him, just SLASHDOT him! It's more fun.
PS: I'm really not an anonymous coward, I just forgot my password. (My main PC - where the password is insecurely stored in a cookie - is running an "emerge sync -u world" right now and I didnt' want to waste cpu cycles with a gui. I'm also too lazy to copy the cookies.txt to this PC. Oh well...)
OSX 10.2 Server. I'd imagine it comes with some RDBMS, maybe jboss/tomcat, etc. OpenDirectory is OpenLDAP with a custom backend and slightly different schema I belive.
- RustyTaco
Is it really so hard to read documentation and present a straightforward comparison? We're all on the same team here, yet I consistantly see the same BSD FUD spread around here on Slashdot unabided.
Grow up already.
I just wanted to point out that it most likely wasn't the OS that was his problem.
And yes neatly stacked data, and I wouldn't get even close to such write speed, I barely reach it with reads.
On the other hand, this system would cost 500 USD if built today
But 20+ minutes for 17 Megs? I'd say something is badly wrong.
And I don't think it's the OS.
My 486 DX2/66 copies 17 Megs in less than 30 seconds (and it's running OpenBSD).
Executive Pope (small) Kallisti Engineering
Calling him a fag is giving him too much credit.
He specifically didn't test 4.8 because he was told 5.1 would scale better. He knows 4.8 is more stable, that wasn't really the point of the article. 5.1 did an excellent job.
Fuck Beta. Fuck Dice
There's also some distros out there that have it included by default. So don't say nobody use it, because that's bogus. Anyway it's only one part of the equation and MAC is important too, but obsd doesn't have that. Oh yeah, it has systrace, but that's so damn hard to use nobody use it. Oh yeah, that means obsd security is shit. See, that's what happens when you generalize everything.
That made me laugh our loud. You hit it on the head. Thanks...
Who is this Anonymous Coward character, how does he post so much, and why is he always such a whore?
I read the tests and many of the comments on them.
/if/ the tester happened to be more familiar or friendly with one of the OS's tested. That would really help improve the value of the test. Unprejudiced==scientifical==professional==a virtue.
I was very curious about the test, but very disappointed with everything after I read it.
The person who performed the tests was very naive when he said this ought to be the end of flamewars about network performance/scalability among the Linux/*BSD users. His test was just not good enough to be meaningful to anyone in the respect of offering acceptable conclusions.
A proper test would be
- Performed on proper hardware. I mean, doing a network performance test on an old laptop with inadequate harddisk size, with four very much server-oriented operating systems, one of which (OpenBSD) couldn't even be installed on the same part of the harddisk as the others! That's like organising a race between a Lamborghini, a Lotus, a Porsche and a Ferrari in downtown New York during rush hour, and disqualifying the Ferrari on the grounds that it is difficult to park backwards.
- Unbiased. This one was very much biased against OpenBSD, and in favour of Linux/'Leanux', as follows from many of the comments made by the tester.
- Performed by somebody who knew enough about installing and running all of the OS's involved to run tests on them (the tester seems knowledgeable about Linux, but is totally clueless as far as for instance OpenBSD is concerned.)
- Performed with a test programme that wasn't developed with a bias toward one of the OS's in the test (the test programme involved was developed on Linux, later 'ported to' *BSD.)
- Described plain fact, by someone who would be objective and who would avoid showing emotions about the subject, much less a general favour or disgust toward the OS's tested. This is especially important because of the sensitive nature of the test subject. There are so many flamewars already!
- Described withoud prejudice even
- Carefully giving minute details about the test conditions (hardware, software, test programme details, OS installation details..) This test wasn't remotely accurate, look how it even fails to mention at what time OpenBSD-CURRENT was downloaded. That's crucial information.
- Compare equally. The test is already invalid because it compares an ancient -STABLE NetBSD with a -CURRENT FreeBSD.
- Be clear about its subject. The test focuses exclusively on network performance/scalability, and then goes on to praise or totally disqualify the OS's tested solely on this ground. The test doesn't look at crucial aspects like security, maintainability, documentation, correctness of design, etcetera, but will still not hesitate to draw very un-subtle conclusions.
- Ask those who are in the know for comments on the test conclusions before making the results public.
In this precise case, it would have prevented several stupid factual mistakes from being published (the OpenBSD installation problem and IPv6 idiosyncrasies for instance.)
- Call into the test all relevant players, or at least represent different groups properly.
Much as I dislike it - I'm an open source adept as well - Windows is used as a server system on the public internet by many organisations. It should be compared with the other systems in this test, and be given a fair chance. I wouldn't have been surprised if it performed very well, seeing that the TCP/IP stack of modern Windows versions has been largely copied from FreeBSD..
Also I would have liked it if there were at least one proprietary Unix system (such as Solaris) in the test. Just for the sake of the comparison.
- Look closer into the reasons, backgrounds, pros and cons of faults that the tests find. Again, if the tester had done this, he would have found that some of the badness he found wasn't a design mistake, but a design decision based on healthily made trade-offs (security trade-offs in the case of, not surprisingly, OpenBSD.)
where his data is incorrect or his conclusions are wrong.
remember to include your own data supporting your assertions.
Some Google searching turned up a thread discussing this, and as suggested there I replaced -lpthread with -pthread in the Makefile. It built fine after that.
Enjoy your job, make lots of money, work within the law. Choose any two.
Hmm, perhaps I should have been more obvious about that :) I figured you have to try hard to mess up the spelling of Mensa, especially in a post about spelling. On the Internet, spelling 'lose' incorrectly is somewhat of a phenomenon, a mistake that's easy to make because the phonetics come out 'oo'. I admit that I have thought about comitting that mistake once or twice :)
A month or so ago someone on Slashdot posted an inspiring flame about how people very often spell 'lose' incorrectly, so I thought I'd take my turn at carrying the torch, especially when I saw the Mensa member bit. If people in the top two percentile of intelligence engage in persistent spelling mistake, what hope is there for the rest of the world? Card carrying Mensa people should be ones to set a standard in things like spelling, learning from others' horendously common mistakes, and humility.
For most people, I make allowances, since not everyone has an eye for spelling. But my faith in humanity is pretty low and I have to find hope somewhere. Admitdetly, I don't even have a great deal of confidence even in 98th percentile IQ testees, but that is another matter.
The 2nd most common spelling mistake I can think of is one of ignorance, and not accidental: that of spelling 'a lot' as one word instead of two. But if you inadvertantly spell Mensa with to e's, you probably have peanut butter stuck in your keyboard or something. That makes me think... if Star Trek style teleportation existed, I wonder if there would be mischievous people who teleported bits of peanut butter into people's brains at the exact right spot so that they always spell 'lose' incorrectly. Or maybe there already are! Hmm I must go think now.....
I sympathize for you. I know it hurts to have your ass kicked by Linux.
It is disappointing to see many of the comments here. Some are good and beneficial comments, others are not.
/. .
Where is the spirit of Sharing Ideas here?
#Define Flame On
I suggest that some here Grow Up! Zip your pants up, and quit the p*ss*ng contest. Your 15 seconds of fame are over! Now start acting like team players in this world!
#Define Flame Off
How about several of the people here that apparently seem that they know their stuff the best (and they may), smash heads together to come up with some benchmarks that test appropriately each of the different Operating Systems. I do mean all of the different OSs, such as OS/2, HPUX, Solaris 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9, Windows (3.1, 95, 98, 2000, NT, etc....) Linux 2.1, 2.2, 2.4, etc... FreeBSD 4.8, 4.9, OpenBSD, NetBSD, etc.....
I would suggest the following:
1. Get some people together to come up with a general test that measures specific areas that the kernel is programmed for. These would be tests that would be subject to each OS developers approval (ie, that they test appropriately the area specified). Also, acquire each OS tuning parameters for each specific test.
2. Use these tests to benchmark where an OS has issues.
3. Forward these tests to developers who care about the OS they program, to use these as starting points where code may be improved.
4. Post the tests for each OS. Both before and after fixes.
Do I know what I am talking about? Probably not, but I was a test engineer for several years and we did perform quite a few test just like Fefe did to see what would happen. As time went on, yes we came up with better and better tests. Accept this fellows test as for what it is, a start of people getting together to put together better and better tests. By the way, he did a fairly good job for one that has gone out of their way to help the nerds out there think about things, some apparently think, while others just like to
Which OS do I use? Which ever is handy at the moment! Sometimes we don't get a choice!
HGTTG: "I knew that there was something fundementally wrong with the Universe."
Mach is a microkernel. There are a lot more pieces that compose whole kernel.
From the The Unofficial Darwin FAQ:
# What is Darwin's lineage? Justin Walker's description of Darwin's heritage is: Mac OS X began life as a child of OpenStep 4.x. The first stage in the evolution was the move from OpenStep 4.x to Rhapsody, which was based on BSD Lite2, with a batch of NeXT-instigated changes. When we shifted to Mac OS X from Rhapsody/Mac OS X Server, we incorporated FreeBSD 3.2 changes for the networking piece. The rest of the BSD portion of the kernel remained more or less as it was. At the same time, we (i.e., Fred, with your [Darwin's] help) pulled in command and library updates. Most of these are from FreeBSD, although I'm not positive about the heritage of the pieces that are now in the system.
Please try again with some correct facts.
Yes, please ;))
get a life
and no, berratting people on slasdot does not count.
Redoing all those benchmarks without the strange libraries that the author uses and instead using plain syscalls shows that what the benchmarks measure is how the benchmark code performs, not the actual syscalls.
A real bind benchmarks has a O(1) time on OpenBSD and NetBSD.
The connect latency benchmark seems seriously confused. And since I don't find any source for it, I must conclude (based on the rest of the benchmarks) that it must be flawed in some way.
The "read one byte from each page" benchmark on OpenBSD performs that badly because the file used doesn't fit in the buffer cache.OpenBSD doesn't use all the memory for the file cache. When the file fits in the buffer cache OpenBSD has a performance comparable to linux 2.4. Anyone with half a clue about how a computer works would have listened to the disk and noticed that it's making noise.
And the rants about ipv6 only show that he doesn't know how to write proper ipv6 code and instead relies on a compatibility interfaces that were created for lazy programmers to allow for quick conversion of old ipv4 programs into ipv6.
The author is clueless and the benchmarks more often measure the performance of the benchmark itself than the operating system.
No MrCuck, we're cutting you off. Cause you'rs head is up your ass. all of the posted version numbers for panther server were 100% taken right off the apple web site. you have not a clue what you are blathering about.
What We Can Learn from BSD
By Chinese Karma Whore, Version 1.0
Everyone knows about BSD's failure and imminent demise. As we pore over the history of BSD, we'll uncover a story of fatal mistakes, poor priorities, and personal rivalry, and we'll learn what mistakes to avoid so as to save Linux from a similarly grisly fate.
Let's not be overly morbid and give BSD credit for its early successes. In the 1970s, Ken Thompson and Bill Joy both made significant contributions to the computing world on the BSD platform. In the 80s, DARPA saw BSD as the premiere open platform, and, after initial successes with the 4.1BSD product, gave the BSD company a 2 year contract.
These early triumphs would soon be forgotten in a series of internal conflicts that would mar BSD's progress. In 1992, AT&T filed suit against Berkeley Software, claiming that proprietary code agreements had been haphazardly violated. In the same year, BSD filed countersuit, reciprocating bad intentions and fueling internal rivalry. While AT&T and Berkeley Software lawyers battled in court, lead developers of various BSD distributions quarreled on Usenet. In 1995, Theo de Raadt, one of the founders of the NetBSD project, formed his own rival distribution, OpenBSD, as the result of a quarrel that he documents on his website. Mr. de Raadt's stubborn arrogance was later seen in his clash with Darren Reed, which resulted in the expulsion of IPF from the OpenBSD distribution.
As personal rivalries took precedence over a quality product, BSD's codebase became worse and worse. As we all know, incompatibilities between each BSD distribution make code sharing an arduous task. Research conducted at MIT found BSD's filesystem implementation to be "very poorly performing." Even BSD's acclaimed TCP/IP stack has lagged behind, according to this study.
Problems with BSD's codebase were compounded by fundamental flaws in the BSD design approach. As argued by Eric Raymond in his watershed essay, The Cathedral and the Bazaar, rapid, decentralized development models are inherently superior to slow, centralized ones in software development. BSD developers never heeded Mr. Raymond's lesson and insisted that centralized models lead to 'cleaner code.' Don't believe their hype - BSD's development model has significantly impaired its progress. Any achievements that BSD managed to make were nullified by the BSD license, which allows corporations and coders alike to reap profits without reciprocating the goodwill of open-source. Fortunately, Linux is not prone to this exploitation, as it is licensed under the GPL.
The failure of BSD culminated in the resignation of Jordan Hubbard and Michael Smith from the FreeBSD core team. They both believed that FreeBSD had long lost its earlier vitality. Like an empire in decline, BSD had become bureaucratic and stagnant. As Linux gains market share and as BSD sinks deeper into the mire of decay, their parting addresses will resound as fitting eulogies to BSD's demise.
It is perhaps anybody's guess as to which *BSD is the worst off of an admittedly suffering *BSD community. The numbers continue to decline for *BSD but FreeBSD may be hurting the most. Look at the numbers. The erosion of user base for FreeBSD continues in a head spinning downward spiral.
Consider that because of the many troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to yet another charnel house.
All major marketing surveys show that *BSD has steadily declined in market share. *BSD is very sick and its long term survival prospects are infinitesimally dim. If *BSD is to survive at all it will be among hobbyist dilettante dabblers. In truth, for all practical purposes *BSD is already dead. It is a dead man walking.
Consider that because of the many troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to yet another charnel house.
All major marketing surveys show that *BSD has steadily declined in market share. *BSD is very sick and its long term survival prospects are infinitesimally dim. If *BSD is to survive at all it will be among hobbyist dilettante dabblers. In truth, for all practical purposes *BSD is already dead. It is a dead man walking.
Loose still holds second place to viola and walla being used in place of "voila". Though I have given up all hope as I've recently seen "anyways" and "that's a whole nother thing" used in would-be serious posts.
The numbers continue to decline for *BSD but FreeBSD may be hurting the most. Look at the numbers. The erosion of user base for FreeBSD continues in a dizzying, head spinning downward spiral.
Consider that because of the many troubles of Walnut Creek, abysmal sales and so on, FreeBSD went out of business and was taken over by BSDI who sell another troubled OS. Now BSDI is also dead, its corpse turned over to yet another charnel house.
All major marketing surveys show that *BSD has steadily declined in market share. *BSD is very sick and its long term survival prospects are infinitesimally dim. If *BSD is to survive at all it will be among hobbyist dilettante dabblers. In truth, for all practical purposes *BSD is already dead. It is a dead man walking.
Once you get past the fact that *BSD is fragmented between a myriad of incompatible kernels, there is the historical record of failure and of failed operating systems. *BSD experienced moderate success about 15 years ago in academic circles. Since then it has been in steady decline. We all know *BSD keeps losing market share but why? Is it the problematic personalities of many of the key players? Or is it larger than their troubled personae?
The record is clear on one thing: no operating system has ever come back from the grave. Efforts to resuscitate *BSD are one step away from spiritualists wishing to communicate with the dead. As the situation grows more desperate for the adherents of this doomed OS, the sorrow takes hold. An unremitting gloom hangs like a death shroud over a once hopeful *BSD community. That hope is long gone, replaced by an inconsolable despair. A mournful, plaintive nostalgia has settled in. Now is the end time for *BSD.
The record is clear on one thing: no operating system has ever come back from the grave. Efforts to resuscitate *BSD are one step away from spiritualists wishing to communicate with the dead. As the situation grows more desperate for the adherents of this doomed OS, the sorrow takes hold. An unremitting gloom hangs like a death shroud over a once hopeful *BSD community. That hope is long gone, replaced by an inconsolable despair. A mournful, plaintive nostalgia has settled in. Now is the end time for *BSD.
Everyone knows about BSD's failure and imminent demise. As we pore over the history of BSD, we'll uncover a story of fatal mistakes, poor priorities, and personal rivalry, and we'll learn what mistakes to avoid so as to save Linux from a similarly grisly fate.
Let's not be overly morbid and give BSD credit for its early successes. In the 1970s, Ken Thompson and Bill Joy both made significant contributions to the computing world on the BSD platform. In the 80s, DARPA saw BSD as the premiere open platform, and, after initial successes with the 4.1BSD product, gave the BSD company a 2 year contract.
These early triumphs would soon be forgotten in a series of internal conflicts that would mar BSD's progress. In 1992, AT&T filed suit against Berkeley Software, claiming that proprietary code agreements had been haphazardly violated. In the same year, BSD filed countersuit, reciprocating bad intentions and fueling internal rivalry. While AT&T and Berkeley Software lawyers battled in court, lead developers of various BSD distributions quarreled on Usenet. In 1995, Theo de Raadt, one of the founders of the NetBSD project, formed his own rival distribution, OpenBSD, as the result of a quarrel that he documents on his website. Mr. de Raadt's stubborn arrogance was later seen in his clash with Darren Reed, which resulted in the expulsion of IPF from the OpenBSD distribution.
As personal rivalries took precedence over a quality product, BSD's codebase became worse and worse. As we all know, incompatibilities between each BSD distribution make code sharing an arduous task. Research conducted at MIT found BSD's filesystem implementation to be "very poorly performing." Even BSD's acclaimed TCP/IP stack has lagged behind, according to this study.
Problems with BSD's codebase were compounded by fundamental flaws in the BSD design approach. As argued by Eric Raymond in his watershed essay, The Cathedral and the Bazaar, rapid, decentralized development models are inherently superior to slow, centralized ones in software development. BSD developers never heeded Mr. Raymond's lesson and insisted that centralized models lead to 'cleaner code.' Don't believe their hype - BSD's development model has significantly impaired its progress. Any achievements that BSD managed to make were nullified by the BSD license, which allows corporations and coders alike to reap profits without reciprocating the goodwill of open-source. Fortunately, Linux is not prone to this exploitation, as it is licensed under the GPL.
The failure of BSD culminated in the resignation of Jordan Hubbard and Michael Smith from the FreeBSD core team. They both believed that FreeBSD had long lost its earlier vitality. Like an empire in decline, BSD had become bureaucratic and stagnant. As Linux gains market share and as BSD sinks deeper into the mire of decay, their parting addresses will resound as fitting eulogies to BSD's demise.
"You see, even though I have never contributed code to any BSD project, I thought it was my duty to be a big asshole to others which don't use the OS I do, because it just 0wnz.", said one FreeBSD user. "Now that I know it sux0rs, though, I have to go find something else to be an asshole about."
One notorious OpenBSD fanatic known as WideOpen, told reporters, "I have to kill myself. This isn't how it was supposed to happen. My BSD has always been the best, and shouting that opinion in other people's faces at every chance I got has been my only hobby. It was all I ever did. It was what got me out of bed in the morning. Now I have to die. I will jam my bedpost up my ass until I hit my brain. It is the only way to go: BSD style."
In the volatile world of operating systems anything can happen. "At least we don't sux0r as much as Windows users", BigAzz, a relatively well-known NetBSD user said. "Screaming things in people's faces is my calling. Now I need to scream that BSD sux0rs. What a sad world. At least I won't kill myself like those uber-asshole OpenBSD guys. They are just way over the top. Or were, at least."
Nobody knows for sure what the future holds for the state of operating systems, but with Netcraft confirming the sux0r status, *BSD users all over the world will have to stick something else up their asses from now on or risk looking even more gay than they used to.
Thisis an excellent family-oriented animal adventure film with plenty of human interaction and comedy. Marshall Thompson is Doctor "Daktari" Marsh Tracy, head of an animal study compound in Africa. Widowed, he takes care of his teenage daughter Paula played by cute and perky Cheryl Miller.
Sort of an American Hayley Mills, Paula Tracy is a bit of tom-boy (she has a python for a pet) but is growing up into a woman (she tapes her stockings to her thighs with masking tape to hold them up).
Betsy Drake (the former Mrs. Cary Grant) is Julie Harper, a sort of Jane Goodhall type character who studies apes in their natural habitat and is the romantic interest for Dr. Tracy.
The great character actor who made a career of rolling his r's, Richard Haydn ("Five Days in a Balloon"), is excellent as the comic relief Rupert Rowbotham, Paula's tutor, who is afraid of all the animals (especially Mary Lou the python and Clarence, the lion who tend to snuggle up to him).
Add some wrestling with wild cheetahs, a few dangerous gorilla poachers, the antics of Doris the chimpanzee, and of course Clarence, the cross-eyed lion and what you have is the most consistently entertaining of the African animal adventure films which include John Wayne's "Hatari" and Hugh O'Brien's "Africa--Texas Style!. Ivan Tors ("Flipper," "Sea Hunt," "Gentle Ben," "Zebra in the Kitchen") produced this film which later became the TV series "Daktari."
The record is clear on one thing: no operating system has ever come back from the grave. Efforts to resuscitate FreeBSD are one step away from spiritualists wishing to communicate with the dead. As the situation grows more desperate for the adherents of this doomed OS, the sorrow takes hold. An unremitting gloom hangs like a death shroud over a once hopeful FreeBSD community. The hope is gone; a mournful nostalgia has settled in. Now is the end time for FreeBSD.