Slashdot Mirror
FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks
An anonymous reader writes "Phoronix has brought benchmarks comparing the FreeBSD 8.0-RC and Ubuntu 9.10 Alpha 6 operating systems. FreeBSD rather ends up taking a wallop to Ubuntu Linux, but there are a few areas where FreeBSD 8 ran well. They also posted benchmarks comparing this near-final FreeBSD 8.0 build to that of FreeBSD 7.2 to show performance improvements there but with a few regressions."
...once I'm done compiling.
I'm sort of curious what the point is of comparing an alpha to a release candidate. Additionally it's a minor update versus a major update. Throwing in an older release makes it all the more pointless as I'm not seeing anywhere in the summary that they disabled debugging.
The article is of little use other than tell the general populace of Slashdot that FreeBSD 8.0 and Ubuntu 9.10 are right around the corner, and that we should be hyped. Also FreeBSD 8 is a little faster than FreeBSD 7.2 but a lot slower than Ubuntu Linux 9.10
I'm not surprised, however I do belong to the group that does not really care about relative performance to other OS's as performance is only one of the aspects from the vector of decisions we had to make to finally choose FreeBSD for mass-scale deployment.
Benchmarks are useless. There are way more important things to judge an operating system then "speed".
Does Ubuntu have nearly as good of documentation? No. It has that "info" nonsense.
Does Ubuntu provide a stable platform to build a server? No. It, like most linux distros, changes whole versions during updates. That isn't stable.
Does Ubuntu provide a way to strip itself down to the bare metal? Ain't as easy as the BSD's.
Is Ubuntu built around solid engineering and design, or politics? Depends--Ubuntu seems to be less afraid of the big bad FSF as other distros, but it still is steeped in an OS built for politics. FreeBSD is pretty tame and tends to focus on solid engineering rather than political maneuvering.
But really, Comparing FreeBSD to Ubuntu is like comparing OpenSolaris to Windows 7. FreeBSD is largely a server operating system were as Ubuntu is an end user operating system. And if you are comparing server operating systems, there are far more important criteria than "speed". Things like version stability are vastly more important.
When my friends ask me about Linux, I usually steer them toward Ubuntu first, as it's the most user friendly and well-supported distro out there. Canonical really puts a lot of development work into it, and it shows (in this result and many others). In the past, I usually avoided the Linux topic altogether, as there were so many confusing distros that even trying to explain the concept of Linux to non-geeks (and even many geeks) was a huge pain in the ass. So, I for one welcome our new Ubuntu overlords.
SJW: Someone who has run out of real oppression, and has to fake it.
The areas where FreeBSD gets its ass kicked by Ubuntu start on page 7...
It seems to me like FreeBSD's real problem is incredibly bad I/O compared to Linux. The majority of the CPU-heavy tests were nearly neck-in-neck.
Eight pages of bar charts, each gray-on-gray. On half of them, shorter bars mean better performance, on the other half, longer is better; the only way to know which is which is in a legend, written in a small font.
Here's a suggestion: color-code the bars! Green is good, red is bad, yellow is in the middle of the road. For bonus points, choose the saturation based on magnitude of the differences. If the numbers are close, go with grayer bars, if the differences are dramatic, use dramatic colors.
Finally, how about a line chart at the end showing all of the numbers in one place? Yeah, you'd need to convert everything to be consistent if longer or shorter is better, but that's a good idea anyway.
Nothing for 6-digit uids?
I'm not a BSD user, but I don't see BSD taking a real kicking in these benchmarks. In the majority of the benchmarks, the average user could not discern a speed difference.
Yes, because the masses of casual users struggling under the crushing weight of FreeBSD that was preinstalled on their PC will naturally flock to Ubuntu...
That's to be expected considering the defaults of ext4 vs ufs2. You can increase flush time on ufs2 and expect a similar increase. Revert to ext3 and it would be a completely different outcome. Interesting to see all the chest pounding on choice for default settings in a desktop enviro vs a traditionally server one. Would have been a been comparsion to use the upcoming PCBSD's release vs Ubuntu's, but we've seen the bias from Phoronix before.
brandelf -t FreeBSD
From the update notes in /usr/src/UPDATING:
Since the article didn't mention anything about disabling all the debugging options, I'll consider this an invalid benchmark until shown otherwise.
Dewey, what part of this looks like authorities should be involved?
Yet again a benchmark against a pre-release version of FreeBSD where the testers didn't even bother reading the documentation. Anyone actually familiar with the FreeBSD development and release process would know that a release candidate has a considerable amount of debugging options turned on. This is to help diagnose any problems as the last issues are shaken out of a release, but has an adverse impact on performance.
I stopped reading when I realized they didn't even use the same version of GCC in their compilation comparison.
You can only do an equally good job with MacOS.
For this grant priveledge you get a meagre selection of hardware that will
either gouge you or leave you wanting. You also get an alien environment
with a number of annoying quirks and inferior package management to any
Unix. Package managment is a "Unix thing" and not just a Linux thing. This
is one area where MacOS demonstrates it's not really Unix.
Some of the proprietary tools you get with MacOS might be moderately more
useful but they will have quirks of their own, suffer from NIH syndrome
and may also suffer from addressing problems in a superficial manner.
BSD is at least a proper Unix.
A Pirate and a Puritan look the same on a balance sheet.
Actually, you'll probably do quite a bit of compiling in FreeBSD too. While you can pull binary packages, most people I know prefer to use a package manager like portsnap and compile source packages.
If you have a script to take care of compiling the common stuff, then just "make config" everything that need user input and let it fly. Granted, most servers I've used BSD on have been dual or quad core Xeon/AMD PC's with a fair bit of RAM, so it is overall a fairly quick process, though it was still not too bad on a few P4-era celerons.
The nice thing about FreeBSD source-compiled apps that I truly did love compared to BSD is the little tweaks you could do to avoid tons of crap dependencies. Debian used to be fairly "clean" as far as deps, but both Ubuntu and Deb are now getting quite ugly in that you get rather unwanted stuff in order to get the package you want. In most FreeBSD stuff, for example, I can check the "No X Server" box and happily compile my apps without any X support whatsoever. On Debian/Ubuntu I end up trying to install some CLI system monitoring tool or CUPS, whatever, and end up with a whackload of x.org stuff because it's tied to some font which is loosely tied to the actual package I'm trying to install. On Ubuntu desktops, trying to exorcize the demons of "Evolution" without having it remove other important stuff due to deps is near impossible. Sometimes there's a separate branch for a "cleaner" install, but often enough not.
Of course, BSD ain't perfect. Linux tends to have a lot of "new" stuff that BSD is a little more "conservative" in bringing into the mainline (iSCSI support for example was a fairly recent addition compared to 'nix), but overall the package system is powerful indeed.
Having not used Gentoo (yet) but knowing others who have, it seems like it might be somewhat similar in concept. The issues I've heard with are mostly in people getting to the up-and-running stage, but - similar to BSD - avoid annoying little conflicts or unwanted cruft is a lot easier than Debian/Ubuntu's precompiled binaries. Of course, I've also heard of much frustration in both if you have find halfway through a *long* compile that you missed something you should have flagged/included.
People mentioned the self-checks and debugging features that used to be turned on in FreeBSD development branches and beta releases.
Self-checks, which are the major source of kernel slowdowns in those kernel options, are not turned on in the 8.0 release candidates.
Debugging is on, but unless you are very short of memory it should not cause a noticeable slowdown.
FreeBSD's slowness in these benchmarks can be attributed to two factors:
1) the compiler. The GPL v3 is unacceptable for FreeBSD, so newest GCCs cannot be used as the base compiler. Users can choose to install a new gcc on their own (as a port) and then even go and compile all ports and even the base system with that new compiler (some parts might not have been cleaned up to comply with new language strictness that might have come with new gccs).
2) threading, as in the userland threading library support. It is very hard to tell whether there is some performance problem in FreeBSD's thread libraries or whether applications just happen to have been optimized and tested only with Linux.
That happens a lot and you can also see Solaris with it's M:N threading fail miserably for some threading benchmarks that do dump things, such as just creating and destroying threads at a high rate.
%%
The problem of "thread benchmarks" benchmarking bogus things and/or just having been written with Linux' thread model in mind has been going on for 12 something years now. Benchmarking thread systems in a realistic manner is very difficult. In real-world applications you don't create and destroy threads at a high rate and you minimize locking. Benchmarking this is almost as hard as benchmarking programming languages.
I haven't benchmarked threading libraries, knowing that it will take time that I don't have right now. I can tell you, however, that just the I/O subsystem in FreeBSD, as in filesystems and networking, isn't any slower than Linux. Not to mention GbE and today's disks are too slow to really show an OS difference for most tests anyway.
%%
The real question of I/O subsystems will come in when ZFS+Zraid is standard in FreeBSD and BTRFS is standard in Linux. In a couple of years from now nobody will understand why we ran today with no snapshots, with the raid hole (from block layer raid systems) and without transparent compression in the subtrees where we want it.
But these filesystems are complicated and there's some real performance difference visible.
%%
An area completely overlooked in the benchmark is the VM subsystem. Namely - what happens when you overload your RAM and paging commences? Linux used to make very bad choices here (dropping readonly pages, which is a wise thing as such, at rates about 10 times higher than wise).
FreeBSD used to be the go-to OS on memory shortage thanks to John Dyson's VM work (backed by a large database company that provided support and a realistic benchmarking environment during that development).
But today? Nobody knows. I'm not aware of any benchmarks that you can download that simulate memory stress and map the tradeoffs that the OS makes.
In general, the biggest obstacle to improve Linux, FreeBSD and everybody's else's OS performance is a lack of high-quality benchmarks.
Why don't people develop more benchmarks? Because they get annoyed that today, in 2009, no realistic OS benchmark will show a single number as a result. All OS benchmarks today can only make a map of what tradeoffs the OS chose, what part of the running application suite got the short end in favor of what other part. This isn't sexy and publishers don't like it.
People like reality reduced to single numbers, but in the area os OS benchmarks (and language benchmarks) that party is over.
Myself, I found myself gasping many years ago when I benchmarked network I/O versus userland CPU load. I hammered a couple of GbE interfaces while at the same time running moderate memory-intensive CPU benchmarks (with no network access from those CPU lo