Slashdot Mirror
Benchmarks For Ubuntu vs. OpenSolaris vs. FreeBSD
Ashmash writes "After their Mac OS X versus Ubuntu benchmarks earlier this month, Phoronix.com has now carried out a performance comparison between Ubuntu 8.10, OpenSolaris 2008.11 and FreeBSD 7.1. They used a dual quad-core workstation with the Phoronix Test Suite to run primarily Java, disk, and computational benchmarks. The 64-bit build of Ubuntu 8.10 was the fastest overall, but FreeBSD and OpenSolaris were first in other areas."
Always does
Various versions of GCC. While one could argue that the compiler is part of the OS it's indeed replaceable so I would had prefered if they had used the same version of GCC and not different for each OS.
It would had been very interesting to see the Solaris results using Sun Studios CC as well (I think it's also available for Linux nowadays?)
.. but what it's missing is the ability to easily uninstall it. It's not the only distro not to be easily uninstallable, but it seems daft that you have to start messing about restoring the boot record via Windows boot disk if you want to take it off.
Test all three distros out of the box, to see which one is better... Even with all that extra 'default' weight Ubuntu still shines on except when running, eh, Java.
Very informative!
I was a bit disappointed by the results, being a FreeBSD fan myself. However, in my quick scan of the article, I didn't see any mention of how they configured the OS. If they truly used the stock 7.1-BETA2 install, that would mean that debugging mode is enabled in the kernel (and maybe the userland, I'm not 100% sure here). Unless I've misunderstood FreeBSD's release methods over the years, they don't disable the debugging until either the RC builds or maybe even the final release tag.
Still, FreeBSD came out on top on 3 of the tests -- not bad for a beta release. I can't wait for 7.1, as using 7.0 on my desktop since its release has been great. I just hope the fully-virtualized IP stack within jails made it into 7.1, as well as a slightly more stable ZFS.
Method of processing duck feet
For desktop use, I find this to be an accurate test (as many desktop users will likely use the default configuration). However, I feel this to be a generalized test only suited for the purpose of picking a Desktop OS.
For Servers and anybody that has intimate knowledge of the capabilities of each OS, it should be duly noted that Linux and FreeBSD could have performed better. Linux could have been configured to use the EXT-4 file-system, which may or may-not have performed better (or at least on-par) with Solaris' ZFS. Similarly, I would have liked to see the results if FreeBSD-7 was configured to use ZFS, so we could have seen an Apples-to-Apples comparison between Solaris and FreeBSD. I also wonder if FreeBSD's implementation of ZFS is faster than Solaris'.
The reason that I never really seriously used Linux on my PC laptop was that Ubuntu was sluggish, even with the newest ATI drivers, compared to Windows. Maybe people have good experience with nVidia drivers there, but Windows is a lot more usable as a desktop for me on the performance side of things. Granted, my main computer is a MacBook Pro running Leopard, but I can't imagine putting Linux back on my old PC laptop for when I need to use it.
Could be nice to do those comparisions in the same hardware betweeen i.e. Ubuntu, Gentoo and OpenSUSE, all for 64 bits, as is not clear when they are measuring against Linux or against optimizations or not that do a particular distribution. Or put where it applies (i.e. the java tests) the numbers for Windows and MacOS.
Interesting results, and great if you're planning a server, but what about desktop use?
How well does each OS do when doing something like playing back audio/video, and handling background processing loads? What about performance and system response as the load climbs up? (load averages of 5/10/20 ?).
Only because I've seen Linux systems start to crumble around 5 (uniproc machine), and easily get unusuable, but have heard reports of BSD machines being able to still play MP3s without skipping/suttering even around 20 or so...
(And yes, I'll allow tweaking system priorities - it only gets you so far, and impacts the other background processing tasks, to which we'll also be interested in how long they take to run. So renicing the media player to -20 works, but not if it makes all the other tasks take 10x as long to finish...).
According to those benchmarks FreeBSD with ZFS would be the best bet?
Sorry, I got a bit carried away there. Eh, what was the article about again?
Genesis 1:32 And God typed
This couldn't come at a better time. I was recently wondering if FreeBSD was a good platform for deploying our first Java EE application (since we use fbsd for everything else) or that Linux or Solaris might be better. It's good to see that FreeBSD isn't all that bad, but I know now that switching to (Open)Solaris might be worth it. But as far as I see, OpenSolaris is mainly geared towards desktop use, isn't it?
I have not played with Open Solaris but with normal Solaris you need to set parameters in the /etc/system file to get good performance. By default Solaris is set very conservative. In many tests I have run Solaris may not be the fastest with single test but under a heavy load with many applications running my experience has been it can handle a much bigger load then Linux on the same hardware. I use both but for backend heavy loaded servers I would choose Solaris.
Does anyone know if the Phoronix Test Suite will work under OpenBSD and NetBSD too? Says on the website: "Runs On Linux, OpenSolaris, Mac OS X, & FreeBSD Operating Systems"
I'm curious as to how the other BSDs would perform.
It seems a bit lop sided and ridiculous to compare Ubuntu 8.10
a released distro against 2 BETA distributions.
Ubuntu release runs faster than 2 other unixes that are still in debug and haven't been released yet.
Who'd have thunk it?
I am curious to how it actually performs and not just what most the slashdotters say.
It may actually suck and I am curious as to how much.
I am contemplating leaving vista where I do php, apache, and java development. I wonder if there is an advantage at all.
http://saveie6.com/
Except for Bonnie++, all of their benchmarks are compute-bound. In other words, they're benchmarking the bundled compiler, not the distribution.
The one exception is Bonnie++, on page 6, which measures raw filesystem performance... and is something that is known to greatly depend on how old and how full a given filesystem is.
What's the default scheduler in the 7.1 beta? Most sites say that ULE will be default in the 7.1 release, does this mean beta as well?
Why do they even bother comparing ubuntu with freebsd when freebsd is obviously much faster then ubuntu
I agree. But the test is a bit of a WTF.
They are using 64-bit kernels when they only have 4 GB of RAM. It would have been far more interesting to use 32-bit kernels instead.
One loses performance when one goes to 64-bits, not gains it. One should only use a 64-bit kernel when one has more than 4 GB of RAM, and actually USES it.
Otherwise, just stick to 32-bits.
If they had done that comparison, my suspicion is that FreeBSD would've done even better.
All three come with tunable performance parameters. All three can have their performance boosted even further by recompiling everything optimized for the particular hardware being used, possibly using specialized compilers (e.g. from Sun or Intel). But that's not the point, IMO. This isn't (or shouldn't be) a pissing match--this should be an opportunity to improve all three systems by seeing where their strengths and weaknesses are, and working to bolster their weaknesses and improve their strengths.
In my experience, these sorts of tests on free/libre/open-source systems quickly become out-of-date because the developers take them as a challenge, and that's a good thing for everyone! :)
Ff your tests were more than a couple of years ago, they're probably so out-of-date as to be utterly meaningless, but that's a separate issue. Personally, I'm a big fan of all three systems and want to see all three thrive and grow and improve. This kind of testing can only help with that, once you get past all the dick-waving by narrow-minded advocates.
That someone would actually be modded down because they happen to think that Linux isn't all that as a desktop. That sort of thing is why I think that any moderation system that allows users to mod down rather than only up is broken.
I like BSD better than Lin-SUCKS, but OS X is still far and away the best platform for computing. It is faster, more stable, more secure and better designed than anything else available. PERIOD.
That sort of thing is why I think that any moderation system that allows users to mod down rather than only up is broken.
Nigger?
Nigger. Nigger Nigger!
Get me some popcorn.
See? A use for downmodding.
...to pay your $699 licensing fee you cock smoking teabaggers!
Pretty piss poor article. FreeBSD betas are built with debugging enabled, so performance is not indicative of what the final release will be like.
If you take a look at the OS kernel models and would form base predictions from the kernel architecture alone it would closely mimic the results of the tests.
There is a reason kernel architecture is a highly engineered science, and why even old models of inherent pluses or negatives would still manifest even in today's latest incarnations of these kernel architecture models.
If you look at Linux, with its microkernel heritage, it is going to offer better low level kernel multitasking and kernel messaging. Also if you look at a classic monolithic kernel design (even with Apple duct tape) the OS X kernel is going to be better at straight non-conflictive messaging and queues at the kernel level with a lighter API, but have a harder time moving past the multi-threading bottlenecks.
This is also why OS X being tied to 'specific' hardware is a plus for OS X, because if it had to deal with more diverse hardware and kernel level exceptions it would be even harder to squeeze performance out of the system and not run into multi-threading bottlenecks from the MACH kernel with the BSD API interface.
The reason I am stating (in generic terms) the obvious, is that the kernel models in use in all these OSes are very DATED concepts. In fact, every kernel architecture compared in these tests and OS X where deemed to be too primative for even the MS NT team back in 1990, which gave rise to the NT kernel which is neither a microkernel nor monolithic in nature, and why MS to this day, with even as 'bloated' as Windows is seen in the technical world, can use the NT architecture to shove around some suprising numbers not only in the consumer desktop markets, but even now in the supercomputer markets, all using the same code.
So with this in mind, it isn't a NT is great speech, but food for thought that the OSS world still needs to rethink its heritage, use Virtual Machine concepts and rally around a new set of kernel architectures that can shove NT into the ground.
Until this happens, these 'old' kernel models and architectures are going to trail NT just based on something as simple as the basic kernel theory and architecture that MS chose to use and abandon the 'in use' kernel concepts of 1990 and instead build NT around kernel technologies that were nothing but a group of theories at the time.
And there is no reason that almost 20 years later the OSS world cannot do the same and give up the primatvie kernel architectures it has been rehashing and slapping bandaids on to move forward and remain competitive.
Yes there have been some really good work on the existing technologies and bringing some 'new' ideas back to these dated concepts (even Apple has done well with putting bandaids on the monolithic nature of BSD/MACH), but why continue to work backwards and keep 'repairing or patching' technologies to move them to new hardware and instead rally behind a set of new kernel designs that are just now mainly theory.
If MS could do this 20 years ago and even choose to NOT use a *nix model as well (they owned XENIX remember), there is no reason the OSS community cannot dig deeper and start from scratch as well and VM existing OS models on the new kernel technologies.
Is not exactly what I'd say "average nix user hardware".
I bet that less than 5% of NIX users have that kind of machine on their desktops.
Why not running the test on either a "normal" desktop or even better on a laptop?
And, by the way, Opensolaris 2008.11 is still a release candidate!
Maybe Computers will never be as intelligent as Humans.
For sure they won't ever become so stupid. [VR-1988]
I installed OpenSolaris on my laptop about one week ago.
Laptop: 1.7mhz celeron, 512mb ram.
Previous OS: Xubuntu.
IMO: OpenSolaris loses by a mile.
OpenSolaris installation was easy, but very slow. It hangs at 99% for hours. This is a well know bug. OpenSolaris was noticeably slower than xubuntu. Bootup and shutdown are especially slow. The worst problem is applications, I can not find a chm reader, or mp3 player that works - on xubuntu (or any linux) this is a cinch. The applications that come with OpenSolaris are old, for example firefox 2.0. After installation, I had additional configuration to get OpenSolaris on the internet, usually linux does not require such additional configuration. I find package management in solaris to be much difficult than with any debian based, or redhat based, linux. HW detection was not a problem for me with either linux, or opensolaris.
I installed OpenSolaris for the learning experience, so I am getting that. Although it may just be a personal preference, I dislike the whole Solaris way of doing anything. Ironically, the Linux CLI, directory hierarchy, and configuration files, are a lot closer to generic UNIX than Solaris - especially Solaris 10. It might be easier for me to learn solaris, if I didn't already know some unix and linux. These days, solaris is practically it's own OS, as opposed to a version of UNIX.
All JMHO, of course.
I don't fucking get these benchmarks lately, I mean how credible is this crock of shit? You're benchmarking one STABLE system (Ubuntu) against an almost stable system (OpenSolaris) and against a Beta release (FreeBSD)... WOW that sure seems like a bad benchmark to me! In a typical benchmark, FreeBSD would've simply owned Ubuntu, and OpenSolaris for that matter! So, don't pay attention to these bullshit benchmarks, because they're worth shit, in my opinion. Puh-lease, bitch.
Insanity: doing the same thing over and over again and expecting different results.
At the end of the day benchmarking default installs is fundamentally flawed. Who runs a default install in a production environment? Don't even get me started on benchmarking a Beta or RC build against a production release.
There was no clearly defined testing methodology, and I would have expected an objective test to include more test relating to network performance.
All operating systems perform differentially when tuned for specific tasks. (ignoring Windows)