FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks

An anonymous reader writes "Phoronix has brought benchmarks comparing the FreeBSD 8.0-RC and Ubuntu 9.10 Alpha 6 operating systems. FreeBSD rather ends up taking a wallop to Ubuntu Linux, but there are a few areas where FreeBSD 8 ran well. They also posted benchmarks comparing this near-final FreeBSD 8.0 build to that of FreeBSD 7.2 to show performance improvements there but with a few regressions."

Comments

Re:For those that want to skim TFA for the bad res

[amorsen]

It's fairly difficult to bungle traditional CPU-heavy loads. The kernel just needs to get out of the way and let userspace do whatever it wants to do. Try the same tests on Mac OS X and Windows (assuming they compile), and you'll see just about the same performance.

Re:Benchmarks...

[nxtw]

Does Ubuntu provide a stable platform to build a server? No. It, like most linux distros, changes whole versions during updates. That isn't stable.

Ubuntu has LTS (long-term support) releases which are supported for 5 years on the server side. The last was 8.04 and the next will be 10.04.

I prefer RHEL/CentOS, however. I wonder how many people use Ubuntu LTS instead of using RHEL or SLES instead.

Does Ubuntu provide a way to strip itself down to the bare metal? Ain't as easy as the BSD's.

How often is this important? I can think only of a few situations, such as when fitting a system into a small/cheap flash.

But really, Comparing FreeBSD to Ubuntu is like comparing OpenSolaris to Windows 7. FreeBSD is largely a server operating system were as Ubuntu is an end user operating system. And if you are comparing server operating systems, there are far more important criteria than "speed". Things like version stability are vastly more important.

Ubuntu has a separate 'server' version (which really just includes a different set of packages and a different kernel build.)

FreeBSD is still in debugging mode

[Just Some Guy]

From the update notes in /usr/src/UPDATING:

NOTE TO PEOPLE WHO THINK THAT FreeBSD 8.x IS SLOW:
FreeBSD 8.x has many debugging features turned on, in both the kernel and userland. These features attempt to detect incorrect use of system primitives, and encourage loud failure through extra sanity checking and fail stop semantics. They also substantially impact system performance. If you want to do performance measurement, benchmarking, and optimization, you'll want to turn them off. This includes various WITNESS- related kernel options, INVARIANTS, malloc debugging flags in userland, and various verbose features in the kernel. Many developers choose to disable these features on build machines to maximize performance. (To disable malloc debugging, run ln -s aj /etc/malloc.conf.)

Since the article didn't mention anything about disabling all the debugging options, I'll consider this an invalid benchmark until shown otherwise.

Re:FreeBSD is still in debugging mode

[Anonymous Coward]

WITNESS and INVARIANTS were disabled on Sep 10 (http://svn.freebsd.org/viewvc/base?view=revision&revision=197065). FreebSD 8.0-RC1 was built on Sep 17th.

Phoronix? Moronix more like.

[LizardKing]

Yet again a benchmark against a pre-release version of FreeBSD where the testers didn't even bother reading the documentation. Anyone actually familiar with the FreeBSD development and release process would know that a release candidate has a considerable amount of debugging options turned on. This is to help diagnose any problems as the last issues are shaken out of a release, but has an adverse impact on performance.

Re:Phoronix? Moronix more like.

[Conley Index]

Anyone actually familiar with the FreeBSD development and release process would know that a release candidate has a considerable amount of debugging options turned on.

On Sep-10, most debugging was disabled: http://lists.freebsd.org/pipermail/svn-src-all/2009-September/013399.html

On Sep-17, there was the last commit before 8.0-RC1: http://lists.freebsd.org/pipermail/svn-src-all/2009-September/013645.html

Anyone familiar with the FreeBSD development and release process would know that there are no fixed rules rules when certain stuff happens and there are no sweeping changes like turning off debugging between a late RC and the actual release. (Other debugging stuff like kernel and module symbols are kept for the release.)

terrible review methods

[buddyglass]

I stopped reading when I realized they didn't even use the same version of GCC in their compilation comparison.

Re:What's the point.

[geminidomino]

Is there any actual benefit to be gained from removing "cruft", other than saving a smidgen of memory?

Long, but not long enough answer:

Performance: Unless the cruft is a bunch of data or NOPs, it will be executed at some point, which is pointless (or it wouldn't be cruft.) And whether it's data or instruction, if "good" data is swapped out of the cache in favor of the cruft, then it will have to be read back in (cache misses).

Security: Bugs love to hide in cruft.

tl;dr version: Yes.

Re:Summary

[TheRaven64]

FreeBSD ships with GCC 4.2.1 as the system compiler because it was the last release to be GPLv2 and GPLv3 stuff is not allowed in the base system. GCC 4.4 is in ports, and you can use this to compile ports easily by just setting a flag in make.conf if you care. FreeBSD 9 will hopefully be using LLVM/Clang as the system compiler, which should give it a nice boost.

Phoronix has a history of doing long and misleading benchmarks between Linux and *BSD/Solaris, where they manage to include so many extraneous factors that the results are meaningless.

Re:Benchmarks...

[Abreu]

Ubuntu Server Edition does not install X by default

Once again, benchmarks fail

[bonch]

How about we see this against a version of FreeBSD that doesn't have debug on according to /usr/src/UPDATING?

NOTE TO PEOPLE WHO THINK THAT FreeBSD 8.x IS SLOW:
FreeBSD 8.x has many debugging features turned on, in both the kernel and userland. These features attempt to detect incorrect use of system primitives, and encourage loud failure through extra sanity checking and fail stop semantics. They also substantially impact system performance. If you want to do performance measurement, benchmarking, and optimization, you'll want to turn them off. This includes various WITNESS- related kernel options, INVARIANTS, malloc debugging flags in userland, and various verbose features in the kernel. Many developers choose to disable these features on build machines to maximize performance. (To disable malloc debugging, run ln -s aj /etc/malloc.conf.)

Re:Once again, benchmarks fail

[ens0niq]

Sorry, sir

From Phoronix:

"The tests that were carried out under FreeBSD 7.2, FreeBSD 8.0 RC1, and Ubuntu 9.10 Alpha 6 included timed ImageMagick compilation"

From freebsd-current mailing list:

> Hi,
>
> I would like to ask that the FreeBSD 8.0-RC1 ISO-s free from the
> debugging features (WITNESS, malloc debugging, etc.)? Or these
> services are still being active?

They are gone, for the most part. r197065: http://svn.freebsd.org/viewvc/base?view=revision&revision=197065

Remove extra debugging support that is turned on for head but turned off for stable branches:

- shift to MALLOC_PRODUCTION
- turn off automatic crash dumps
- Remove kernel debuggers, INVARIANTS*[1], WITNESS* from
GENERIC kernel config files[2]


[1] INVARIANTS* left on for ia64 by request marcel
[2] sun4v was left as-is

Re:Sigh...

[Just Some Guy]

freebsd is 100% binary compatible with linux.

Snicker. I love FreeBSD and run it on all the servers I administer, but the Linux compatibility stuff pretty much ends at usermode. Good luck firing up VMWare or anything else that requires an un-ported kernel module! In practice, every program I personally want to run on FreeBSD is available as a native binary or in ports, with the exception of programs that require kernel mods, in which case they won't work at all anyway.