A Comparison of Solaris, Linux, and FreeBSD Kernel

v1x writes "An article at OpenSolaris examines three of the basic subsystems of the kernel and compares implementation between Solaris 10, Linux 2.6, and FreeBSD 5.3. From the article: 'Solaris, FreeBSD, and Linux are obviously benefiting from each other. With Solaris going open source, I expect this to continue at a faster rate. My impression is that change is most rapid in Linux. The benefits of this are that new technology has a quick incorporation into the system.'"

Comments

wishfull thinking

[scenestar]

If only they could compare the NT kernel along with them

*sigh*

Re:wishfull thinking

[wlan0]

Can one really see how the NT kernel works, with all the stuff stuck together like Windows is?

When will OSI licenses really start working?

[Nuclear Elephant]

Solaris 10, Linux 2.6, and FreeBSD 5.3... all have strengths, weaknesses, features, and deficiencies... so why hasn't the OSI succeeded in the cross-pollination of these three great OS's? If they're really going to benefit from each other, why not get some linux kernels with SMF or better SMP out there? When will apt finally replace /usr/ports in FreeBSD? And when will Soalris' TCP stack not suck by implementing code from Linux or BSD? I hug all three of these OS's on a daily basis, but if open source is really working why can't we seem to make an OS out of these three that flat out rocks?

Re:When will OSI licenses really start working?

[WindBourne]

Off hand, I would say that all three flavors rock. And they also currently borrow from each other.

BTW, you mention Solaris's network stack. For Solaris 9.9.x, just before the release, Sun did an internal test comparing between Solaris and all the major OSs. It turned out that Solaris lost big to Linux 2.6 when it came to networking. So Sun delayed it so that the internal team could re-design it to beat Linux's networking. According to one of my friends there, they believe that they have done so. But he also said that they borrowed ideas from Linux and BSD. So yes, the x-pollination is occuring.

Re:When will OSI licenses really start working?

[MBCook]

I'd have to agree. As each OS adds something (like randomizing the starting number (can't remember right name now) for TCP a few years ago in one of the BSDs), the others look at it and add it. It can take some time because the code can't be directly lifted due to the differences that exist at the kernel level (unlike user-space where a port between Linux and FreeBSD may require VERY little work). I remember talk about Linux's TCP/IP implementation not being up to snuff with some of the BSD stacks. They are quire competitive now, I think. I remember comparisons about how slow threads were to start in Linux compared to other OSes (although Windows is even worse, I think). But that lead to (or at least put a fire under) the NPTL project (and the others doing the same thing). I image that FreeBSD worked on their version of the Big Kernel Lock and SMP support because of Linux having better and better SMP support in the last few years, and working to remove Linux's Big Kernel Lock (still remains to some degrees).

The system has been working very good. Plus there are obvious connections. FreeBSD (and I assume Solaris) can both read ext2 (and I assume 3). Both have DevFS (which Linux has had, at least in some form, I don't know how close/far apart they were). So code which can be easily adapted does get moved. I would be VERY surprised if there were only a handful of drivers for FreeBSD that said something along the lines of "Based on the Linux driver by Mr. Reverse Engineer", and I'd imagine there are drivers that go the other way too (I'm not nearly as familiar with FreeBSD as I am with Linux).

Re:When will OSI licenses really start working?

[jhines]

A port of Dtrace from Solaris to FreeBSD has been announced, and pf from OpenBSD has been ported to the others.

It is happening. Solaris is the new kid on the block, it will take time for the code to be grokked and made use of. Or vice versa.

Re:When will OSI licenses really start working?

[discogravy]

When will apt finally replace /usr/ports in FreeBSD?

Is this something that's like, wildly wanted? Most FreeBSD users seem to like ports...and if you're in a hurry or using old hardware, there's always pkg_add -r [packagename]. I do love apt, but why bother?

Re:When will OSI licenses really start working?

[Arkus]

I seriously doubt you have to worry about apt replacing ports ever. As slick as apt is to use on the x86 platform, I hear from my friends lucky enough to run 64-bit x86 Linux systems that there are all sorts of problems when trying to get apt working properly in that environment.
For instance when browsing the web, some of the plugins out there are only available in the 32-bit x86 flavor, not 64-bit. Fedora can use yum as the dependency checker and even my friend the die hard apt user has switched his systems to using yum for updates. Perhaps yum will become the new open source standard.

Re:When will OSI licenses really start working?

[adrianmonk]

FreeBSD (and I assume Solaris) can both read ext2 (and I assume 3). Both have DevFS (which Linux has had, at least in some form, I don't know how close/far apart they were).

I'm fairly sure Solaris was the first to have an automatically-managed /dev. Solaris has had its /dev and /devices arrangement (in which everything in /dev is a symlink to something in /devices, there is no such thing as MAKEDEV anymore, and everything is automatically maintained) since at least Solaris 2.4, and quite possibly since the original Solaris 2.0. And Solaris 2.4 came out in 1994, and Solaris 2.0 came out in 1992, so it seems that Solaris has had an automatically-managed /dev much longer than Linux has.

In fact, it would seems that Solaris has had an automatically-managed /dev since before Linux even hit version 1.0 (in 1994). Meanwhile, Linux is still dealing with issues like its devfs being declared obsolete before udev (the replacement) was even out of beta. Don't get me wrong. I think the Linux kernel has a lot to offer. But, I think in this one particular area, Linux is literally 10 years behind Solaris.

(By the way, if someone wants to offer corrections, feel free to go ahead. One of the things that makes me really uneasy about Linux is the way that /dev management seems, from my limited experience, to be an afterthought. If If I were to find out that this perception of mine is only due to ignorance, I would probably feel like the world is a better place and even sleep a tiny bit better at night knowing that there are few potential headaches out there waiting to be dealt with.)

Re:When will OSI licenses really start working?

[csirac]

You're right about the sub-par /dev managment in Linux. When I first switched to udev from the deprecated devfs (which I never had any trouble with), I couldn't boot properly from my software raid1 volumes because udev needed the root volume mounted to create the mdN device nodes, but the root volume couldn't be mounted until mdadm was pointed to an mdN device node to assemble...

That was about 2-3 years ago. It all seems to work smoothly these days, I think they just patched some work-arounds (guess the major/minor numbers) in the mdadm tool itself. Considering I can do a plain Debian install, arranging things so that / (incl. /boot) lives on LVM living on software raid, without hand-tuning a thing is a good sign that things have settled down. Additionally, I can modprobe any driver and have the dev node magically show up in /dev automagically. rmmod removes the driver and also the dev node. So I think things have matured in the /dev side of things over the last few years, even if it isn't as elegant as the way Solaris does it (which I'm not familiar with).

If the /dev management is your only issue, then that says a lot for Linux I think ;-)

Filesystems

[Bogtha]

Does anybody know why ReiserFS 3 hasn't been ported to any of the BSDs yet? ReiserFS 4 looks as though it's pretty revolutionary, if distributions settle on that as a default, I can see that giving quite an advantage to Linux compared with the other kernels.

I noticed that the article didn't mention LUFS. This alone allows for tremenduous possibilities, not least of which is rapid development of filesystems. Do any other systems (besides GNU HURD) have userspace filesystems?

Re:Filesystems

[ajs]

"It has to be tested for compatibility, quality and performance. You can't be losing data, and if it doesn't offer a performance benefit over UFS or UFS2 then there's very little point in porting it."

Clearly you don't know about Reiser (no offense, it's just that that question shows a stark lack of understanding with regards to why Reiser is interesting in the first place).

Reiser solves one of the oldest problems facing the old Unix-style filesystem: the adoption of btree-order performance directory lookups (using Reiser's "dancing trees") without significant loss in other areas of filesystem performance, e.g. directory entry creation and deletion, etc. This is something which was long thought impossible.

This lead to further development, since the major reason to avoid creating thousands of temporary files has always been directory lookup times. So, now the question is: how far do you go with files? Reiser 4 answers that question by adding significant semantics to files which were not practical with slower filesystems (again, keep in mind that when I say "slower" I refer to the performance bottleneck surrounding large directories primarily).

The problem with Reiser is that it is Reiser, and none of the exisiting filesystems can match its performance in these areas. That means that if you write an application that relies on Reiser's performance, it really RELIES on Reiser, and cannot perform well under normal filesystems without significant engineering (e.g. writing special-case code for Reiser and non-Reiser filesystems). In some cases (e.g. databases) this might be worthwhile, but in the case of more mundane applications, having filesystem-specific code is not always viable.

For more information, see the Reiser4 site: http://www.namesys.com/v4/v4.html

Re:Filesystems

[SnowZero]

The problem with Reiser is that it is Reiser,

I'd say the real problem is often Hans Reiser. He's usually got good ideas, but he tends to quickly get on people's bad side with his argumentative style. What's strange is that this is usually limited to the first 10-20 posts in some big discussion, then he calms down and works more directly and constructively with developers. If you look at the past few Reiser discussions on lkml, they seem to follow this pattern.

Linux kernel better than Solaris kernel.

[Anonymous Coward]

Interesting comparison between the Linux and Solaris kernels from someone who used to work SunSoft in the kernel group.

http://www.ultralinux.org/faq.html#q_1_15

Hyperthreading

[MBCook]

This was an interesting little article, I read it earlier today when I saw a link from OSNews. But one thing struck me as odd (there were a few others, but this was the one I was sure about).

For hyperthreaded CPUs, FreeBSD has a mechanism to help keep threads on the same CPU node (though possibly a different hyperthread). Solaris has a similar mechanism, but it is under control of the user and application, and is not restricted to hyperthreads (called "processor sets" in Solaris and "processor groups" in FreeBSD).

I am positive that the 2.6 kernel understands hyperthreading and does something similar to FreeBSD. Why wasn't that mentioned? Did the author not know that?

Overall through, it was interesting. I'd read it as a longer series, if they had one. This is an area that I'm interested in. I read kernel-traffic, and subscribe to LWN (you should to!) almost entirely to read the kernel page. I've learned so much about operating systems and computers from reading about the improvements in the Linux kernel, why the old version wasn't good enough, etc. While I no longer use Linux since I got my Mac (OS X fills all my needs), I continue to learn a large amount about computer architecture and operating system concepts from it.

Serious things were missed....

[postbigbang]

1. The SELinux kernel wasn't mentioned; it's security model is different, perhaps better in the final analysis than OpenSolaris.

2. The concept of Solaris containers is nearly science fiction. Building them and then watching them through dtrace is a work of art, as in the Sistine Chapel. LVM is a different school of thought that gets to a similar conclusion; this all skewed by the beauties of VMWare and multiple instance/clustering management possibilities.

3. The licenses-- very important differences in licenses-- are glossed==> ignored. There's all that messy intermingling of licensing trivia that's somehow an invisible characteristic of all of this. Fooey.

4. While Sun can speak anytime Sun wants, at least there were other citations mentioned early. This is Sun propaganda. Remember that. Well thought out propaganda, but propaganda, not a third party examination of the facts and implications.

5. There are other *nixes missing. Consider that real-time OS and embedded OS considerations are real, and while BSD and Linux has made progress there, Solaris is essentially missing, unless you consider weird programming profiles still based on non-Solaris OS.

These are just the extemporaneous thoughts. Take this article with a grain of salt, although it's not bad for a vendor-hosted view.

Re:The Answer is Clear

[Anonymous Coward]

You are. Solaris has a microkernel. Two peice static core and everything else is dynamically loaded. IMHO a far superior model to using modprobe.

Re:The Answer is Clear

[Anonymous Coward]

Dynamically loaded kernel modules != microkernel, retard. Solaris is not a microkernel.

Re:Big lock

[DigitalNate]

This is where you should take advantage of one of FreeBSD's greatest assets, it's documentation. The SMP Project has a lot of info...

http://www.freebsd.org/smp/

Interesting Model Breakdown...

[TemporalBeing]

I find it quite interesting that (at least according to the article) Solaris (which supports a few x86, and mostly Sun's Sparc line) has a full abstraction, while Linux (which supports some large number processor architectures) goes with less abstraction; with FreeBSD somewhere in the middle. It certainly does yield higher performance for Linux, and makes sense in that respect...It's just interesting that the OS that seemingly runs on fewer processor architectures and has been controlled by an incorporated company would take the abstraction route, while the OS that runs on a far greater number of processor architectures and is not tied to corporate funding (directly, at least) is more focussed on less abstraction & fewer layers.

P.S. Sorry to repeat myself on that...just not sure how best to say it.

Re:kprobes?

[ChipMonk]

For the typical user app, profiling is the answer. A program bottleneck, such as a file being read over and over, will be revealed in profiling. A bottleneck in the kernel needs to be resolved in the kernel, not in userspace.

This is not meant to disparage Solaris dev tools. This is merely to point out that Linux has its own, very powerful developer-oriented tools.

Re:How much of Solaris has gone open source?

[adrianmonk]

Its been what, 2-3 years since the open-source solaris announcement came out? How much has been open sourced? AFAIK, all the have opened sourced is DTrace (a very cool tool/framework), but nay else.

Well, it's good that you said "AFAIK", because what YK turns out to be out of date. Browse the Solaris source code right here.

Lets see them open up the kernel internals like the thread model

OK, here's the directory with the dispatcher stuff and here's thread.c specifically.

Re:FreeBSD Ports

[molnarcs]

Wow! Is that all it takes to get a +5 Insightful now? - I could ask the same of your own post, because some parts are factually wrong.

Now this is where Ports and Portage, IMHO, really suck. When I uninstall a package from my system I want it gone. apt-get remove --purge and a properly packaged deb will do that for you. Ports and Portage will leave "package cruft" on your system.

I don't know about portage, but ports don't leave cruft behind. When you remove a port, you effectively remove a package. FreeBSD does not differentiate between ports and package deinstallation. Portinstall/upgrade tools, share the same command for package removal, whether it was built from ports or installed as a binary package. pkg_deinstall -rd fooo* will recursively deinstall every fooo* package and those packages that depend on it. Not only that, but it will remove the directory as well - unless you modified some files by hand, in which case if will tell you which files it couldn't remove (because they didn't match original checksum). Ports leaves "cruft behind" is a myth - or sounds to me like superstition or something. ALL installed files by a port or package are recorded in the plist, and all those files will be removed on deinstall - unless you changed them manually, in which case you will notice (and probably don't want to delete them anyway before making a backup).

Ports (and Portage) does Rock, but only if all you ever care about is package installation and not package management (package building, installation and uninstallation).

WTF? The reason I use ports is because I care about package management. I have to take care of a small lab with old computers. 3 of them run freebsd+blackbox+opera/gaim/irc combo (and a simplified menu so all can use it). What makes the whole setting really nice is that I can compile every installed package (there aren't that many - ~ 110) optimized for that hardware on my home puter. Than I can point portupgrade -PP to my ftp repo, and don't have to worry about dependencies and whatnot: everything will be upgraded/installed in the right order (and removed if I wish to). Yes, this is possible with deb and rpm as well, it is just not that easy. Forgot putting a required package in the repo? You can create binary packages with pkg_create -b pkgname from a package installed on your system. It doesn't matter if it was installed from source using ports or from a package. Package management knows of every file belonging to that package and the proper paths, and it will create a binary package for you, which will similar to any .deb: it will register all the required dependencies as well (unlike slack .tgz). Again, how could you do that if package management would not accurately keep track of everything? In other words - you pull this "cruft" stuff out of your ass (don't know about gentoo though) - ports doesn't leave any cruft behind (no more than apt does).

Re:The Answer is Clear

[@madeus]

Ok, so as a USER, why would you care about MySQL? Because as a SYSTEM ADMINISTRATOR, what I really care about is stability and easy of administration. Once performance reaches "good enough", I could give a shit about improved performance. Hardware is cheap, a $5,000 1U MP system can blast down just about anything I'm going to care about.

It's fair to say then, you obviously have very modest requirements.

Unless your technology depends on Windows (I feel for you if it does) any of the *nixes out there are "good enough" to do just about anything up to the very high end. (Linux/BSD/Solaris/AIX/OSX/etc)

Wrong (sadly).

Even at the very high end, it's unlikely the choosing the "worst" OS will cost more than switching to the "best" OS!

Also wrong, there is a huge difference, though the gap varies depending on the role.

For example, Mac OS X - which I love dearly - is the slowest operating system this side of 'Slowaris' (which has that nickname for a reason), and is many times slower than Linux (on the same hardware). It's embarrassingly slow at some things. If it was a person, it would be part of a 'Care in the Community' program. It's as close to being 'autistic' as an operating system can be.

However, it has some really great features, including the quartz UI and really good multitasking for user software, which are what make the poor performance otherwise very bearable.

We have lots of FreeBSD systems here, while there are no issues with lower end systems (where there are multiple systems in a load balanced group not doing very much) they don't live up to our needs under stress, you start finding bugs with software under high load (often these are not always the fault of the OS - but exist none the less). You can also run up against kernel limitations (and find FreeBSD doesn't like really large file shares, or it buckles under certain types of load, or it behaves in a way that is not agreeable with some existing software you need to use).

This is especially with multiple CPU dual HT systems IME, even with just regular dual CPU HP DL360/DL380 systems. You can certainly coax better performance out of the FreeBSD systems but for a whole host of tasks (mail, web, dns, proxies, databases, etc) even after significant time doing manual optimisation of your software and multiple kernel recompiles (to determine optimum settings) Linux wins hands down every time IME, not least because more developer hours are spent optimising and testing those applications (and libraries) on Linux.

Re:FreeBSD Ports

[Just Some Guy]

When I uninstall a package from my system I want it gone. apt-get remove --purge and a properly packaged deb will do that for you.

So [package uninstaller] on [OS] completely removes that package. Golf clap. A properly packaged (your words) application will work that way on any OS, not just Debian.

I love Debian, and have used it for years. It's great. However, the real admin nightmare comes when you decide you want some non-standard feature supported systemwide. On FreeBSD, for example, if I want LDAP support then I install the OpenLDAP client port (or let it get brought in as a dependency when I request LDAP support in some random program). Et voila, ports compiled from then on get LDAP support where appropriate. Gentoo fans: yeah, I know you have this too. Portage earns its name, I'll give you that.

Debian, on the other hand, requires you to hand-roll special locally-built versions of every application you want to have LDAP support - when a new version comes out, it's hand building time again. Yay! If LDAP is too common to be a good example, check out Kerberos. Debian/unstable has OpenSSH 4.2p1. Nice! If you want a version with Kerberos support, though, plan on rolling back to OpenSSH 3.8.1p1 and losing all of the other features you might have liked to use, or cranking up gcc and your beloved packaging tools.

As I said, I like Debian. Still, I can't pretend that it doesn't have several critical issues that make its administration a lot harder than need be. If the official packages cover all your needs, then it's a dream. If not, things get ugly quickly. You might not like the tradeoffs that FreeBSD and Gentoo have made, but they're there for a reason and are exactly what many of us want and need.