Slashdot Mirror
Tanenbaum-Torvalds Microkernel Debate Continues
twasserman writes "Andy Tanenbaum's recent article in the May 2006 issue of IEEE Computer restarted the longstanding Slashdot discussion about microkernels. He has posted a message on his website that responds to the various comments, describes numerous microkernel operating systems, including Minix3, and addresses his goal of building highly reliable, self-healing operating systems."
Since I know that this story is going to turn into flame-fest central, I'm going to try to head things off by interjecting an intelligent conversion about some issues that are on my mind at the moment.
:-)
First and foremost, does anyone have a torrent of Minix3? Tanenbaum is a bit worried about getting slashdotted. If you've got one seeded, please share.
Now with that out of the way. I don't know if anyone else has tried it yet, but Minix3 is kind of neat. It's a complete OS that implements the Microkernel concepts that he's been expounding on for years now. The upsides are that it supports POSIX standards (mostly), can run X-Windows, and is a useful development platform. Everything is very open, and still simple enough to trudge through without getting confused by the myriads of "gotchas" most OS code-bases contain. Unfortunately, it's still a long way from a usable OS.
The biggest issue is that the system is lacking proper memory management. It currently uses static data segments which have to be predefined before the program is run. If the program goes over its data segment, it will start failing on mallocs. The result is that you often have to massively increase the data segment just to handle the peak usage. Right now I have BASH running with a segment size of about 80 megs just so I can run configure scripts. That means that every instance of BASH is taking up that much memory! There's apparently a Virtual Memory system in progress to help solve this issue, so this is (thankfully) a temporary problem.
The other big issue is a lack of threading support. I'm trying to compile GNU PThreads to cover over this deficiency, but it's been a slow process. (It keeps failing on the mctx stack configuration. I wish I understood what that was so I wouldn't have to blindly try different settings.)
On the other hand, the usermode servers do work as advertised. For example, the network stack occasionally crashes under VMWare. (I'm guessing it's the same memory problems I mentioned earlier.) Simply killing and restarting dhcpd actually does get the system back up and running. It's kind of neat, even though it does take some getting used to.
All in all, I think it's a really cool project that could go places. The key thing is that it needs attention from programmers with both the desire and time to help. Tossing lame criticisms won't help the project reach that goal. So if you're looking to help out a cool operating system that's focused on stability, security, and ease of development, come check out Minix for a bit. The worst that could happen is that you'll decide that it isn't worth investing the time and energy. And who knows? With some work, Minix might turn out to be a good alternative to QNX.
Javascript + Nintendo DSi = DSiCade
Tanenbaum wrote (in TFA):The average user does not care about even more features or squeezing the last drop of performance out of the hardware, but cares a lot about having the computer work flawlessly 100% of the time and never crashing. Ask your grandma.
Interesting. My mom recently bought a computer for my grandma. Grandma doesn't have a problem with the computer crashing at all. Her secret? She never turns it on.
When did we collectively forget that everything has its place...I doubt I'll ever see anything but a monolithic kernel on my desktops. No different than any given OS having its place. Windows and Ubuntu (until something better) will live on my desktops, not on my server. Why can't we just all get along?
Why go fast when you can go anywhere? O|||||||O
He developed Minix along with tons of other research work in distributed systems, networks, and other computer science topics.
If you have a computer science degree you have probably used at least one if not more of his textbooks. He's one of the more prominent computer science researchers of the last couple decades.
It seems to me the whole issue boils down to memory isolation. If you always have to pass messages to communicate you have good isolation but costly syncronization of data/state and hence potential performance hits. And vica versa: Linux is prone to instability and security breaches from every non-iolated portion of it.
As I understand it, as a novice, the only way to communincate or syncronize data is via copies of data passed via something analogous to a socket. A Socket is a serial interface. If you think about this for a moment, you realize this could be thought of as one byte of shared memory. Thus a copy operation is in effect the iteration of this one byte over the data to share. At any one moment you can only syncronize that one byte.
But this suggests it's own solution. Why not share pages of memory in parallel between processes. This is short of full access to all of the state of another process. But it would allow locking and syncronization processes on entire system states and the rapid passing of data without copies.
Then it would seem like the isolation of mickrokernels would be fully gained without the complications that arrise in multi processing, or compartmentalization.
Or is there a bigger picture I'm missing.
Some drink at the fountain of knowledge. Others just gargle.
yeah, and in that same vein we'd all have Betamax players.
:-D!
I am NOT implying that uKernels are better, I am playing devils advocate.
Not everything that "wins" is the best... Look at Windows
I'd like to point out that Minix is already FAR FAR *FAR* ahead of Linux in the version numbering war. Minix recently moved to version 3
And Linux seems to be stuck on version 2.6
And v3.12 (I think, I'm going from memory here) will finally support the X windowing system
Oh...maybe I should have left out that last sentence...kinda kills my argument
"TVs don't have reset buttons. Stereos don't have reset buttons. Cars don't have reset buttons."
They may not be labeled "reset" but they *do* have them. And, no offense, but I like having a reset button.
Linux is very reliable for me, even on newer hardware with a bleeding edge kernel. Why should I care whether it has a microkernel or monolithic kernel? Everything I deal with is user space. If it runs GNOME, is POSIX-like, and supports some kind of automatic package management, I'll be happy as a clam.
Will hardware drivers be developed faster and more reliably with a microkernel? That seems to be the biggest hurdle in reliable OS development these days... Anyone have a good answer for that, I honestly don't know.
My bicyles
I almost died of boredom looking for them. Here's the link, for the lazy:
http://www.minix3.org/doc/screenies.html
Open Source Drum Kit, LPLC deve board - mjhdesigns.com
That's a very good point, and one that people keep forgetting. If microkernels are so great, where are they? Let's take a look at notable microkernels:
* QNX Neutrino. This is the most successful microkernel ever. It deserves all the praise it gets. Yet it is still a niche product.
* Hurd. After twenty years we're still waiting for a halfway stable release. Hurd development is almost an argument *for* monolithic kernels!
*Minix. This is still an educational kernel. A teaching tool. It remains unsuitable for "real world" use.
* Mach. People claim OSX is a microkernel since it is built on top of Mach. But that ignores the real world fact that OSX is monolithic. People have been misled by the name.
* NT. This is NOT a microkernel! You don't believe anything else Microsoft says, so why do you believe this fairy tale?
In short, QNX is the only successful real world Microkernel. Linus happens to be right on this one: microkernels add too much complexity to the software. From ten thousand feet the high level architecture looks simple and elegant, but the low level implementation is a fraught with difficulties and hidden pitfalls.
A Government Is a Body of People, Usually Notably Ungoverned
You're on to something...you are very close to the cache. Why are we "debating" this when the asnwer seems very clear once one takes a step back: They (the kernels) can exist in harmony, each in its own place. Tanenbaum makes a decent showing of examples about where and why micros are used. This isn't a "which is better" argument. This should be a "where is one better utilized than the other in situation X" debate. That flamewar I could tolerate. Bottom line is that neither will replace the other, at least in a timely enough manner that it is worth wasting time over now.
Why go fast when you can go anywhere? O|||||||O
He also likes to get into flame wars with Linus Torvalds when he gets bored.
Really? And what exactly do you base this on? According to the article, which it's clear that you did not read, Tanenbaum simply had a recent article printed in IEEE Computer and someone on Slashdot posted a link to it, which caused Linus to weigh in with his 2 cents about something that was never directed at him. It sounds more to me like Linus is obsessed with proving that macrokernels are the only way to go. Why does he even care? It's not like Minix is a threat to Linux. If he believes so strongly that microkernels are wrong, he should just let Tanenbaum and company waste their time on them instead of endlessing arguing the same points he made years ago.
Hello everybody out there using minix -
:-)
I'm doing a (free) operating system (just a hobby, won't be big and
professional like gnu) for 386(486) AT clones. This has been brewing
since april, and is starting to get ready. I'd like any feedback on
things people like/dislike in minix, as my OS resembles it somewhat
(same physical layout of the file-system (due to practical reasons)
among other things).
I've currently ported bash and gcc, and things seem to work.
This implies that I'll get something practical within a few months, and
I'd like to know what features most people would want. Any suggestions
are welcome, but I won't promise I'll implement them
http://www.ussg.iu.edu/hypermail/linux/kernel/9906 .0/0746.html
He is, of course, referring to all the research in the '80s and '90s on microkernels and IPC-based operating systems.
I have never experienced the "stalling" problem that affected a very small number of 2004 and 2005 Priuses last year. (OK, hubris correction, make that "not yet..." although my car's VIN is outside the range of VINs supposedly affected).
It was apparently due to a firmware bug.
In any case, when it happened, according to personal reports in Prius forums from owners to whom it happened, the result was loss of internal-combustion-engine power, meaning they had about of mile of electric-powered travel to get to a safe stopping location. At that point, if you reset the computer by cycling the "power" button three times, most of the warning lights would go off, and the car would be fine again. Of course many to whom this happened didn't know the three-push trick... and those to whom it did happen usually elected to drive to the nearest Toyota dealer for a "TSB" ("technical service bulletin" = firmware patch).
These days, conventional-technology cars have a lot of firmware in them, and I'll bet they have a "reset" function available, even if it's not on the dashboard and visible to the driver.
"How to Do Nothing," kids activities, back in print!
...so I can't spend a lot of time in dicussing this, but I always that the main benefit of micro-kernels is completely wasted unless you actually have utilities that can work in partially-functioning environments. What good is it to be able to continue to run a kernel even with your SCSI drive disabled, if all your software to fix the problem is on the SCSI drive?
Now in theory I could see a high-availability microkernel being a good, less expensive alternative, to a classic mainframe environment, especially if you had a well written auto-healing system built in as a default. But that would require a lot of work outside the kernel that just isn't being done right now. And until it is, micro-kernels don't have anything more to offer than monolithic kernerls.
To put it in API terms - it doesn't matter very much whether your library correctly returns an error code for every possible circumstance, when most user level code doesn't bother to check it (or just exits immediately on even addressable errors).
A simple way to put the question is this:
If you were given the choice between rebooting your machine every 3 months or so for updates/driver install or never rebooting your machine and but taking a 3-5% performance hit (I think this is what the most efficient uKernels waste on address space switches) which would you choose.
I know my answer. For embedded systems/media center type stuff I don't care about the 3-5% performance hit. I don't ever want to screw with them.
For my computer I don't care about rebooting every 3 months or so. I want that extra little bit of speed.
If you liked this thought maybe you would find my blog nice too:
Forgetting something?
*Minix. This is still an educational kernel. A teaching tool. It remains unsuitable for "real world" use.
Actually, it's a start of a full-up Microkernel operating system. This isn't your grand-pappy's Minix, it's a brand new code base under the BSD license, intended to be developed out into a complete system. It's still taking baby-steps at the moment, but it's coming along quite nicely.
* NT. This is NOT a microkernel!
NT is a hybrid. It has Microkernel facilities that are constantly being used for something different in each version. Early versions of NT were apparently full Microkernels, but this was changed for performance.
* QNX Neutrino. This is the most successful microkernel ever. It deserves all the praise it gets. Yet it is still a niche product.
I would hardly call QNX a "niche" product. Running on everything from your car engine to Kiosk PCs (yes, that stupid iOpener ran it too), it's an extremely powerful and versatile operating system. Its Microkernel architecture even gives it the ability to be heavily customized for the needs of the application. Don't need networking? So don't run the server! Need a GUI? Just add the Graphics server to the startup.
Microkernels haven't failed. However, you may notice that nearly all the popular Operating Systems we use today were all developed back in the late 80's and early 90's. The real problem is that there hasn't been a need to develop new OSes until now. Now that Security and Stability are more difficult pressing issues than performance, we can go back to the drawing board and start designing new OSes to meet our needs for the next decade and a half.
Javascript + Nintendo DSi = DSiCade
You would be one of those uninformed pontificators Andy so eloquently railed against.
"For small embedded environments where speed or device support isn't a main concern. Micro-kernels will excel for their stability but take a look around and that's not reality or what we have today. We have lots of different hardware, lots of different interfaces and to manage that all via objects it'll just be extremely large."
And none of that has anything to do with monolithic versus microkernel, except perhaps tangentially. Microkernels do not ask each device driver to be a server all its own with zero code reuse, they use generic servers to wrap drivers for specific hardware while still isolating them from kernel space. This means there's no functional difference to the driver programmer from a monolithic to a microkernel architecture, either way you look at the driver interface and write the necessary code.
"If you think the linux kernel is big the relevant code for this would be numerous times larger. It just pushes the code from the kernel into userspace and you will definitely need more code to manage and access data structure"
Why do you suddenly need more code to the same thing? Andy's point is that when you stop sharing data structures, and instead start passing messages from one discrete server to another through well defined interfaces you reduce the amount of complexity (and therefor code) involved in protecting the coherency of those data structures. You will end up with more interfaces, but thats not necessarily a bad thing. I'd gladly trade all of the critical section protection logic for some nice interface logic. Especially since making the latter work reliably is a hell of a lot easier to do, and gives each subsystem the freedom to rework their internals without requiring me to lift a finger.
"If you can isolate your facets and only plan on supporting X number of devices/platforms/chipsets/etc and don't expect any blazing performance. Microkernels are great. Beyond that? With the rate that technology moves, it just becomes a management nightmare."
There's still no credible evidence to suggest that microkernel performance is that horrible, especially with modern clock speeds. Aside from gaming and large scientific compute clusters, very little being done today on a computer uses any significant measure of their speed. We've already covered how you're totally off base on device support (i.e. its orthogonal to the entire debate), and you throw "management nightmare" out there without bothering to define it, let alone defend it.
Large unix systems are already complex as hell to manage. A lot of that complexity is "hidden" in the kernel, which while fine for desktop users is a big pain for system administrators, and would be exposed for manageability in a microkernel setup.
As for OS X and its performance, its not horribly slow. Especially considering that your complaint almost certainly centers around PPC performance not x86, where it was hampered by lower clock speeds that were not counterbalanced by better IPC in any significant fashion. OS X's memory hunger has little to do with the kernel and lots to do with their operating environment, and all of the gee whiz graphical functionality that OS X brings along with it.
Ultimately though, OSX performance is a success story because on a G3 700mhz with 256M of ram its actually useable. Have you tried running Windows XP on a similar setup? Tried turning all of the eye candy on? Bet you didn't like the way it performed either.
More to the point, ``because it's faster'' has been the bane of Unix. To see that in stark relief, look at the shambles of NFS being in the kernel. Rather than fix the generic problems of providing a user-space nfsd, we saw a race into the kernel for a cheap my-code-only win, plus the horror of system calls that never return. Look at the vogue for in-kernel windowing systems (Suntools, for example) although X mercifully killed that off. Repeatedly we've seen massively complex and invasive kernel subsystems produced, when a generic solution to the problems that going into the kernel allegedly solves would have benefitted everyone for longer.
You've got a problem. You decide to solve it with a kernel extension. Now you've got two problems.
ian
Uhm, I'm pretty sure niche doesn't mean exceptionally widely deployed.
QNX is everywhere, you just don't realize it. ATMs run it, lots of medical equipment runs it, lots of other embedded apps that you don't even think of run it.
The examples Andy cites prove that in fact the microkernel concept has won in every single field where stability has gone beyond being something people wanted to something they demand. As soon as the general public realizes computers don't HAVE to crash, they'll win there too.
You can't seriously believe that running MINIX is going to magically give you expertise that lets you talk about operating system kernel design.
It's apparent from this thread that one needs no expertise whatsoever to talk about operating system kernel design, so running MINIX should if anything overqualify you.
code goes from spaghetti to whatever the opposite is:
Pizza
Find coupons in Greeley
So that's what you need to know about microkernels.
Before we get into arguments or understanding arguments, two most important things to note:
- AST is a prefessor. His interest in doing research and building the best systems for the *future* that he believes in.
- Linus is an engineer. His interest is building a system that works best *today*.
We simply need both. Without pioneering work done before in other OSes (this included failures), Linux wouldn't have been like this today. The greatest reason for its success it not it's doing something cool, but it's doing things that are proven to work.
So who is right? I'd say both. Linus has said this in 1992: "Linux wins heavily on points of being available now."
Linus admits microkernels are "cooler", but he didn't (doesn't) believe in it *today* because none of the available microkernels could compete with Linux as a *general purpose* OS. It's funny how AST listed "Hurd" as one of the microkernels - it totally defeats his own arguments. The fact is Hurd is still not available today despite it was started before Linux.
Many people talk about QNX. Sure, in many cases (especially mission critical, RTOS, where reliablility is so much more important than performance and usability) microkernels are better, but we really shouldn't compare a general-purpose OS with real-time or special purpose OS.
So we go back to the old way: code talks. So far microkernel proponents keep saying "it's possible to do microkernel fast, etc" but the fact is they have never had an OS that could replace Linux and other popular OS that everybody could run on their desktop with enough functionality. There are two possible reasons:
1. Lack of developers. But why? Do people tend to contribute to Linux because Linus is more handsome (than Richard Stallman that is)? There gotta be some reasons behind it other than oppotunities right?
2. Monilithic kernels are actually more engineerable than microkernels, at least for today.
Maybe 2 is actually the real reason?
Think about it.
No-no-no-no-NO! I swear this kills me... Why does this myth continue to propogate? The ONLY thing about NT that was EVER uKernelish was that it did alot of IPC (message passing) and that it implemented "personalities" (but it did so in a most decidedly non-microkernel way). Both of these traits were commonly associated with microKernels at the time, but regardless the things that ACTUALLY make a kernel a microKernel never existed in NT... EVER...
- sigs are stupid
Linux as a kernel is sufficiently mature that the problems Linus is spending time on are management and scalability problems--organizing the large-scale kernel hacking effort and dealing with massively parallel processing.
:)
I'd like to see Linus say "I've done a monolithic kernel and proven its success. Now I'm going to build a performant microkernel and see what all the fuss is about." He could hand over Linux kernel development to the senior crew that's already taking care of the major modules, and try something else.
Essentially, it would be cool for someone like Linus, with his incredibly strong practical engineering bent, to do again what he did with Linux: semi-clean-sheet a new kernel that meets his performance requirements, but is designed around different strategies for achieving what every OS tries to achieve.
I bet that, in two or three years, he would recant his earlier dismissal of microkernels and say that there's actually some interesting stuff there, and along the way solve some of the perennial complaints that slashdotters always bring up whenever microkernels are mentioned. In his heart of hearts, I'm sure Linus has some legacy issues with the current kernel design that he'd love to jettison, but can't without massively re-organizing the existing architecture, in which too many interested parties are already involved.
And he could put Stallman and the HURD boys to shame *again*, which is a twofer
Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
you mean like Tanenbaum (slashdotted, try later) did?
FTFA:
So **PLEASE** no more comments like "If Tanenbaum thinks microkernels are so great, why doesn't he write an OS based on one?" He did.
i don't reall know what you mean by proof of concept
again, FTFA:
It is definitely not as complete or mature as Linux or BSD yet, but it clearly demonstrates that implementing a reliable, self-healing, multiserver UNIX clone in user space based on a small, easy-to-understand microkernel is doable. Don't confuse lack of maturity (we've only been at it for a bit over a year with three people) with issues relating to microkernels.
i know this is slashdot, and RTFA is some kind of mortal sin, but please at least try.
Stop Computers/Cars Analogies on S
Has anyone thought of that the fact this very conversation may go down in the history of computer science? In 30 more or less years, lecturers will be telling their students about this argument! We're witnessing a more interesting slice of history than our normal mundane day lives :)
Giving IE users a taste of their own medicine since 2005 - http://pods.-is-a-geek.net/