Slashdot Mirror
Torvalds on the Microkernel Debate
diegocgteleline.es writes "Linus Torvalds has chimed in on the recently flamed-up (again) micro vs monolithic kernel, but this time with an interesting and unexpected point of view. From the article: 'The real issue, and it's really fundamental, is the issue of sharing address spaces. Nothing else really matters. Everything else ends up flowing from that fundamental question: do you share the address space with the caller or put in slightly different terms: can the callee look at and change the callers state as if it were its own (and the other way around)?'"
This my favorite Linus quote from that whole thread:
"In the UNIX world, we're very used to the notion of having
many small programs that do one thing, and do it well. And
then connecting those programs with pipes, and solving
often quite complicated problems with simple and independent
building blocks. And this is considered good programming.
That's the microkernel approach. It's undeniably a really
good approach, and it makes it easy to do some complex
things using a few basic building blocks. I'm not arguing
against it at all."
He basically continues his previous argument that monolithic kernels are more efficient and easier to implement. Microkernels may seem simpler, but they have complexity in implementing all but the simple tasks. Microkernels have a more marketable name. "Microkernel" just sounds more advanced than "monolithic". He finishes off with the observation that the term "hybrid kernel" is a trick to grab marketing buzz from the microkernel side of things.
My other first post is car post.
pfff, Linus, what would he know?
Philosophy.
Linus FTFA:
"The fundamental result of access space separation is that you can't share data structures. That means that you can't share locking, it means that you must copy any shared data, and that in turn means that you have a much harder time handling coherency. All your algorithms basically end up being distributed algorithms.
And anybody who tells you that distributed algorithms are "simpler" is just so full of sh*t that it's not even funny.
Microkernels are much harder to write and maintain exactly because of this issue. You can do simple things easily - and in particular, you can do things where the information only passes in one direction quite easily, but anythign else is much much harder, because there is no "shared state" (by design). And in the absense of shared state, you have a hell of a lot of problems trying to make any decision that spans more than one entity in the system.
And I'm not just saying that. This is a fact. It's a fact that has been shown in practice over and over again, not just in kernels. But it's been shown in operating systems too - and not just once. The whole "microkernels are simpler" argument is just bull, and it is clearly shown to be bull by the fact that whenever you compare the speed of development of a microkernel and a traditional kernel, the traditional kernel wins. By a huge amount, too.
The whole argument that microkernels are somehow "more secure" or "more stable" is also total crap. The fact that each individual piece is simple and secure does not make the aggregate either simple or secure."
The whole discussion of micro-kernel vs monolithic kernel is totally pointless. All popular OS kernels are monolithic. We can get back to the debate when we have a working fast microkernel in the market that is actually competitive.
Linus is a pragmatist. He didn't write Linux for academic purpose. He wanted it to work.
But you can always prove him wrong by showing him the code, and I bet he'd be glad to accept he was wrong.
Quick slashdoteffect there, that forum is already down. Anyhow.. mirror: http://www.mirrordot.org/stories/3f6b22ec7a7cffcf2 847b92cd5dec7e7/index.html
Any chance we could do this with my long distance phone service?
A-Bomb
http://pastebin.ca/54695
"The whole "microkernels are simpler" argument is just bull, and it is clearly shown to be bull by the fact that whenever you compare the speed of development of a microkernel and a traditional kernel,the traditional kernel wins. By a huge amount, too. He goes on to say, "It's ludicrous how microkernel proponents claim that their system is "simpler" than a traditional kernel. It's not. It's much much more complicated, exactly because of the barriers that it has raised between data structures." He states that the most fundamental issue is the sharing of address spaces. "Nothing else really matters. Everything else ends up flowing from that fundamental question: do you share the address space with the caller, or put in slightly different terms: can the callee look at and change the callers state as if it were its own?"
Insinct is stronger than Upbringing - Irish Proverb
Buy a copy of Windows Internals and educate yourself. Specifically, page 36: Is Windows a Microkernel-Based System?
Windows and Linux are not THAT different as far as kernel architecture is concerned.
I think Linus hit the spot by pointing out that the future of home computing is going to to focus on parallel processing - it's 2006 and all my computers, including my LAPTOP, are dual-processor systems.
By 2010 I suspect at least desktops are 4-CPU systems and as the numbers of cores increase one of the large drawbacks of microkernels raises it's ugly head: microkernels turn simple locking algorithms into distributed computing-style algorithms.
Every game developer tells us how difficult it is to write multi-threaded code for even our monolithic operating systems (Windows, Linux, OSX). In microkernels you constantly have to worry how to share data with other threads as you can't trust them to give even correct pointers! If you would explicitly trust them, then a single failure at any driver or module would bring down the whole system - just like in monolithic kernels but with a performance penalty that scales nicely with the number of cores. What's even worse is that at a multi-core environment you'll have to be very, very careful when designing and implementing the distribution algorithms or a simple user-space program could easily crash the system or gain superuser privileges.
Capitalization is the difference between "Helping your uncle jack off a horse" and "Helping your uncle Jack off a horse"
http://www.mklinux.org/
That would be monolithic+marketing.
MacOS X is no microkernel system. It does have Mach, sure. Mach is arguably not a microkernel by today's standards, and in any case MacOS X has a full BSD kernel bolted onto the Mach kernel. Mach and BSD are sharing address space. In other words, it's not a microkernel.
NT is the same way.
I don't know all that much about NetWare, but I'd never before heard anyone claim it to be a microkernel. It's not terribly popular anyway. (it was, but back then I'm sure it wasn't a microkernel system) ReactOS isn't much yet. BeOS died for unrelated reasons, so we really can't judge.
Monolithic kernels can be very modular. Microkernels can get really convoluted as the developers struggle with the stupid restrictions.
Individual pieces aren't really any simpler either. In fact, if you want your kernel to scale, to work well with lots of processes, you are going to run into a simple problem: multitasking.
Consider a filesystem driver in a monolithic kernel. If a dozen or so processes are all doing filesystem calls, then, assuming proper locking and in-kernel pre-emption, there's no problem - each process that executes the call enters kernel mode and starts executing the relevant kernel code immediately. If you have a multiprocessor machine, they could even be executing the calls simultaneously. If the processes have different priorities, those priorities will affect the CPU time they get when processing the call too, just as they should.
Now consider a microkernel. The filesystem driver is a separate server process. Executing a system call means sending a message to that server and waiting for an answer. Now, what happens if the server is already executing another call ? The calling process blocks, possibly for a long time if there's lots of other requests queued up. This is an especially fun situation if the calling process has a higher priority than some CPU-consuming process, which in turn has a higher priority than the filesystem server. But, even if there are no other queued requests, and the server is ready and waiting, there's no guarantee that it will be scheduled for execution next, so latencies will be higher on average than on a monolithic kernel even in the best case.
Sure, there are ways around this. The server could be multi-threaded, for example. But how many threads should it spawn ? And how much system resources are they going to waste ? A monolithic kernel has none of these problems.
I don't know if a microkernel is better than monolithic kernel, but it sure isn't simpler - not if you want performance or scalability from it, but if you don't, then a monolithic kernel can be made pretty simple too...
Forget magic. Any technology distinguishable from divine power is insufficiently advanced.
By today's standards, Mach is not much of a microkernel. Mach has been disowned by microkernel proponents because it was so big and nasty.
MacOS has that sharing address space with the monolithic BSD kernel. So a semi-microkernel and a monolithic kernel are firmly bolted together. That's only a microkernel if your degree is in marketing.
"You can do simple things easily - and in particular, you can do things where the information only passes in one direction quite easily, but anythign else is much much harder, because there is no "shared state" (by design). And in the absense of shared state, you have a hell of a lot of problems trying to make any decision that spans more than one entity in the system."
I think you're looking at this the wrong way around.
There has been a lot of research into this over the past 40 years, ever since Dijkstra first talked about coordination on a really big scale in the THE operating system. Any decent CS program has a class on distributed programming. Any decent SW architect can break down these different parts of the OS into weakly-connected pieces that communicate via a message passing interface (check out this comment by a guy talking about how Dragonfly BSD does this).
It's obvious that breaking something like your process dispatcher into a set of processes or threads is silly, but that can be easily separated from the core context switcher. Most device driver bottom halves live fine as a userland process (each with a message-passing interface to their top-halves).
If you're compiling for an embedded system, I'm sure you could even entirely remove the interface via some #define magic; only debug designs could actually have things in separate address spaces.
The point I'm trying to make is: yes, you can access these fancy data structures inside the same address space, but you still have to serialize the access, otherwise your kernel could get into a strange state. If you mapped out the state diagram of your kernel, you'd want the transistions to be explicit and synchronized.
Once you introduce the abstraction that does this, how much harder is it to make that work between processes as well as between threads in the kernel? How much of a benefit do you gain by not having random poorly-written chunks pissing over memory?
How about security benefits from state-machine breakdowns being controlled and sectioned off from the rest of the machine? A buffer overflow is just a clever way of breaking a state diagram and adding your own state where you have control over the IP; by being in a separate address space, that poorly written module can't interact with the rest of the system to give elevated privileges for the attacker (unless, of course, they find flaws in more of the state machines and can chain them all together, which is highly unlikely!).
Clearly there is a security benefit as much as there is a consistency benefit. Provably correct systems will always be better.
--
Internet Explorer (n): Another bug -- that is, a feature that can't be turned off -- in Windows.
You are forgiven for being wrong, but not for spouting off nonsense despite knowing that you don't know what you're talking about, apparently applying the principal "if my argument involves M$ doing the wrong thing, it must be right".
While neither NT nor Mac OS X are true microkernels, the architecture of both is strongly inspired by microkernel ideas. Like Linus, the developers of these kernels recognized the practical difficulties involved in making full-on microkernels work, but unlike Linus, instead of throwing in the towel completely and doing full-on monolithic kernels, they created cleanly seperated layers interacting via well-defined interfaces whenever they practically could.
If you talk to kernel programmers, most will express a high degree of respect for the NT kernel, which is based on the DEC VMS kernel. It mostly the poor design of systems that sit on top of the kernel that has earned Windows its reputation.
Hi folks,
I worked two years for a society that was developing its own micro-kernel system, for embedded targets. I was involved in system programing and adaptation of the whole compiler tools, based on GCC chain.
Linus is right: basic problem is address space sharing, and if you want to implement memory protection, you rapidly falls into address space fragmentation problem.
The main advantage of the system I worked on wasn't really its micro-kernel architecture, but the fact that its design allowed to suppress most of glue code that is needed between a C++ program and a more classic system.
In my opinion, micro-kernel architecture has the same advantage and drawbacks that so-called "object-oriented" programing scheme : it is somewhat intellectually seducive for presentations but it is just a tool.
It would certainly be intersting for Linux to provide the dynamic link management specificities of a micro-kernel system, for instance to allow someone to quickly modify IP stack for its own purpose, but should the whole system being design that way ? I am not sure.
If you want to have an idea of the problem encountered with programing for these systems, one can look at the history of the AmigaOS, which have a design very close to a micro-kernel one.
If the Linux kernel had have been coded using Forth.
Just saying.
abstraction and state isolation considered harmful
That's a pretty big assumption. Or rather, you have basically taken all the hard parts of doing shared code and said "Let's hope someone else already solved this for us".
Sooooo, it's easy to have someone else handle the multi-process bits in a monolithic design. But when it comes to writing services for microkernels suddenly everyone is an idiot?
Besides, as Linus pointed out, when data is going one way microkernels are easy. And in the case of file systems that is really the case. Sure multiple processes can access it at once, but the time scale on handling the incoming signals is extremely fast compared to waiting for data from disk. Only a really, *really* incompetent idiot would write such a server which blocked until the read was finished.
In the end it boils down to the old question centralisation vs. local autonomy. Centralisation is fine for keeping state, it is fine for enforcing a thoroughly similar approach to everything, it helps with 'single points of contact'. Local autonomy helps with less administrational effort, with clearly defined information paths and with clear responsibilities, thus with keeping problems locally.
Both approaches have their merits, and in the real world you will never see a purely central organisation or a purely localized organisation. Every organisation is somehow swinging between both extrema, going more central at one point "to leverage synergies and increase efficiency", or is starting outsourcing and reorganizing itself into profit centers, to "overcome bureaucracy, to clearly define responsibilities and to cut down on administrational spending".
The limits are given by the speed information is created, sent and decoded within the different organisational paths. An increase in Inter Process Communication speed will help with a more modularized microkernel approach, an increase in number and complexity of concurrent requests demands a more centralized kernel.
In the end it boils down to the fact, that transactions have to be atomar operations, either being executed completely or rolled back completely if not finished. Centralized systems are inherently transactional, especially if they are executing tasks sequentially. The limit is given with the numbers of transactions that can be executed per time unit. Parallel execution demands operations to be as independent of each other as possible, thus increasing design efforts, but once the task is (nearly) interlock free, a modularized approach helps with faster, better maintenable code.
Name: Linus Torvalds (torvalds AT osdl.org) 5/9/06
___________________
_Arthur (Arthur_ AT sympatico.ca) on 5/9/06 wrote:
I found that distinction between microkernels and "monolithic" kernels useful: With microkernels, when you call a system service, a "message" is generated to be handled by the kernel *task*, to be dispatched to the proper handler (task). There is likely to be at least 2 levels of task-switching (and ring-level switching) in a microkernel call.
___________________
I don't think you should focus on implementation details.
For example, the task-switching could be basically hidden by hardware, and a "ukernel task switch" is not necessarily the same as a traditional task switch, because you may have things - hardware or software conventions - that basically might turn it into something that acts more like a normal subroutine call.
To make a stupid analogy: a function call is certainly "more expensive" than a straight jump (because the function call implies the setup for returning, and the return itself). But you can optimize certain function calls into plain jumps - and it's such a common optimization that it has a name of its own ("tailcall conversion").
In a similar manner, those task switches for the system call have very specific semantics, so it's possible to do them as less than "real" task-switches.
So I wouldn't focus on them, since they aren't necessarily even the biggest performance problem of an ukernel.
The real issue, and it's really fundamental, is the issue of sharing address spaces. Nothing else really matters. Everything else ends up flowing from that fundamental question: do you share the address space with the caller, or put in slightly different terms: can the callee look at and change the callers state as if it were its own (and the other way around)?
Even for a monolithic kernel, the answer is a very emphatic no when you cross from user space into kernel space. Obviously the user space program cannot change kernel state, but it is equally true that the kernel cannot just consider user space to be equivalent to its own data structures (it might use the exact same physical instructions, but it cannot trust the user pointers, which means that in practice, they are totally different things from kernel pointers).
That's another example of where "implementation" doesn't much matter, this time in the reverse sense. When a kernel accesses user space, the actual implementation of that - depending on hw concepts and implementation - may be exactly the same as when it accesses its own data structures: a normal "load" or "store". But despite that identical low-level implementation, there are high-level issues that radically differ.
And that separation of "access space" is a really big deal. I say "access space", because it really is something conceptually different from "address space". The two parts may even "share" the address space (in a monolithic kernel they normally do), and that has huge advantages (no TLB issues etc), but there are issues that means that you end up having protection differences or simply semantic differences between the accesses.
(Where one common example of "semantic" difference might be that one "access space" might take a page fault, while another one is guaranteed to be pinned down - this has some really huge issues for locking around the access, and for dead-lock avoidance etc etc).
So in a traditional kernel, you usually would share the address space, but you'd have protection issues and some semantic differences that mean that the kernel and user space can't access each other freely. And that makes for some really big issues, but a traditional kernel very much tries to minimize them. And most importantly, a traditional kernel shares the access space across all the basic system calls, so that user/kernel difference is the only access space boundary.
Now, the real problem with split acce
Here is some good readign amterial, maybe people should read and _understand_ it before posting on the subject..
i tem.5d61c1d591162e4b0ef1bd108bcd45f3/index.jsp?&pN ame=computer_level1_article&TheCat=1005&path=compu ter/homepage/0506&file=cover1.xml&xsl=article.xsl&
0 ,1144,0131429388,00.html
This does not mean you have to agree with the guy.
http://www.computer.org/portal/site/computer/menu
http://vig.prenhall.com/catalog/academic/product/
Distributed algorithms are of course difficult to implement with a f***ed up language like C.
Here, it seems, the means justify the ends. Linus basically says "I won't take any challenges".
Linus tells me that we can never write a proper scalable OS for a NUMA machine, or a modular system that can serve well to parallel I/O systems and the like. I highly disagree.
Because these things are not pipe dreams, they have been done. IBM guys have made amazingly abstract and modular OS stuff and they've been using them for years, so I think it's rather pathetic to say that there is only one true path to OS implementation. Why not admit that it is the only path that you have any experience in?
--exa--
I know as a good and faithful /.-ers we should worship Linus and take all of his words as gospel, but in this case I think he is talking out of his arse. Microkernels are "more secure" and "more stable" because only one component needs to work well -- the microkernel, it's main job is to enforce security policies and that is it. If it works correctly it will be able to bring the system to a certain state during the failure of any of the other components.
Microkernels are used and have been used for a long time in "real" and "serious" operating systems, not just toy examples. Everytime /.-ers fly over the Atlantic it is a microkernel OS in all probability that makes sure they don't crash and burn. The size of those microkernel is kept at no more than 10k lines -- and even so it can take years to prove its correctness. It would be impossible to do it with Linus's kernel. So if Linus and others are so against the microkernel acrhitectures I would want to see them trust their lives to a Linux 2.6 -- put their lives were their mouth (or code) is, so to speak.
I don't want to repost this old debate that I believe every geeks should have read it; but since nobody post it yet. I repost it for anybody who haven't read about this famous debate between Linus and Prof. Tanenbaum on microkernel.
Linus vs. Tanenbaum - "Linux is obsolete" Jan,1992
(Save your mod point for someone who really need them thanks!)
The analogy of centralisation vs. local autonomy is not totally accurate either. Both the monolithic and the microkernel are centralized, except that in the first case there a large beaurocratic structure and in the second case it just a dictator and a couple of "advisors". If the dictator or the king is chosen well, the system will be more predictable and will work much better. If case of the large beaurocratic system, if some of its members get corrupted [and they will because there are so many of them] the whole system will fail. It is like saying that a small bug in the mouse driver will freeze and crash the system with a monolithic kernel. Good thing if the system was only running Doom at the time and not controlling a reactor, or administering a drug. If the same happens in the microkernel system, the kernel will reload the driver, raise an alarm, or in general -- be able to take the system to a predictable predetermined state. Going back to the analogy is it is like having the dictator execute a corrupted staff member and replace him immediately.
Andy likes microkernels because they force you to do that. Time spent on design leads to insight, which may well point to better and cleaner ways to do the task you originally set out to acomplish.
Linus hates microkernels because they force you to do that. Time spent on design is time lost getting working code out the door, and working code will give you experience that will point to better and cleaner ways to do the task you originally set out to acomplish.
Individual pieces aren't really any simpler either. In fact, if you want your kernel to scale, to work well with lots of processes, you are going to run into a simple problem: multitasking.
This is very true.
Consider a filesystem driver in a monolithic kernel. If a dozen or so processes are all doing filesystem calls, then, assuming proper locking and in-kernel pre-emption, there's no problem - each process that executes the call enters kernel mode and starts executing the relevant kernel code immediately.
OK, here's where things start getting a little tricky. The whole locking setup in a monolithic kernel is pretty tricky. Early multi-processor kernels often took the course of "one big lock" at the top of the call stack - essentially only one process could be executing in the kernel. Why? Because all that "proper locking" is tricky. Took years to get this working right. Of course it's done now in Linux so you can take advantage of it, but it wasn't easy.
Now consider a microkernel. The filesystem driver is a separate server process. Executing a system call means sending a message to that server and waiting for an answer.
OK, now here, you're kind of running off the rails. What is a "message"? There is no magical processor construct called a "message" - it's something that the OS provides. How messages are implemented can vary quite a bit. What you're thinking of is a messaging system ala sockets - that is the message would be placed onto a queue and then a process switch would happen sometime and the server on the other end would read messages out of the queue and do something. That's how microkernels are usually presented conceptually so it tends to get stuck in peoples' heads.
However, messages can be implemented in other ways. For example, you could make a message be more like a procedure call - you create a new stack, swap your address table around, and then jump into the function in the "server". No need to instantiate threads in the "server" anymore than there is a need to instantiate threads within a monolithic kernel. The server would essentially share the thread of the caller. I've worked on microkernel architectures that were implemented just this way.
If the number of data structures that you can directly access is smaller, the amount of locking that you have to take into account is smaller. Modularity and protection makes most people's tasks easier.
Many of the arguments made for monolithic kernels are similar to the arguments you used to hear from Mac programmers who didn't want to admit that protected memory and multi-tasking were good things. Mac programmers liked to (as I used to say) "look in each other's underware". Programs rummaged about through system data structures and other apps data structures sometimes, changing things where they felt like it. This can be pretty fun sometimes and you can do some really spiffy things. However, set one byte the wrong way and the whole system comes crashing down.
The only way the monolithic vs microkernel debate will go away is if CPUs provide a better way of sharing resources between modules.
One solution to the problem is to use memory maps. Right now each process has its own address space, and that creates lots of problems. It would have been much better if each module had its own memory map, ala virtual memory, so as that the degree of sharing was defined by the O/S. Two modules could then see each other as if they belong to the same address space, but other modules would be inaccessible. In other words, each module should have its own unique view of the memory.
Of course the above is hard to implement, so there is another solution: the ring protection scheme of 80x86 should move down to paging level. Each page shall have its own ring number for read, write, and execute access. Code in page A could access code/data in page B only if the ring number of A is less than or equal to the ring number of B. That's a very easy to implement solution that would greatly enhance modularity of operating systems.
A third solution is to provide implicit segmentation. Right now 80x86 has an explicit segmentation model that forces inter-segment addresses to be 48 bits wide on 32-bit machines (32 bits for the target address and 16 bits for the segment id). The implicit segmentation model is to use a 32-bit flat addressing mode but load the segment from a table indexed by the destination address, as it is done with virtual memory. Each segment shall have a base address and a limit, as it is right now. If a 32-bit address falls within the current segment, then the instruction is executed, otherwise a new segment is loaded from the address and a security check is performed. This is also a very easy to implement solution that would provide better modularization of code without the problems associated with monolithic kernels.
There are various technical solutions that can be supported at CPU level that are not very complex and do not impose a big performance hit. These solutions must be adopted by CPU manufacturers if software is to be improved.
My carreer started in Operating System Research, this was circa 1993. Even in those days there were many people addressing the shared memory issue and coming up with good ways to share memory and address the context switch issue. However this took some overhead and did not make it to the mainstream because of that.
Today the CPUs are much faster and even sacrificing 10%-20% of CPU power is not considered too much if it results in a system that is (more) stable and easier to maintain. e.g. a device driver can no longer bring down the entire system and a spyware program can no longer sniff all keys pressed...
I must admit to have lost contact with that field of research but even the old results are promising, with today's CPU speeds.
What's the problem with monoliths, that they are supposed to be less marketable? Ever since 1968, Monoliths have been doing great!
I'm in a Unix state of mind.
That's interesting because those are exactly my thoughts every time I hear the arguments people use to defend microkernels: Java is to microkernels as C/C++ is to monolithic kernels.
Linus Torvalds summed it well when he mentioned that microkernels are simpler only when data flow goes in one direction only. It's very hard to get a function to fill a complicated data structure for you if you cannot work with pointers. Passing a reference will do only for simple structures, it will not work if there are structures within structures, it is very hard to do if the called function must itself pass some subset of that structure to another function. And for operating systems where one must contend with multiple access and locking, it's almost impossible to do without a performance penalty.
Let's face it: pointer manipulation is necessary because there are real life problems that are more complex than textbook examples. If there weren't, inventing the C language wouldn't have been necessary, we could have stuck with Fortran all along.
Well maybe that's how *you* would design *your* Microkernel. And yes, it would suck.
The way I would design the filesystem driver, would be to accept a request, add it to a queue of pending requests to serve. If there are no initiated requests, find the request that can most efficiently be served based upon your preferred policy (closest seek time, for example, or first come first serve, your choice), and initiate that request. Add some smarts for multiple devices, so multiple requests can be initiated at the same time to different devices. When data comes back, answer the requesting process with their data. Rather than sitting around blocking on a request, go grab more requests from other processes and queue them up. No need to block. When an initiated request comes back, send back the data to the requesting process, and everyone's happy. Just because things are separated out into different processes, doesn't mean that they can't do some asynchronous juggling to be efficient. Add multi-threading, and the coding becomes a bit easier; but multi-threading isn't necessary to rely upon to have this work well.
I'm pretty sure the monolithic kernels do things somewhat similarly; build a request queue, service that queue. They could also block until they're done other requests, but that would be bad design. Don't assume a Microkernel Filesystem server has to suffer from similarly bad design.
Love many, trust a few, do harm to none.
Microkernels like Mach have been unsuccessful because putting everything into separate address spaces makes a lot of things quite difficult.
C-based monolithic kernels like Linux and UNIX run into software engineering problems--it gets harder and harder to ensure stability and robustness as the code mushrooms because there is no fault isolation.
The solution? Simple: get the best of both worlds through language-supported fault isolation (this can even be a pure compile-time mechanism, with no runtime overhead). It's not rocket science, it's been done many times before. You get all the fault isolation of microkernels and still everything can access anything else when it needs to, as long as the programmer just states what he is doing clearly.
C-based monolithic kernels were a detour caused by the UNIX operating system, an accident of history. UNIX has contributed enormously to information technology, but its choice of C as the programming language has been more a curse than a blessing.
Whilst microkernels are a lovely idea in theory, they don't deliver in practice. There is already a bottleneck between user space and kernel space and this will impact upon performance. No matter what you are trying to do, the slowest part of the process will always determine the maximum rate at which you can do it.
Monolithic, Linux/Netware-style modular and so-called hybrid kernels get around this limitation by moving things to the other side of the bottleneck. It makes sense on this basis to put a hardware driver in kernel space. You usually only pass "idealised" data to a driver; the driver generally has to pass a lot more to the device because it isn't ideal. For example, when talking to a filesystem driver, you generally only want to send it the data to stick into some file. The filesystem driver has to do all the donkey work of shunting the heads back and forth and waiting for the right spot of disc to pass under them.
It might be "beautiful" to have as little code as possible situated on one side of the division, but it's most practical to have as little data as possible having to travel through the division.
Je fume. Tu fumes. Nous fûmes!
The kernel is becoming too big, and some parts of it (think: hamradio, USB gadgets
I am still craving for the day when Linus will define a "kernel ABI" for driver modules, and some parts of the kernel source will get kicked out of the
I believe this day will come; and there are good reason to believe it
-
suppose FOSS dream comes true, and linux ever become the mainstream desktop OS, and every vendor supplies FOSS drivers for their hw.... it will not make sense to ship any single gadget/protocol driver in the same
.tar.bz2
- even today, it does not make sense to ship drivers forever, for hw that is now unavailable to buy; but at the same time it would be unfair to
just drop the code to people who still own that hw.
- having 220MB of source code without a published and enforced ABI for modules means that any change to some parts of the kernel, such as memory management, force almost everybody to rewrite their code ; this, in the long term, may foster innovation. It would be much better if there were some stable ABIs for drivers for lesser demanding drivers (such as webcams).
Summarizing, IMHO the current monolithic situation cannot scale up forever.Only a really, *really* incompetent idiot would write such a server which blocked until the read was finished.
This sounds like a veiled reference to something; would you care to name it?
"Sometimes the truth is stupid." - Lawrence, creator of Prime Intellect
Whenever this issue comes up, I swear to myself that proponents of microkernel architectures created the term which they use to address their opponent. The terms used to discuss this are heavily loaded. “Microkernel” sounds lean, quick, and simple, while by subjective contrast “monolithic” sounds bulky, old, and unwieldy. I think that when engaging in this debate, it is best that we prefer to at least use “unified kernel” in place of “monolithic”, being it is more accurate and contrasts with “microkernel” objectively. The term most people use for kernels like Linux and NT seems to imply that there is no logical separation of components and that all pieces are somehow a gigantic (dare I say monolithic) glob and that is nonsense.
Join Tor today!
While neither NT nor Mac OS X are true microkernels, the architecture of both is strongly inspired by microkernel ideas.
What exactly does "inspired" mean in this case? I am "inspired" by John Holmes but that doesn't mean I have a 12" cock does it?
If you talk to kernel programmers, most will express a high degree of respect for the NT kernel, which is based on the DEC VMS kernel. It mostly the poor design of systems that sit on top of the kernel that has earned Windows its reputation.
So, did VMS have a graphics subsystem in the kernel as well? Also can you provide some examples of kernel experts praising the NT kernel for its microkernel properties? Thanks in advance.
The problem is that it doesn't match the way most people work right now.
Check out this brilliant paper by Alistair Cockburn (spoken as Co-burn) - Characterizing People as Non-Linear, First-Order Components in Software Development. Over and over in this paper he says:
- Problem 1. The people on the projects were not interested in learning our system.
- Problem 2. They were successfully able to ignore us, and were still delivering software, anyway.
Alistair comes to an equally brilliant conclusion: In short, without Linus, microkernels may help. With Linus, a monolithic kernel works fine.If you've ever worked on a software project with more than four people, didn't the personality and skills of the people involved make more of a difference than any methodology, abstraction, or even the language used? That's always been true in my experience.
Shae Erisson - ScannedInAvian.com
When people go and by a new computer, many are willing to spend hundreds of dollars more to get a little extra performance. They will not like to have that taking away from them just so that they can run a microkernel.
Agreed, but these are the same people who don't really care if they're at the bleeding edge of technology and have to deal with super-l33t video driver 0.8.2b crashing every so often.
It is just a false premise, just because some monolithic (or hybrid) kernels are unreliable does not mean that it is necessary or better to use microkernels to get reliability.
I'm afraid the false assumption has fallen upon you here. It's proven, time and again, that microkernels make it simpler to guarantee security and reliability. Yes, microkernels hurt performance a bit. However, it's throwing the baby out with the bathwater to discard microkernel architecture because of performance. By discarding microkernel architecture, you also discard the architectural segregation of system services, which makes for a very simple way to segragate effort when you've got say, an open-source project with developers located around the globe. Microkernels also ease maintenance, again, because the kernel remains small.
A HUGE misstep that the monolithic kernel camp has made by pointing fingers at microkernels' performance is this: smart coders write good, clean, testable, reliable, secure programs first, and optimize later. One should never, EVER attempt to optimize code on the first pass. Write it, test it, fix it, THEN optimize it. Leave the clearly non-optimal code in during the first few passes, just for the sake of maintenance. Then, you'll have a good, clean codebase from which to begin profiling and optimizing.
I can personally testify to spending exorbitant amounts of time debugging architecturally un-sound code, written intentionally so, because a wise architectural decision would've resulted in lesser performance. Out of curiosity, I usually run profilers over this code. Most often, these architectural decisions result in such a negligible performance increase that it pales in comparison to the maintenance nightmare that ensues from it.
There's nothing magical about microkernels that prevents them from being optimized, just like any other program. The benefits of the microkernel architecture certainly outweigh the performance hit in my book.
I recommend sodium chloride (5mg) if you disagree with any of the above.
I pity the foo that isn't metasyntactic
Sooooo, it's easy to have someone else handle the multi-process bits in a monolithic design. But when it comes to writing services for microkernels suddenly everyone is an idiot? I don't think that was what was meant. Thing is, with a monolithic kernel every process can run in userspace or in kernel space. (OK, I'm assuming a 1:1 kernel threads to user threads, but lets ignore that issue for now) Because all the processes have a kernel context, it's quite simple to manage multiple processes making use of the kernel at once. Run kernel code to service the filesystem request in the kernel-mode context of the process that made the request. At this point, if the kernel supports pre-emption then it's possible to pre-empt that process during kernel-mode execution and resume it later. In a microkernel you don't get that. The filesystem server doesn't implicitly get its multithreading from each process being represented by a kernel thread - because it can't access the kernel. If it wants an implementation of multithreading it has to implement its own (nb. could use a library to simplify this). But lots of tasks now require explicit extra code: the FS server might need to handle scheduling priorities itself, because the internals of the server aren't under direct control of the kernel; threading in the FS server must be made explicit because it doesn't come "for free" when processes are created. This is a problem for every server that needs to serve multiple clients simultaneously. A lot of this should be solvable with library code, I have thought. But the basic argument here (AFAIK) is that you have to code up somewhere in userspace extra implementations to take care of things that are implicitly provided for in a monolithic design.
What you're talking about is very close to CPS.
There are some ways to convert normal function calls to CPS.
And there is something called monads used to convert imperative algorithms to functional style.
And yes, continuations can be a very powerful technique.
However, CPS functional code is still coding an algorithm. Any way to compute something is an algorithm. May be you should name your critic "I dislike imperative algorithms, and I like CPS functional algorithms."
We are Turing O-Machines. The Oracle is out there.
IBM is shipping it. Novell, RedHat, WindRiver, LinuxWorks, Motorola, Sharp, Sony, Hell even I'm shipping it in embedded products. It is easy to "prove it works" as alluded to in another post.
Microkernels are also shipping from QNX and, uh and, oh I'm sure there are a few more. (Not knocking QNX, considered it but tossed it, due to cost and liscensing.)
Is one more secure or stable than another, is really the wrong question.
The question is really is the "System" designed with microkernel more or less stable or secure or functional then the alternative.
I think it has, to my satisfaction, been settled. From RevolutionOS the movie (BuyIt!) Stallman is asked why HURD is so far behind Linux. His answer, (paraphrased, sorry RMS) Turns out a microkernel is very difficult to pull off because of the constant stream of messages required for the simplest of tasks. This forced ovehead only makes the Kernel more secure not the system, if the "drivers" keep crashing out and restarting you could go months without noticing critical flaws. "But the kernel is rock solid" doesn't really help if I can't ship the "System", does it. The only evidence you need, is the development pace of Hurd or even QNX to show this.
I respect the professor and his work, but it was an inspiration for a much more scalable design that clearly is superior for the rapid development a modern OS is expected to have.
As an engineer I see the beauty, but as a Production Engineer I can also see the added complexity a microkernel brings.
Of course you could, argue theoretically that I'm wrong or prove it by making a GNU/Minix distribution to compete in the real world with Linux. Almost 15yrs and a flood of Students haven't helped Professor T, produce it yet. Admittedly not his goal, but come on I know students and CompSci students have a knack for carrying their favorite teacher/classes with them throughout their career and it shows up in their projects.
OSGGFG - Open Source Gamers Guide to Free Games
I worked on a commercial microkernel OS.
The learning curve was very steep. New developers took at least half a year to be productive. A number of people never became productive and had to be fired.
Linux is really clean and tidy compared to that. Even BSD is clean and tidy compared to that microkernel OS.
Separated components tend to get complex interactions. Sharing data can be very awkward, even if you are co-located.
NFS.
/mnt/]$ mount compy2:/share /mnt/compy2 /mnt/compy2]$ ls -la
1) [root@compy1
2) Unplug cable (power or network) to COMPY2
3) [root@compy1
It will be done right when Duke Nukem: Forever finishes installing.
When modding "Informative", please make sure it both has a source and IS actually informative.
Also can you provide some examples of kernel experts praising the NT kernel for its microkernel properties?
W2K does not have a pure microkernel architecture but what Microsoft refers to as a modified microkernel architecture. As with a pure microkernel architecture, W2K is highly modular. Each system function is managed by just one component of the operating system. The rest of the operating system and all applications access that function through the responsible component using a standard interface. Key system data can only be accessed through the appropriate function. In principle, any module can be removed, upgraded, or replaced without rewriting the entire system or its standard application program interfaces.
William Stallings, "Operating Systems Internals and Design Principals", Fourth Edition, pp. 86-87.
So, did VMS have a graphics subsystem in the kernel as well?
No. In NT 3.1, the graphics subsystem ran in user-space. In NT 4.0, it was moved into kernel-mode to avoid the performance hit of the context switch. As this history suggests, the actual architectures of the executive and the graphics subsystem are not tightly coupled. They share an address space for performance reasons, not in order to share state. (To be clear, I don't think this is a great thing, and it is a violation of microkernel principals. But the programmers made the smallest possible departure from microkernel principals to achieve their performance requirement.)
I find it highly amusing that many of the same people who defend the monolithic Linux kernel architecture that tightly couples so many subsystems to the kernel also attack Windows for running the graphics subsystem in kernel mode.
You choose for that filesystem call to wait forever.
/mnt/compy2
/mnt/compy2
Try
mount -o soft compy2:/share
or
mount -o intr compy2:/share
instead
man 8 mount
--- Hindsight is 20/20, but walking backwards is not the answer.
I don't really give a fork.
We see this misconception way too often. Modular design, well defined interfaces, etc. have nothing to do with a microkernel. You can apply those excellent design principles in a monolithic kernel, you can even apply them in regular applications :-) The one distinguishing feature of a microkernel is the separation of address spaces of its components.