Slashdot Mirror

https://www.l4ka.org/126.php

Opteron 1.6GHz - 0.14s context switch

This on a 11 year old CPU...

Not that slow today, provided you use the right microkernel. Look at L4Ka::Pistachio, for example, if you're looking for very fast context switching and message passing in registers without overhead. Now, if you talk Mach, then you're right.

The speed and memory overhead differences of C++ are large and noticeable.

Speaking of kernels, it IS possible to write efficient and fast ones in C++ too, e.g. L4Ka::Pistachio. You only have to know C++ very well to avoid its weaknesses.

I hope the new OS will be a microkernel one, like L4Ka (or L4 in general) or Minix.

Please people, could someone tell me why is this modded funny? I'm serious, minix isn't a good idea here?

I hope the new OS will be a microkernel one, like L4Ka (or L4 in general) or Minix.

<punAlert>
Well, at least then they could join the Hurd.
</punAlert>

I hope the new OS will be a microkernel one, like L4Ka (or L4 in general) or Minix.

Another good example of a very performant microkernel written in C++ is L4Ka.

The distribution terms for Bison-generated parsers permit using the parsers in nonfree programs. Before Bison version 2.2, these extra permissions applied only when Bison was generating LALR(1) parsers in C. And before Bison version 1.24, Bison-generated parsers could be used only in programs that were free software.

Another example? Okay, how about IDL4? Suppose you want to write a BSD-only OS on top of L4Ka::Pistachio (which is BSD licensed too), and you wanted to make use of IDL4 (which is GPLed) to simplify IPC. You're in again for a bad surprise, because the IDL4 generated stubs are (probably) GPLed too, nullifying your intention to create a BSD-only OS. A simple special rule w.r.t. the generated stubs could have been added by IDL4 developers just as the bison(1) developers thankfully did, but IDL4 devs didn't. So it's not only the "bad greedy companies" that play tricks on build environments, it happens in the OSS community as well (sometimes intentionally, sometimes by accident).

The distribution terms for Bison-generated parsers permit using the parsers in nonfree programs. Before Bison version 2.2, these extra permissions applied only when Bison was generating LALR(1) parsers in C. And before Bison version 1.24, Bison-generated parsers could be used only in programs that were free software.

Another example? Okay, how about IDL4? Suppose you want to write a BSD-only OS on top of L4Ka::Pistachio (which is BSD licensed too), and you wanted to make use of IDL4 (which is GPLed) to simplify IPC. You're in again for a bad surprise, because the IDL4 generated stubs are (probably) GPLed too, nullifying your intention to create a BSD-only OS. A simple special rule w.r.t. the generated stubs could have been added by IDL4 developers just as the bison(1) developers thankfully did, but IDL4 devs didn't. So it's not only the "bad greedy companies" that play tricks on build environments, it happens in the OSS community as well (sometimes intentionally, sometimes by accident).

Oh, how wrong this is.

You should look into the L4 microkernel project some time, and its follow-ups (e.g. Pistachio, Fiasco)

In a nutshell: The reason most "microkernels" have bad performance is that they are not anywhere near "micro" enough.

That is essentially what Apple has done.

No, it isn't. He's talking about a microkernel, like QNX or L4. Mach is also considered a microkernel, but OS X is not based on Mach; it is based on XNU, a hybrid kernel that consists of Mach bolted into the FreeBSD monolithic kernel, with I/O Kit thrown in for good measure.

You give performance arguments, but the speed of OX X does not imply that it uses a microkernel; quite the opposite, microkernels have been known to run significanly slower than monolithic ones, which is presumably why Apple went the hybrid route instead of using pure Mach. To be fair, Mach is rather outdated among microkernels, and "modern" microkernels like the aforementioned L4 have shown that the speed penalty might be as low as 5-10%, but the point still stands. Don't confuse simplicity of code with performance.

Honestly, if there was a single Free hypervisor that worked with VMI (and L4 doesn't count ;-)), VMI would already be in the kernel.

Except that VMI is not a closed-source interface. The VMI interface is an ABI specification that any hypervisor can use. It basically replaces the privledged instructions with calls to a memory block that the hypervisor sets up. The hypervisor can make them calls to the hypervisor. On a virtualized processor, it could leave them alone. Or if there is no hypervisor, the kernel can run natively. There are already open source hypervisors that use VMI. L4 is being used as a hypervisor using VMI. They also have a technique called pre-virtualization which replaces the priviledged instructions in a binary OS (ie Windows) with the VMI calls to allow them to be virtualized without patching the source like Xen (and normal VMI) requires.

Greg was talking about the modules that VMware distributes for the host-side device drivers. They are distributed as proprietary source which need to be compiled for the local kernel.

Except that VMI is not a closed-source interface. The VMI interface is an ABI specification that any hypervisor can use. It basically replaces the privledged instructions with calls to a memory block that the hypervisor sets up. The hypervisor can make them calls to the hypervisor. On a virtualized processor, it could leave them alone. Or if there is no hypervisor, the kernel can run natively. There are already open source hypervisors that use VMI. L4 is being used as a hypervisor using VMI. They also have a technique called pre-virtualization which replaces the priviledged instructions in a binary OS (ie Windows) with the VMI calls to allow them to be virtualized without patching the source like Xen (and normal VMI) requires.

Greg was talking about the modules that VMware distributes for the host-side device drivers. They are distributed as proprietary source which need to be compiled for the local kernel.

there's always these pistachio people, and i've heard you can even run linux on top of it.

Trying to turn a monolithic kernel into a microkernel has been attempted before. The result was the abomination which is Mach.

1. CapROS (successor to EROS)
2. L4Ka and Fiasco implementations of the L4 microkernel architecture

Ice anyone ?

Good idea ! Pistachio icecream for me :)

Perhaps Afterburning will help. The original source is re-compiled with a modified toolchain that makes it easy to run the binary (such as a Linux kernel) unmodified on raw hardware or to fix it up to run cleanly in Xen or L4 (amongst others). The hardware changes will still be needed for proprietary OS like Windows, but would be a big help for any Free OS on older hardware.

It does not forbid variations in field padding, however. The rules for how fields are padded are predictable, but that does not change the fact that they are not "byte for byte" translations.

Right, which is why you stick to machine words.

Hardly. Even microkernels are full of logic (validating user input, for example). Even if you say a microkernel spends 50% of its code banging registers and memory locations (likely a gross overestimation), that's still 50% you don't have to write in a low-level language.

I think we're getting our signals crossed here. I've never stated C is ideal for writing kernels. I can honestly say I really don't like C. I would love to be able to write a kernel in a higher level language. What I am saying, is that no language is there yet; no language other than C, C++, and Ada currently provides the three features points I've outlined (low-level control, runtime control, compiler availability). C++ is possible, but you have to jump through a lot of hoops to get where you're going. Ada is great because it provides much more safety, but it has some bloating issues with the binaries it produces (gnat) which wreaks havoc on the caches critical for microkernel performance. So we're left with C, which is not as good as assembler for direct control, but it doesn't really get in our way the way the other two do for most of our purposes. It's unfortunate, but there it is.

It sounds to me increasingly like "microkernel" to you means "a kernel implemented in ANSI C that uses a hand-crafted message passing object system".

Not necessarily C, but any low-level language that gives the programmer direct control over representation and runtime (this is the only real requirement). So why not C++, or BitC?

The defining characteristic of microkernels is a small set of kernel-implemented primitives upon which an entire system can be built in userspace. EROS/CapROS, and Coyotos have only one system call and a few kernel objects for instance.

The biggest problem seems to be that that extra layer of abstraction slows things down (which makes sense really). [...] If heavy usage servers can be run as virtual machines in Xen then why not use a microkernel too?

Funny you should mention Xen, because it's essentially a microkernel running other kernels as protected processes.

So. Any examples of microkernel OS's that handle heavy server load, function well as a desktop, and can handle multimedia tasks like gaming?

Other posts mention QNX, so I won't bother.

I'd find it hard to believe that with solid numbers showing that microkernel is just as fast and without additional overhead that someone like Linus wouldn't use it since it's an easier programming model (better for security, stability, etc).

You'd be surprised. There's a lot of vested interest in the current programming paradigms and existing codebase. A principled microkernel architecture might just be incompatible with POSIX, which eliminates a large swath of portable and useful software.

If you want performance, you need look no further than L4, EROS (and it's successor CapROS). For a principled design, I'd go with EROS/CapROS or the next generation capability system Coyotos (who's designers are trying very hard to implement POSIX while maintaining capability security).

Something useful right now, doesn't exist as far as I know.

The biggest problem seems to be that that extra layer of abstraction slows things down (which makes sense really). [...] If heavy usage servers can be run as virtual machines in Xen then why not use a microkernel too?

Funny you should mention Xen, because it's essentially a microkernel running other kernels as protected processes.

So. Any examples of microkernel OS's that handle heavy server load, function well as a desktop, and can handle multimedia tasks like gaming?

Other posts mention QNX, so I won't bother.

I'd find it hard to believe that with solid numbers showing that microkernel is just as fast and without additional overhead that someone like Linus wouldn't use it since it's an easier programming model (better for security, stability, etc).

You'd be surprised. There's a lot of vested interest in the current programming paradigms and existing codebase. A principled microkernel architecture might just be incompatible with POSIX, which eliminates a large swath of portable and useful software.

If you want performance, you need look no further than L4, EROS (and it's successor CapROS). For a principled design, I'd go with EROS/CapROS or the next generation capability system Coyotos (who's designers are trying very hard to implement POSIX while maintaining capability security).

Something useful right now, doesn't exist as far as I know.

Check out the microkernel work done by the late Jochen Leudtke at L4Ka.org. The HURD is most certainly "not going to be the first to get there," since Mr. Leudtke's research has shown that a Microkernel can indeed reduce context switching cost by aggregating system calls can only making the transition at necessary moments rather than on each individual system call. Further, by using the new (since PII) SYSENTER/SYSEXIT instructions rather than the more traditional interrupt/trap gate, the cost of a context switch can be reduced from several thousand cycles down to approximately 800 or so on modern (P4/AMD64) processors.

Okay, dinner, dishes, bathing kids: done.

Let me clarify a couple points.

"Most use 'intuition.' But this 'intuition' is generally nothing more than familiarity. And familiarity does not fix the current, demonstrable problems. "

The last line should have read "Familiarity is generally just the reuse of old paradigms, which reimplemented the same demonstrable problems."

The benchmarks showing OS X's problems are here. There are several articles on why Mach (and to some extent the original microkernal design) is poor at handling lots of things at once. Just go read up on microkernels and MACH. New microkernels are better at handling this. L4Ka::Pistachio is well known as it was chosen for the resurrected GNU HURD OS.

I would like to reiterate that I was a die in the wool, cut me I will bleed green, yellow, orange, red, purple, and blue IN THAT ORDER, M$ SUCKS, Mac Freakboy. Then I learned how we interact with computers, what our limits and abilities are, how to fix the current problems, and eventually just got pissed every time I use an interface. Which includes locks, cars, dishwashers, ad infinitum.

As a final thought I would like to interject that we design programs knowing the abilities of computers. To a large extent the best software is the one written by those that understand the fundamental things that are going on with every new thing we tackle in programming. Those who design with the fundamentals in mind get good results. Why are we borrowing from MS blindly? Who are known to screw up how they engineer interfaces from the start. We don't do that with any other major part of the OS unless _forced_. To the user the interface IS the operating system, the hardware, to a great extent it is wholly the computer.

You know that x86 processors don't support full virtualization very well? True virtualization software like VMWare does it with a performance hit, which Xen evades. Thus, one can argue that Xen is better option than true virtualization.

And now for something completely different...

People from L4Ka have built a pre-virtualizer, which allows to easily compile virtualizable kernels from unmodified source, which can be then run on Xen or L4Ka microkernel.
Pre-virtualization with Compiler Afterburning

HURD abandoned Mach because of performance issues and is being reimplemented on L4.

If Apple had chosen L4, would it have been necessary from a performance perspective to include BSD at a peer level with the microkernel?

Is it now far too late for Apple to dump Mach?

Mach is painfully slow. It's an old microkernel and it uses async IPC (to allow for passing messages over the network). This is slow because you have to do a ton of context switches and copy the message between address spaces.

L4, on the other hand, uses sync IPC. It has a bunch of neat optimizations, but the main reason why it's faster than Mach is that it doesn't have to copy messages. You send an IPC and it goes into the part of your VM space that L4 sets aside for IPC and then L4 does a quick context switch to the target task, it processes the IPC, and then you get your data back. (Simplifyed a ton).

So, microkernels rock. Mach sucks.

Just a short note on avoiding the problems of drivers relying on API breakage or some given kernel semantics. An alternative approach to supporting such drivers (in source or binary form) is to embed these drivers in a virtual machine. Such an approach incurs a little bit of overhead, sure, but you also get the added benefit that the direver need not necessarily be trusted by the rest of the system.

There's a link on the L4Ka site to a port of the linux kernel to the L4Ka architecture, with a pleasingly higher ratio of the frequency of the words "cvs" to "advanced".

(and what's with "now he has granted an interview..."? is he the pope?)

There's a link on the L4Ka site to a port of the linux kernel to the L4Ka architecture, with a pleasingly higher ratio of the frequency of the words "cvs" to "advanced".

(and what's with "now he has granted an interview..."? is he the pope?)

Well, there are seven actual system calls but a few dozen support routines built on them. The actual system calls are just like, "act on this kernel interface page which contanges message(s) and/or parameter updates and/or other instructions." And also the kernel ABI has more than the API does, IIRC.

Yes, MacOS X is Mach-based. Mach, however, is not really a microkernel in the true sense of the word. Compared to L4's size, Mach is a huge monster. Somebody else already provided a link to an introductory (if old, from 1996) article by the L4 creator Jochen Liedtke.

would it make any sense for Apple to look at L4?

As a matter of fact, the L4KA group is looking into this. See, for example, this thesis currently in progress.

The main benefit is that L4 is actually a true microkernel. It has something like seven system calls that basically handle interprocess communication and address spaces. Everything else needs to be done by user-level processes. L4 separates policy from mechanism, that is, it provides the basic mechanisms and leaves it to the actual OS implementation what policies to implement. It has a very tiny memory footprint and about every optimization possible to operate extremely fast. This is important as in a microkernel environment, much more IPC takes place. Mach sucks here as their IPC operations are terribly slow. L4's IPC speed and its general size show that it's actually feasible to write a real microkernel without taking a non-negligible performance hit. L4Ka::Pistachio is an L4 implementation done completely in C++, which makes code maintenance much nicer, and goes to prove that it is in fact possible to create an efficient microkernel implementation in C++.

It would probably make sense for Apple to look at L4 (as it does for the Hurd), but of course L4 provides much, much less than Mach does (which is good! It's a microkernel after all) and therefore Apple (or, maybe, my friend working on that thesis I mentioned above) would have to reimplement all low-level system services besides address space handling and IPC around L4, in user-level processes. It is probably feasible, but still a long way away.

I'll seriously consider a switch due to architectural issues. Yes, I'm in the Tanenbaum camp. :)

So switch now... Efficient Microkernel Linux on x86

If you believe that a microkernel is a good idea and you want to see that concept come to fruition, then it is obvious that you WOULDN'T work on Linux

Nothing is obvious - those who work on microkernels actively work on linux... and not on HURD!

What this article is about is that the HURD people have ported HURD to L4. But the L4 people had ported Linux to L4 back in 2.2! What's more, linux 2.4 on L4 works, now.

How about a microkernel Linux?

Take the time to read their paper about performance and issues related to linux on L4 - interesting stats like a 5% penalty for L4, as compared to a 25% for Mach...

given the fact that OS X represents the most compelling reason to switch to Apple computers in years, and given the fact that in just a few years the OS has amassed a compartively huge following of developers and applications...

Would it make sense for Apple to now completely rewrite it DOWN TO THE KERNEL LEVEL!!!

Palm OS is on its 4th kernel. Did anyone notice? I didn't. I've been a full-time Palm developer for two years, and I couldn't even tell you which version has which kernel (except that I'm pretty sure they switched kernels when they ditched 68k processors for ARM). Did they have to "completely rewrite it down to the kernel level"? Nope, that's just the point: they did the opposite. They left it the same all the way down to the kernel level; it's just the stuff below the kernel level (and a few minor piece above it) that they changed.

The point is, switching out kernels is not necessarily that tough a thing. Sure, it can't be done overnight, but it doesn't force you to rewrite your entire OS.

Much more to the point, if you research it a little, you'll find that Linux has already been ported to L4Ka. And the version of Linux that was ported still runs exactly the same software as regular Linux. If some small team of researchers can port Linux to L4Ka just to give themselves a convenient development platform, then I guess Apple could do the same thing to OS X if they had any interest in doing so.

And the final question on every /. lips is, in regard to anything, when will HURD run Linux? ;-)


It already does: l4linux.

This article explains the philosphy behind L4, and how it's different from Mach.

But the small size doesn't make most systems faster. Running the same Unix API, L4 adds execution time overhead beyond the default monolithic Linux kernel, about 5%. (Does anyone know the figure for Linux-on-Mach? I know it's much greater than 5%....) However, there are some significant advantages having to do with debugging, maintainability, SMP, real time gaurentees, memory management, configurability, robustness, etc. Detailed discussion here.

From the overview:

Kernels based on the L4 API are second-generation -kernels. They are very lean and feature fast, message-based, synchronous IPC, simple-to-use external paging mechanisms, and a security mechanism based on secure domains (tasks, clans and chiefs). The kernels try to implement only a minimal set of abstractions on which operating systems can be built flexibly.

Other links: L4KA homepage, background info, more info with some historical L3 links.

Frankly, I think L4 is very much the right way to do things. I wish I could say the same for other parts of HURD.

Disclaimer: I work for VMware, but they don't pay me to post on slashdot.

There are a lot of replies of the form, "Wait a minute, Xen requires that you hack up your guests! What a crock! Typical slashdot hype!" It's true that Xen is more limited than VMware's products, in that you do need to modify guests. However, this doesn't mean that Xen is a joke. (Plex86, for instance, really is a joke, because Kevin Lawton seems to pursue it only in fulfillment of an elaborate VMware-centered revenge fantasy.)

The Xen folks, on the other hand, are smart and mostly serious people. Xen, along with appropriately modified guests, solves some of the problems that our products solve, and for those areas where it fits the bill, it does so in a way that should have lasting performance advantages over full x86 virtualization. What Xen is not, in my opinion, is a virtual machine monitor, for any reasonable definition of VMM. Xen is a microkernel. They don't call it that, because it's hard to get papers about microkernels published these days, but if you think about it, the process of porting an OS to run as a guest under Xen isn't cocnceptually distinct from porting it to run as a personality under Mach or Chorus or whatever. The L4 people didn't even bother renaming their microkernel before repurposing it as a paravirtualization platform.

I think the microkernel analogy helps clarify ones thinking about the promises and limitations of so-called "paravirtualization." Hypervisors are microkernels. In the mid-90's, there was a hope that the whole world would be able to settle on the Mach microkernel. It never happened. Anybody hoping to become the only 'para-hypervisor' will face the same political and commercial challenges.

So to recap: Xen is not a replacement for VMware's products. Xen will probably not take over the world to the degree that its creators would like. Xen is not, however, a joke. The Xen researchers are mostly conscientious, smart people who, fairly enough, would like to see their work have some commercial impact. I really wish they'd stop beating their chests over benchmarks that show them beating a three year old version of our desktop product, though.

It's funny you should mention that. The L4 team came up with the same idea. The difference is that OSes require pretty small patches to run on Xen, but much bigger patches to run on a microkernel like L4.

Actually it might be a bit more than a function call...

If operator == is defined outside a class defintion either the right or the left hand side of == could have an implicit conversion performed if there is a single argument, non-explicit-labelled, constructor to go from the type on the left to the type on the right or the right to the left.

Truth is by looking at a line of code you have to know a lot more about the types involved, how they are defined, and other stuff to know what's going on.

No good C++ programmer will tell you that it's easier to account for the impact of a line of C++ vs a line of C.

Don't get me wrong either. C++ kernels can exist and can be VERY fast. Check out L4 Pistachio if you don't believe me. It's also highly cross platform compatible [works on many systems].

They have wisely restricted themselves to a subset of features of the C++ language but have done a great job using the language without going nuts on features. I think it all comes down to discipline.

If I had it my way you wouldn't write kernels in any C-like language at all. I think C is the root of buffer overruns and other security problems. More rigorously type checked languages like Ada95 or even some functional langs like Haskell would make ideal languages for a kernel if they generate good enough code :). I think Pre-Scheme developers would agree with me.

In the meantime people can look at safe C dialects like Cyclone and get similar benefits.

Linus Torvalds has said he's been working on a type checking lint tool for his kernel. Good on him, I say :). That's exactly the right direction for the system to go. I don't think he'd be against a language with stronger typing either if he could get similar guarantees for code generation as C. It almost always comes down to implementation details [of the language].

If you really need tiny kernels, you'll have to abandon Linux and BSD altogether and switch to someting like the L4KA::Pistachio microkernel. Of course, you'll still need userland apps for almost anything, like memory servers, file system servers, device driver servers... That's an awful lot of stuff to cut out of BSD or Linux codebase.

Writing an efficient kernel is much more difficult. L4ka::Pistachio is a good example.

Yes, we can. It's called a microkernel.

True.

The most popular one is Mach

Barf. Not to sound rude, but Mach is a horrid base for an operating system. I'm sorry Apple went with it.

If you mean popular as in "most widely used", then yes, Mach is the most popular "microkernel" (though it doesn't really fit the definition).

Mach is far from the most popular in hacker or academic circles (ie. those who know any better). L4 and EROS are far more suitable hosts for a guest operating system. L4 already has Linux 2.2 and 2.4 running as hosts in fact.

I think it would be more interesting to see (a clone of) the NT and Linux kernels running side-by-side under a nanokernel like L4. I ran Linux in user-space under L4 for a little while a few years ago, but the port wasn't very stable. Some of it may have been that I hacked the default L4 user-space memory manager to support more than 256 MB of RAM. (This has been long fixed in L4.)

Sounds to me like Xen is a microkernel like thingy (what with the hardware abstraction layer and all) which you have to port your operating system personality to. What would make this different than the port of linux to the L4 microkernel (besides the Windows XP part)?

While Linus's opinion on the matter is well known, Microkernels are far from dead. It's just that Mach gave them a bad name. Mach was too bloated and too slow, while the new breed of microkernels have unbelievably fast IPC primitives and therefore the potential to revolutionize the way operating systems are built. Mach sucks != Microkernels suck.

See, for example, the L4 project.