Slashdot Mirror
Hurd/L4 Developer Marcus Brinkmann Interviewed
wikinerd writes "A few years ago when the GNU OS was almost complete, the kernel was the last missing piece, and most distributors combined GNU with the Linux kernel. But the GNU developers continued their efforts and unveiled the Hurd in 1990s, which is currently a functioning prototype. After the Mach microkernel was considered insufficient, some developers decided to start a new project porting the Hurd on the more advanced L4 microkernel using cutting-edge operating system design, thus creating the Hurd/L4. Last February one of the main developers, Marcus Brinkmann, completed the process initialization code and showed a screenshot of the first program executed on Hurd/L4 saying 'The dinner is prepared!' Now he has granted an interview about Hurd/L4, explaining the advantages of microkernels, the Hurd/L4 architecture, the project's goals and how he started the Debian port to Hurd."
GNU made most of the core programs that Linux normally uses, and they are universally considered excellent. So why is it so hard for them to make a kernel?
Is it just loss of interest after Linux became popular?
# cat
Damn, my RAM is full of llamas.
I think this hits the nail on the head. That's why I have enjoyed tinkering with Hurd over the years. I currently have a bootable Debian/Hurd partition, and I have recently built the L4/Hurd system up to its current state. I haven't been able to get banner running like Marcus did, but its not for a lack of trying.
Many Slashdotters will say "Why waste your time with Hurd because BSD/Linux/Windows/OSX/etc already works great and needs more contributors?" Well, its my time and if I want to play around with experimental source code then that's what I am going to do.
I already have a nicely working Gentoo Linux system that I use most of the time, and I'm happy with it. However, I am one of those types that wants to always learn, and by following the progress of Mach/Hurd and now L4/Hurd I get to grow up with the operating system and there is a small chance that I will be able to make a useful contribution here or there occasionally.
Hurd isn't trying to sell itself to become a replacement for your current favorite operating system. It is simply a project to create an OS based on advanced and sometimes theoretical computer science ideas.
People like Marcus put a lot of effort into realizing these abstractions in code. Sometimes it doesn't work out and they have to backstep, but progress continues. I have been on the developer's mailing list for years, and honestly I don't understand 90% of what they are talking about but it is pretty interesting nonetheless.
Hurd makes pretty heavy use of GNU autotools; i.e. "./configure" and a lot of the real benefits of the old Mach based Debian/Hurd are that upstream sources have been patched so that you can hopefully build them on Hurd just by running configure and make. L4-Hurd is still Hurd, so all the work done is still relevent. When they whip the sheet off of the shiny new engine, the rest of the parts waiting on the shelf are already there.
And they are making good progress. They have it now to the point where a lot of people are working on getting libc to build and once the kernel and libc are working that is the keystone that lets all the other peices of the puzzle come together.
It's totally a research project. There is no agenda other than some people like Marcus thought it was a cool project and decided to fool around with it. I'm the same way, sometimes I get bored with Linux so I put on my Hurd cap and play around with it for a while.
Clickety Click
VM is really a hypervisor, or "virtual machine monitor". The abstraction it offers to the application looks like the bare machine. So you have to run another OS under VM to get useful services.
The PC analog of VM is VMware. VMware is much bigger and uglier than VM because x86 hardware doesn't virtualize properly, and horrible hacks, including code scanning and patching, have to be employed to make VMware work at all. IBM mainframe hardware has "channels", which support protected-mode I/O. So drivers in user space work quite well.
QNX is a widely used embedded operating system. It's most commonly run on small machines like the ARM, but it works quite nicely on x86. You can run QNX on a desktop; it runs Firefox and all the GNU command-line tools.
QNX got interprocess communication right. Most academic microkernels, including Mach and L4, get it wrong. The key concept can be summed up in one phrase - "What you want is a subroutine call. What the OS usually gives you is an I/O operation". QNX has an interprocess communication primitive, "MsgSend", which works like a subroutine call - you pass a structure in, wait for the other process to respond, and you get another structure back. This makes interprocess communication not only convenient, but fast.
The performance advantage comes because a single call does the necessary send, block, and call other process operations. But the issue isn't overhead. It's CPU scheduling. If the receiving process is waiting (blocked at a "MsgReceive"), control transfers immediately to the receiving process. There's no ambiguity over whether the message is complete, as there is with pipe/socket type IPC. There's no trip through the scheduler looking for a process to run. And, most important, there's no loss of scheduling quantum.
This last is subtle, but crucial. It's why interprocess communication on UNIX, Linux, etc. loses responsiveness on heavily loaded systems. The basic trouble with one-way IPC mechanisms, which include those of System V and Mach, is that sending creates an ambiguity about who runs next.
When you send a System V type IPC message (which is what Linux offers), the sender keeps on running and the receiving process is unblocked. Shortly thereafter, the sending process usually does an IPC receive to get a reply back, and finally blocks. This seems reasonable enough. But it kills performance, because it leads to bad scheduling decisions.
The problem is that the sending process keeps going after the send, and runs until it blocks. At this point, the kernel has to take a trip through the scheduler to find which thread to run next. If you're in a compute-bound situation, and there are several ready threads, one of them starts up. Probably not the one that just received a message, because it's just become ready to run and isn't at the head of the queue yet. So each IPC causes the process to lose its turn and go to the back of the queue. So each interprocess operation carries a big latency penalty.
This is the source of the usual observation that "IPC under Linux is slow". It's not that the implementation is slow, it's that the ill-chosen primitives have terrible scheduling properties. This is an absolutely crucial decision in the design of a microkernel. If it's botched, the design never recovers. Mach botched it, and was never able to fix it. L4 isn't that great in this department, either.
Sockets and pipes are even worse, because you not only have the scheduling problem, you have the problem of determining when a message is complete and it's time to transfer control. The best the kernel can do is guess.
QNX is a good system technically. The main problem with QNX, from the small user perspective, is the company's marketing operation, which is dominated by inside sales people who want to cut big