Slashdot Mirror
Hurd/L4 Developer Marcus Brinkmann Interviewed
wikinerd writes "A few years ago when the GNU OS was almost complete, the kernel was the last missing piece, and most distributors combined GNU with the Linux kernel. But the GNU developers continued their efforts and unveiled the Hurd in 1990s, which is currently a functioning prototype. After the Mach microkernel was considered insufficient, some developers decided to start a new project porting the Hurd on the more advanced L4 microkernel using cutting-edge operating system design, thus creating the Hurd/L4. Last February one of the main developers, Marcus Brinkmann, completed the process initialization code and showed a screenshot of the first program executed on Hurd/L4 saying 'The dinner is prepared!' Now he has granted an interview about Hurd/L4, explaining the advantages of microkernels, the Hurd/L4 architecture, the project's goals and how he started the Debian port to Hurd."
So your theory is that Microsoft, Apple and Sun are all united together behind a single vision? Sorry to break it to you, but they're "fragmented" too. There's no more reason for all free software developers to be working on the same project than there is for all proprietary software developers to do so.
GNU made most of the core programs that Linux normally uses, and they are universally considered excellent. So why is it so hard for them to make a kernel?
Is it just loss of interest after Linux became popular?
# cat
Damn, my RAM is full of llamas.
You just don't see this in the proprietary companies. Sure, they compete with each other, but within the companies themselves there's much tighter integration.
Rubbish. A proprietary company with the number of employees equal to the number of OSS developers doesn't exist, but there are smaller but still huge ones, such as IBM and Microsoft, in which the infighting is, uh, legendary.
Zot. Buh-bye Hurd.
Actually, the free developers are the one who focus like a laser on what they want to develop. At a Big Dumb Company(tm) the developers may not focus as sharply as the "decision makers". Here, the decisions are made by the developers and hence there is better focus on the goals. If it were universally agreed what the goal should be, everyone would focus. Since it's not a given, people will latch onto things others may think are unnecessary. My own experience shows that most people will not be convinced that an alternative is better until you show them. That means those interested in the Hurd must build it and show the rest of you. Only then should we decide.
You can't get 'too many' open source kernels. All right, the HURD is old, and development is slow. But at least it is another choice.
Consider the alternative if GNU project wasn't started by RMS... even Linux wouldn't have been around...
While we're on the topic of microkernels, wouldn't it be a good idea to gradually make the linux kernel less monolithic, finally turning it into a nifty microkernel based OS? Is there anything going on in this direction?
i see many opst here and i wonder if y'all know that HURD has a key feature, namely:
>>it should become possible to replace a running kernel
in other words NEVER REBOOT AGAIN!
in practise this is still hard to accomplish: but at least people are working on it. and yes, to implement this takes time.
I think this hits the nail on the head. That's why I have enjoyed tinkering with Hurd over the years. I currently have a bootable Debian/Hurd partition, and I have recently built the L4/Hurd system up to its current state. I haven't been able to get banner running like Marcus did, but its not for a lack of trying.
Many Slashdotters will say "Why waste your time with Hurd because BSD/Linux/Windows/OSX/etc already works great and needs more contributors?" Well, its my time and if I want to play around with experimental source code then that's what I am going to do.
I already have a nicely working Gentoo Linux system that I use most of the time, and I'm happy with it. However, I am one of those types that wants to always learn, and by following the progress of Mach/Hurd and now L4/Hurd I get to grow up with the operating system and there is a small chance that I will be able to make a useful contribution here or there occasionally.
Hurd isn't trying to sell itself to become a replacement for your current favorite operating system. It is simply a project to create an OS based on advanced and sometimes theoretical computer science ideas.
People like Marcus put a lot of effort into realizing these abstractions in code. Sometimes it doesn't work out and they have to backstep, but progress continues. I have been on the developer's mailing list for years, and honestly I don't understand 90% of what they are talking about but it is pretty interesting nonetheless.
Hurd makes pretty heavy use of GNU autotools; i.e. "./configure" and a lot of the real benefits of the old Mach based Debian/Hurd are that upstream sources have been patched so that you can hopefully build them on Hurd just by running configure and make. L4-Hurd is still Hurd, so all the work done is still relevent. When they whip the sheet off of the shiny new engine, the rest of the parts waiting on the shelf are already there.
And they are making good progress. They have it now to the point where a lot of people are working on getting libc to build and once the kernel and libc are working that is the keystone that lets all the other peices of the puzzle come together.
It's totally a research project. There is no agenda other than some people like Marcus thought it was a cool project and decided to fool around with it. I'm the same way, sometimes I get bored with Linux so I put on my Hurd cap and play around with it for a while.
Clickety Click
From: ast@cs.vu.nl (Andy Tanenbaum)
Subject: Re: LINUX is obsolete
"I still maintain the point that designing a monolithic kernel in 1991 is a fundamental error. Be thankful you are not my student. You would not get a high grade for such a design"
To Linus Torvalds.
Linux is slowly moving that direction, in a natural-selection sort of way. Initramfs and klibc ("early userspace") will allow things like root-on-nfs and in-kernel DHCP (which arguably shouldn't have been in the kernel in the first place) to move to userspace, but developers are already beginning to talk about using it to put things like partition table parsing and console terminal support into userspace as well. I'd guess that more people will start to see applications for early userspace once it's "complete" and distributions start using it (initramfs is already in the kernel; it's the anomalous rootfs that shows up in /proc/mounts).
Filesystems and the network stack are never going to move to userspace, though; Linus is dead-set against it, for the same reasons he always has been. If you were completely batshit, you could probably convince FUSE to run from initramfs, and get wholly userspace filesystems that way, but FUSE doesn't support any non-network filesystems, AFAIK.
"That's all I have to say about that" --Forrest Gump
For those with Non German Language Disorder here a rough translation:
"Dilettants! Dilettants! - so are those called, who are doing a science or an art, out of love or joy for it, per il loco diletto, with disrespect by those, who do it for the gain, because they like the money that can be earned by it. This disrespect is founded in the mean conviction, that no one seriously does anything if not forced by misery, hunger or another greed. The public has the same mind and thus the same opinion: from here comes the full respect for "people of the craft" and his mistrust for dilettants. In reality for the dilettant the thing is the end, for the craftsman it is just means; only he will do a thing with full seriosity, who is immediately interested, and who is occupied by it out of love, does it con Amore. Those, not the paid servants, always started the greatest things."
VM is really a hypervisor, or "virtual machine monitor". The abstraction it offers to the application looks like the bare machine. So you have to run another OS under VM to get useful services.
The PC analog of VM is VMware. VMware is much bigger and uglier than VM because x86 hardware doesn't virtualize properly, and horrible hacks, including code scanning and patching, have to be employed to make VMware work at all. IBM mainframe hardware has "channels", which support protected-mode I/O. So drivers in user space work quite well.
QNX is a widely used embedded operating system. It's most commonly run on small machines like the ARM, but it works quite nicely on x86. You can run QNX on a desktop; it runs Firefox and all the GNU command-line tools.
QNX got interprocess communication right. Most academic microkernels, including Mach and L4, get it wrong. The key concept can be summed up in one phrase - "What you want is a subroutine call. What the OS usually gives you is an I/O operation". QNX has an interprocess communication primitive, "MsgSend", which works like a subroutine call - you pass a structure in, wait for the other process to respond, and you get another structure back. This makes interprocess communication not only convenient, but fast.
The performance advantage comes because a single call does the necessary send, block, and call other process operations. But the issue isn't overhead. It's CPU scheduling. If the receiving process is waiting (blocked at a "MsgReceive"), control transfers immediately to the receiving process. There's no ambiguity over whether the message is complete, as there is with pipe/socket type IPC. There's no trip through the scheduler looking for a process to run. And, most important, there's no loss of scheduling quantum.
This last is subtle, but crucial. It's why interprocess communication on UNIX, Linux, etc. loses responsiveness on heavily loaded systems. The basic trouble with one-way IPC mechanisms, which include those of System V and Mach, is that sending creates an ambiguity about who runs next.
When you send a System V type IPC message (which is what Linux offers), the sender keeps on running and the receiving process is unblocked. Shortly thereafter, the sending process usually does an IPC receive to get a reply back, and finally blocks. This seems reasonable enough. But it kills performance, because it leads to bad scheduling decisions.
The problem is that the sending process keeps going after the send, and runs until it blocks. At this point, the kernel has to take a trip through the scheduler to find which thread to run next. If you're in a compute-bound situation, and there are several ready threads, one of them starts up. Probably not the one that just received a message, because it's just become ready to run and isn't at the head of the queue yet. So each IPC causes the process to lose its turn and go to the back of the queue. So each interprocess operation carries a big latency penalty.
This is the source of the usual observation that "IPC under Linux is slow". It's not that the implementation is slow, it's that the ill-chosen primitives have terrible scheduling properties. This is an absolutely crucial decision in the design of a microkernel. If it's botched, the design never recovers. Mach botched it, and was never able to fix it. L4 isn't that great in this department, either.
Sockets and pipes are even worse, because you not only have the scheduling problem, you have the problem of determining when a message is complete and it's time to transfer control. The best the kernel can do is guess.
QNX is a good system technically. The main problem with QNX, from the small user perspective, is the company's marketing operation, which is dominated by inside sales people who want to cut big
"I think this has the tendency to make OSS be sort of the breeding ground for the real innovations in tech, but largely unable to provide the sort of polish that proprietary companies can."
This seems to be a popular theory here on slashdot. The idea that the reason linux hasn't broken through is because it lacks polish.
History shows us that a more polished desktop does not always win. The Mac is a perfect example of this. During the 80s the mac was a slick polished interface while the PC had an ugly DOS command line. The PC won handily. Why is that?
I suspect because it was more "open". You could stick cards in it, you could expand it, you could hack it and most importantly it had lotus 123 running on it so the business community embraced it.
FOr the exact same reasons I predict linux continues to advance into the desktop. The final breaking point will come when businesses see competitive advantage in it. Once that happens I expect explosive growth in linux adoption and yes world domination.
evil is as evil does
Others have already pointed out how valuable research is, so I'll ignore that for now.
Hurd is also attempting to solve very real practical problems. Consider a typical UNIX network daemon:
(1) Must be started as root to listen on a privileged port
(2) Upon an incoming connection, must accept and then "drop privileges"
This causes mnay, MANY problems, that are very practical. If you took all the remote root security exploits ever in UNIX, and you subtracted those that involved a network daemon that needed to be run as root to listen on a privileged port, you'd be left with a rather secure system.
I just can't imagine why nobody recognizes a problem like that. You don't inherently need to run Apache/BIND/Sendmail with the privilege to overwrite the boot sector, but people ignore it as if "Oh well, it's a network daemon, of course it needs to be able to rewrite the boot sector. We'll just hope there are no bugs.".
Not only is this a security nightmare (which is only mitigated by the fact that UNIX is compared to windows rather than an ideal), but it's also got many performance implications. If you're measuring raw performance of already-written applications, a monolithic kernel will never be worse than a microkernel architecture. However, on a linux system, a lot of resources are traded in order to jump through hoops that don't need to be there, particularly for security. Maybe you don't really need to start that new process, and you can do everything you need in the current process. Sounds like a serious performance win to me, and not "my kernel avoids that 0.1% performance penalty you have to take on operation X".
For example, what about Apache, CGI and suexec? You really don't need all that when you could just be getting/releasing authentication tokens and using an apache module rather than starting a new process just so you can change privileges.
You can't separate the details of performance characteristics from capabilities. Capabilities may cost overhead, but may reduce algorithmic requirements.
Social scientists are inspired by theories; scientists are humbled by facts.