Slashdot Mirror
Hurd/L4 Developer Marcus Brinkmann Interviewed
wikinerd writes "A few years ago when the GNU OS was almost complete, the kernel was the last missing piece, and most distributors combined GNU with the Linux kernel. But the GNU developers continued their efforts and unveiled the Hurd in 1990s, which is currently a functioning prototype. After the Mach microkernel was considered insufficient, some developers decided to start a new project porting the Hurd on the more advanced L4 microkernel using cutting-edge operating system design, thus creating the Hurd/L4. Last February one of the main developers, Marcus Brinkmann, completed the process initialization code and showed a screenshot of the first program executed on Hurd/L4 saying 'The dinner is prepared!' Now he has granted an interview about Hurd/L4, explaining the advantages of microkernels, the Hurd/L4 architecture, the project's goals and how he started the Debian port to Hurd."
Now they can begin porting Duke Nukem: Forever!
Send email from the afterlife! Write your e-will at Dead Man's Switch.
That's the difference between OSS and proprietary companies. They can focus like a laser on what they want to develop and leave a lot of the infrastructural heavy lifting to those hippy anarchists in the open source scene.
It's win-win for them, because they get the benefit of a lot of what these groups produce, and often can improve upon it (BSD --> OSX). It's like having an unpaid R&D dept. working for you 24/7.
Hi Marcus,
How many people do you think will submit questions thinking that this is a Slashdot interview?
GNU made most of the core programs that Linux normally uses, and they are universally considered excellent. So why is it so hard for them to make a kernel?
Is it just loss of interest after Linux became popular?
# cat
Damn, my RAM is full of llamas.
This was in 1991...
The AACS key is NOT 0xF606EEFD628B1CA427BEA93A9CA9773F
The kernel is the last missing piece? What's the first piece, an integrated browser?
Damn, wrong button!
Get a free iPod Nano 4GB!
http://dilse.net/hurd.html
What a waste of time. What are they trying to accomplish by still working on the HURD?
Be fair: let us all know what you do in your spare time so we can sneer at you too.
The real Ralph Yarro posts as Anonymous Coward. Anyone else is an impostor.
Actually, the free developers are the one who focus like a laser on what they want to develop. At a Big Dumb Company(tm) the developers may not focus as sharply as the "decision makers". Here, the decisions are made by the developers and hence there is better focus on the goals. If it were universally agreed what the goal should be, everyone would focus. Since it's not a given, people will latch onto things others may think are unnecessary. My own experience shows that most people will not be convinced that an alternative is better until you show them. That means those interested in the Hurd must build it and show the rest of you. Only then should we decide.
You can't get 'too many' open source kernels. All right, the HURD is old, and development is slow. But at least it is another choice.
Consider the alternative if GNU project wasn't started by RMS... even Linux wouldn't have been around...
It's theoretically going to be much better. Revolutionarily so in fact. This is a deliberate design decision since the rise of linux: loads of experimental, potential "next big thing" ideas are going into hurd. Linux works, so hurd is trying to make something that's much better, rather than slapping together a working kernel and worrying about the architecture later which is what linux does.
I am trolling
Think of it more like OS research. Lessons learned from HURD can be applied to *BSD and Linux. Some already have.
Almost everyone who has started with a "pure" microkernel design has eventually moved away from it for performance and other reasons.
Many of the problems HURD was trying to address have either become irrelevant, determined to be non-issues, or have been solved. (The same can be said of things like IA64 or SPARC).
There are still interesting ideas in HURD that deserve research. However if they intend to challenge the enormous momentum behind *BSD and Linux, HURD is going to have to offer some truly astounding functionality / killer app or few people are going to use it.
What a waste of time. What are they trying to accomplish by still working on the HURD? Linux has already far surpassed it in every catagory (hardware support, software support, usability, performance, etc.) and is just as Free as the HURD, so what gives?
On the other hand, I guess I'm not the only one of this mind, as it obviously wouldn't have taken 20 years to get to the point where a program can finally run on it if everybody else with development skills didn't also believe it a total waste of their time.
From the interview:
Security and stability are tightly related issues, and they are major motivations for any microkernel based system. However, we feel that security does not need to translate to loss of freedom. With a bit of extra trouble, you can be secure and even increase the freedom of the user. This is what we want to do.
In the Hurd, the operating system is implemented as a set of servers, and each runs in its own address space. Of course there are some essential system services which better not crash, or the system will reboot immediately as a last attempt to salvage the situation. But for many other services, a crash is not fatal. If a filesystem server crashes (except for the root filesystem), you can just restart it (or it is restarted automatically by the system). Dead-locks require manual interaction, and you will have to kill the hanging server to remove it from the system and release associated resources.
The Hurd achieves its stability and security by protocols between components that require no mutual trust. So, although a user can add their own filesystem to the filesystem hierarchy, and the parent filesystem will redirect accesses through such a mount point to the user's filesystem, there is nothing the user's filesystem can do that can affect the rest of the system in a bad way. The Hurd servers are written in a way to assume the worst from a communication partner, namely that it is malicious, as an implication you get fault-tolerance for free.
"It's theoretically going to be much better."
This is why so many projects fail. They try too hard to create a mythical overdesigned piece of software that "works in theory" rather than create something that works and then improve it from there.
2. "unveiled the Hurd in 1990s, which is currently a functioning prototype" -- so, ~15 years (fifteen years!) later, it's still a "functioning" prototype!?
Operating systems develop slowly in their core design and philosophy, and that's no bad thing.
Linux literally exploded into existence, but its design is 80% identical to that of original Unixes so it enjoyed the immense benefits of inherent architectural stability. Linux knows where it's going, but the horizons surrounding the Hurd are very fuzzy indeed. It will take time.
While we're on the topic of microkernels, wouldn't it be a good idea to gradually make the linux kernel less monolithic, finally turning it into a nifty microkernel based OS? Is there anything going on in this direction?
Could you please stop interviewing him and let Marcus get back to work? If you keep interviewing him, we're never going to see Hurd in a usable state.
It's not offtopic, dumbass. It's orthogonal.
This is why so many projects fail. They try too hard to create a mythical overdesigned piece of software that "works in theory" rather than create something that works and then improve it from there.
Ah, the grand old dispute between academics and business people. Linux wouldn't have existed without some quite theoretic early approaches (Babbage, Turing, von Neumann). After all - who cared about the early computers like Babbage's analytical engine? An abacus would be much faster for all relevant computations!
Without theoretic deep-dives like HURD, we would never know if microkernels are a possibility or not. Because it hasn't been proved to work yet. The HURD theorists suggest that microkernels are better than monolithic kernels. Let them explore it, make prototypes, and then someone will make a working kernel to play Duke Nukem Forever on.
Remember: a lot of research just discovers uselessness. It is important to know what's useless, so no-one have to take that path.
Roses are #FF0000, violets are #0000FF, all my base are belong to you
i see many opst here and i wonder if y'all know that HURD has a key feature, namely:
>>it should become possible to replace a running kernel
in other words NEVER REBOOT AGAIN!
in practise this is still hard to accomplish: but at least people are working on it. and yes, to implement this takes time.
GNU/Hurd is still developed, because it has interesting capabilities, capabilities Linux will never have. IMHO, Hurd is the kernel of the future.
Btw, Hurd will be compatible with a lot of linux' device drivers (i heard all linux 2.0 device drivers were ported, but I could be mistaken) , and imho, device drivers aren't the main job of kernel developers, but of hardware makers. That is a bit unrealistic at this time, though, but it should be.
Many of the posts above say Hurd is a waste of time. I suspect the Hurd team just enjoys hacking. I really don't think they care if its a "waste" of time. They just love what they do. I think it's awesome to be so dedicated to your craft. Even if the Hurd never works... I bet they will still look back on the whole experience as something pretty cool.
My own personal experience: I worked on an 8 month student project that in many ways failed in the end. But I would never consider that a waste of time. I learned so much and had a blast doing it.
-bdb
There's no infighting where you work?
Who do you work for and where can I send my resume?
I rarely criticize things I don't care about.
I installed linux on my machine in late 1991 (I think it was version 0.11). At the time, 386bsd wasn't officially out yet; I think it came out in eary-mid 1992. Linux development was pretty rapid, especially during the first year, and 386bsd development was pretty slow.
I think this is the real reason why linux became so popular. By the time netbsd started getting off the ground, there was already a pretty large linux user base. If the timing and development pace of 386bsd had been different, things may have been different today.
(S(SKK)(SKK))(S(SKK)(SKK))
From: ast@cs.vu.nl (Andy Tanenbaum)
Subject: Re: LINUX is obsolete
"I still maintain the point that designing a monolithic kernel in 1991 is a fundamental error. Be thankful you are not my student. You would not get a high grade for such a design"
To Linus Torvalds.
"I have no idea why Linux became more popular in the first place, considering that there was already BSD and the HURD,"
BSD wasn't ready for 386 at the time, and had the AT&T lawsuit hanging over it. And with Hurd not ready to go now, what makes you think it was ready to go in the early 90s?
I rarely criticize things I don't care about.
I strongly agree with the OP. Even if HURD never competes with Linux, it's still important as a research project.
"A language that doesn't affect the way you think about programming, is not worth knowing" - Alan Perlis
Linux is slowly moving that direction, in a natural-selection sort of way. Initramfs and klibc ("early userspace") will allow things like root-on-nfs and in-kernel DHCP (which arguably shouldn't have been in the kernel in the first place) to move to userspace, but developers are already beginning to talk about using it to put things like partition table parsing and console terminal support into userspace as well. I'd guess that more people will start to see applications for early userspace once it's "complete" and distributions start using it (initramfs is already in the kernel; it's the anomalous rootfs that shows up in /proc/mounts).
Filesystems and the network stack are never going to move to userspace, though; Linus is dead-set against it, for the same reasons he always has been. If you were completely batshit, you could probably convince FUSE to run from initramfs, and get wholly userspace filesystems that way, but FUSE doesn't support any non-network filesystems, AFAIK.
"That's all I have to say about that" --Forrest Gump
You can try Hurd/L4 right now by burning a bootable CD (iso) using the Gnuppix.
You can read the interview through its Coral cache or its MirrorDot cache . There is also a Google cache.
There is also a MirrorDot-cached PDF version of the interview that can be downloaded by clicking here.
Thanks
having adminned variously HP-UX, AIX, Irix, and Solaris boxes, one of the first things I did on a machine was install the gnu toolset. The proprietary stuff was years behind (Solaris was probably the worst) and getting almost anything modern to compile with them was a real bitch.
For those with Non German Language Disorder here a rough translation:
"Dilettants! Dilettants! - so are those called, who are doing a science or an art, out of love or joy for it, per il loco diletto, with disrespect by those, who do it for the gain, because they like the money that can be earned by it. This disrespect is founded in the mean conviction, that no one seriously does anything if not forced by misery, hunger or another greed. The public has the same mind and thus the same opinion: from here comes the full respect for "people of the craft" and his mistrust for dilettants. In reality for the dilettant the thing is the end, for the craftsman it is just means; only he will do a thing with full seriosity, who is immediately interested, and who is occupied by it out of love, does it con Amore. Those, not the paid servants, always started the greatest things."
VM is really a hypervisor, or "virtual machine monitor". The abstraction it offers to the application looks like the bare machine. So you have to run another OS under VM to get useful services.
The PC analog of VM is VMware. VMware is much bigger and uglier than VM because x86 hardware doesn't virtualize properly, and horrible hacks, including code scanning and patching, have to be employed to make VMware work at all. IBM mainframe hardware has "channels", which support protected-mode I/O. So drivers in user space work quite well.
QNX is a widely used embedded operating system. It's most commonly run on small machines like the ARM, but it works quite nicely on x86. You can run QNX on a desktop; it runs Firefox and all the GNU command-line tools.
QNX got interprocess communication right. Most academic microkernels, including Mach and L4, get it wrong. The key concept can be summed up in one phrase - "What you want is a subroutine call. What the OS usually gives you is an I/O operation". QNX has an interprocess communication primitive, "MsgSend", which works like a subroutine call - you pass a structure in, wait for the other process to respond, and you get another structure back. This makes interprocess communication not only convenient, but fast.
The performance advantage comes because a single call does the necessary send, block, and call other process operations. But the issue isn't overhead. It's CPU scheduling. If the receiving process is waiting (blocked at a "MsgReceive"), control transfers immediately to the receiving process. There's no ambiguity over whether the message is complete, as there is with pipe/socket type IPC. There's no trip through the scheduler looking for a process to run. And, most important, there's no loss of scheduling quantum.
This last is subtle, but crucial. It's why interprocess communication on UNIX, Linux, etc. loses responsiveness on heavily loaded systems. The basic trouble with one-way IPC mechanisms, which include those of System V and Mach, is that sending creates an ambiguity about who runs next.
When you send a System V type IPC message (which is what Linux offers), the sender keeps on running and the receiving process is unblocked. Shortly thereafter, the sending process usually does an IPC receive to get a reply back, and finally blocks. This seems reasonable enough. But it kills performance, because it leads to bad scheduling decisions.
The problem is that the sending process keeps going after the send, and runs until it blocks. At this point, the kernel has to take a trip through the scheduler to find which thread to run next. If you're in a compute-bound situation, and there are several ready threads, one of them starts up. Probably not the one that just received a message, because it's just become ready to run and isn't at the head of the queue yet. So each IPC causes the process to lose its turn and go to the back of the queue. So each interprocess operation carries a big latency penalty.
This is the source of the usual observation that "IPC under Linux is slow". It's not that the implementation is slow, it's that the ill-chosen primitives have terrible scheduling properties. This is an absolutely crucial decision in the design of a microkernel. If it's botched, the design never recovers. Mach botched it, and was never able to fix it. L4 isn't that great in this department, either.
Sockets and pipes are even worse, because you not only have the scheduling problem, you have the problem of determining when a message is complete and it's time to transfer control. The best the kernel can do is guess.
QNX is a good system technically. The main problem with QNX, from the small user perspective, is the company's marketing operation, which is dominated by inside sales people who want to cut big
GNU's not unix, its an inferior work-alike that serves no purpose.
Serves no purpose? That's funny. Currently, it serves a lot of my needs. I believe that the purpose of the tools is to serve user's needs, but I might be mistaken. Enlighten me.
Roses are #FF0000, violets are #0000FF, all my base are belong to you
There's a spectrum of issues in the computing world, that range from computer science, which involves doing things where you don't know what the algorithms are to do what you want and have to invent them, to software engineering, which is building something which is extremely well known and understood.
Open source projects are on the whole better, or at least achieve success more quickly on software engineering problems than computer science problems. Writing a word processor is a software engineering problem. It's not like the concepts are not well understood and well documented all over the place, the trick is just building a solid and reliable instance of it. Because it's such a simple and well known problem you can bring in dozens to hundreds of programmers to work on it and there are few debates about how to do things, it's usually more an issue of prioritizing feature lists and bug fixes.
Linux is a software engineering project in the classic sense. Linus and others have been rebuilding Unix, which is extremely well understood. Everyone more or less understood and agreed on how Unix systems work. It's not a question of how to build a memory management algorithm with acceptable performance, but rather which existing algorithm has the best peformance. In general, Linux tends to spend more time debating between existing solutions than trying to find a solution to a problem. The reason that Linux has come so far so fast is that it's treading on extremely familiar ground and isn't really trying to do anything new from a computer science level.
Hurd is more in the area of computer science. They don't have thirty years of precedence going in their favor. While there has been plenty of work in microkernels, there's far less of work there than for Unix. The Hurd people are trying to make something new, rather than reinvent something that's familiar, which is a much easier task by far. So the fact that the Hurd people are moving more slowly is more an indication of the difficulty of the task.
Now the question is, why work on Hurd at all? Well, the answer to that is the answer to the question of whether or not there are things with a microkernel that you cannot do with a regular kernel, and whether these things are worth doing. It is entirely possible on a security level for Linux to hit a dead end, running into the limits of a monolithic kernel architecture. That if there is to be any progress past a certain point, that a rearchitecture is needed to switch to a microkernel architecture. I'm not saying this is the case, but I am saying that it is not an impossibility.
If that is the case, then Linus and others will need to do a major rearchitecture in a new release, or they need to switch over to an existing microkernel project that they feel is acceptable to them. Even if the Linux people decide to do their own microkernel architecture from scratch in that case, they will almost certainly be going over the entire history and the results of the GNU/Hurd project with a fine tooth comb for data on how to build a viable microkernel operating system.
To say that microkernels are slower than monolithic kernels is on some level unimportant. CPU speeds have slowed down somewhat but we're still improving the speed of systems. The question becomes are you willing to trade a performance hit for security. Would you rather have a fast system that is more vulnerable to nasty software or would you rather have a slower but more secure system? So the Hurd people are focusing on security since that is potentially the greatest strength of microkernels over monolithic kernels.
So look at the Hurd project, like a bunch of other projects as a research project. And yes, it's taking them a heck of a long time to get results but they're not in any particular hurry. It's like Linux versus Windows, Linux doesn't need to "win" next year. It just keeps on chugging and eventually grinds away at the opposition. Hurd just keeps getting better every year and maybe someday it will clearly surpass Linux in a few areas. Probably not anytime soon, but this isn't a race.
So no, Hurd isn't a waste of time. It's a research project and one that may be of significant importance to Linux down the road.
Others have already pointed out how valuable research is, so I'll ignore that for now.
Hurd is also attempting to solve very real practical problems. Consider a typical UNIX network daemon:
(1) Must be started as root to listen on a privileged port
(2) Upon an incoming connection, must accept and then "drop privileges"
This causes mnay, MANY problems, that are very practical. If you took all the remote root security exploits ever in UNIX, and you subtracted those that involved a network daemon that needed to be run as root to listen on a privileged port, you'd be left with a rather secure system.
I just can't imagine why nobody recognizes a problem like that. You don't inherently need to run Apache/BIND/Sendmail with the privilege to overwrite the boot sector, but people ignore it as if "Oh well, it's a network daemon, of course it needs to be able to rewrite the boot sector. We'll just hope there are no bugs.".
Not only is this a security nightmare (which is only mitigated by the fact that UNIX is compared to windows rather than an ideal), but it's also got many performance implications. If you're measuring raw performance of already-written applications, a monolithic kernel will never be worse than a microkernel architecture. However, on a linux system, a lot of resources are traded in order to jump through hoops that don't need to be there, particularly for security. Maybe you don't really need to start that new process, and you can do everything you need in the current process. Sounds like a serious performance win to me, and not "my kernel avoids that 0.1% performance penalty you have to take on operation X".
For example, what about Apache, CGI and suexec? You really don't need all that when you could just be getting/releasing authentication tokens and using an apache module rather than starting a new process just so you can change privileges.
You can't separate the details of performance characteristics from capabilities. Capabilities may cost overhead, but may reduce algorithmic requirements.
Social scientists are inspired by theories; scientists are humbled by facts.
This is a classic logic fallacy in that you pick and choose the factors that support your argument. The PC didn't win because it was more "open." It won because it was cheaper than the $2000+ priced Macintosh, fueled by commodity PC-clones (remember the phrase "PC-compatible"?) that competed with each other and brought prices down each year.