Slashdot Mirror
KernelTrap Talks WIth GNU/Hurd Developer Neal Walfield
An Anonymous Coward writes: "One of the GNU/Hurd developers, Neal Walfield, was recently interviewed by KernelTrap. Nice read."
← Back to Stories (view on slashdot.org)
← Back to Stories (view on slashdot.org)
Read the article. Hurd is not a kernel. It is a set of servers.
- The early worm gets eaten by the bird.
My employer likes to stay on the leading edge in the operating systems field, and makes it a point to try to integrate up-and-coming technologies into our server farms. It should come as no surprise, then, that our team does use a HURD machine as a file/web/application server.
The HURD machine has been surprisingly stable since we set it up last year. We may have had a few instances where it would get into an undesirable state and need rebooting, but by and large its downtime has been attributable to hardware upgrades and power interruptions. Its integrated userspace/kernel space has provoked us to write some very interesting programs on that box that we would not have been able to create with an ordinary UNIX or clone.
What's interesting about the HURD is that, despite its departures from many UNIX conventions, its developers are striving to form a clean upgrade path from Linux to HURD. Likewise, many HURD features (like POSIX b.1 capabilities) have made it into Linux in recent years. It's too early to tell, but perhaps the future holds a merging of Linux with HURD in a couple of years.
~wally
True.
I think the thing that Hurd is really lacking is a Linus or Theo-like figure with an established cult of personality.
And it looks like Neal's got some hope. He's reasonably pragmatic (important) and can use humor effectively (as in the comment that he hasn't done drugs)
Of course, the problem with Mach is that there's five people working on it, and it's not ready for a mess of people to start hacking on it. It has some interesting ideas, and some really cool directions that it could go in. Somebody's going to have to do some leadership and start setting it on a course from R&D to a real production project sometime.
Gentoo Sucks
Hmm... He says HURD is slower than it should be, and it will get better. I was under the impression that it was doing some high-order operations, and that was why it was going to be slow. Eventually with high-order operations, don't you sortof hit a brick wall in optimization? For example, I thought they were using the very expensive (computationally) Mach messaging system for some of their features. Is this the case? Is their switch to L4 going to improve this issue, since Mach's speed issues are related to operation order? Iduno. I guarantee that I'm less educated in this subject than the majority of /. readers, so can someone elucidate?
There are no trails. There are no trees out here.
Let's be frank. It's too damn slow on any CPU. I think Mach (and Hurd) are really great projects, but they appear to be fundementally flawed (messaging+context-switch). I'm very disappointed that the interviewer didn't go into any details on the well-known problems associated with micro-kernel design. Also, I've never heard a really good comparison of the micro-kernel approach vs. the monolithic design with loadable modules (I'm sure there is one; I just haven't heard it). I'd really like to hear the Hurd developer's comments on these issues.
Someone you trust is one of us.
Seriously, having read the interview it seems like Hurd does some interesting stuff removing features that are part of the kernel in other unix systems and moving them into userspace.
The real question, though, is whether we need an entirely new operating system to gain these features or whether they could instead be implemented into the standard Linux kernel. Unless they can really get a large group of people starting to develop and use it, it may go the way of the buffalo. By working on getting their changes into Linux, however, they would have a much larger userbase to start from.
Sig (appended to the end of comments you post, 120 chars)
Strange. Last time we did measurements here (L4Ka), we ended up with 99 cycles on a 450Mhz PIII to send a message from one process to another. If one has communication within a single task the numbers are an order of magnitude lower (i.e., about 15 cycles).
The whole micro-kernel idea exists in MINIX, with user-space FS, MM etc..
The authentication part looked nice, but I thought I saw a contradiction when he first spoke of the safety of the system because the authentication daemons ups priviliges and second talks about a user-owned authentication daemon which is secure because cracked passwd's cannot be used on daemon's outside this users' space. This would imply that the public authentication server is hackable also in a way that authentication tokens can be had illegally.
Nevertheless I like the removal of root access necessity for a lot of stuff.
By the time the hurd is stable and featureful enough for everyone to use, we'll all be using 20GHZ machines. At that point, it may be worth the loss of speed.
You had me at "dicks fuck assholes".
Didn't you know? There's already a GNU OS. It's also been called "Emacs" in some circles.
Better hope it's the last one. Anything else reflects very negatively on Project GNU's ability to make actual friggin progress. They've been working on Hurd since 1991!
I have been looking at plan9 from bell labs recently. Has anyone here used both hurd and plan9 enough to give advantages of each?
In this age of MS-think, that means it's time to release it!
That said, I would not recommend using the GNU/Hurd on a server. At least not yet.
Hmm, that never stopped our friends in Redmond.
(Seriously, though, an interesting interview.)
Anyways, the long and the short of it was that RMS threw a giant hissy fit about the license so they never did business together. It seems that RMS can't see the forrest for the trees sometimes. Instead of giving the community a rock-n-roll new kernel, he decided to cut off his nose to spite his face.
Yours,
-Jack
Wagner LLC Consulting Co. - Getting it right the first time
Although a Linux kernel has tons of functionality that you wouldn't use if you were running the Hurd on top of it, there would be potential advantages (SMP, running on a lot of hardware, etc). However, there are (at least) two factors which I suspect might stop this show: IPC speed and memory footprint. (1) The hurd uses a lot of interprocess communication calls between one of the hurd servers and another (or between a user application and a server). Therefore a good microkernel will try to make IPC very fast. Without having looked at Linux benchmarks in this area, I don't know how it stacks up, but I doubt it has gotten as much attention as in your average microkernel. (2) Would the Linux kernel use a lot of memory on unused functionality? (This one might be less of a big deal, although I don't happen to remember things like whether Linux swaps out kernel pages which aren't getting touched).
My first comment is that "performance" means different things to different people. To some it means "throughput", that is, the amount of work that the system can do just prior to being overloaded. To some it means how well it can handle overload. To some it means low latency, that is, that the system can respond to an important event quickly. Which one is important for you depends on what you're doing.
Secondly, you're basing your assumptions on "microkernels" like Mach, which dates from around the same era as the original Windows NT. That's an "old style" microkernel. Back then, we thought that the only advantage of using microkernels was flexibility, so kernels didn't have to be very "micro".
Nowadays we know that merely reducing the kernel's domain of influence doesn't buy you much. You also need to simplify your kernel to realise performance gains. You do lose something (cost of context switch etc) but you also gain lots too, so it's not so much of a penalty, but rather it's a tradeoff.
For example, consider this: Linux often has to suspend a task deep inside a system call. If you call read() on a block of disk which is not in the buffer cache, say, you need to suspend until the block is read in from disk. A monolithic kernel may have to do this for a hundred reasons, depending on which modules are loaded. So a context switch consists of dumping registers on the kernel stack then switching stacks.
Now consider a microkernel. You already know in advance what operations you may have to suspend on, and that number is quite small. (In the read() example, you only suspend on IPC while the server responsible for disk I/O does the hard work.) So you can separate the "front half" of each system call from the "back half". You can come up close to the surface and then context switch if necessary. (Note: You have to do this anyway on a modern system, because a higher priority task may have unblocked during the system call.) Once you've done that, each thread doesn't need its own kernel stack, which makes the context switch a little cheaper, saves memory, makes thread creation cheaper, and the kernel can be made re-entrant, delivering an IRQ in the middle of delivering another IRQ, thus improving latency. It also means you don't need to hack around the problem of signal delivery while suspended. (BSD does this by ensuring that the "front half" of every system call is idempotent, and thus possibly less efficient than it could be.)
So you can see that focussing on the cost of context switching alone can be misleading.
Plus, of course, keep this in mind: if raw throughput was our most important criterion, we wouldn't have virtual memory.
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
If I understand correctly, the Hurd will, in future be moving to a new microkernel called OSKit-Mach. OSKit-mach is based (as you may guess) on OSKit and OSKit (which is distributed and maintained at the University of Utah) contains Linux device drivers. As you may know, the (vast?) majority of Linux code is actually the device drivers - so most of Linux is now available for users of the Hurd.
So in answer to your point: they have considered the device drivers.
Nope. The Linux developers are hell-bent on sticking to their monolithic design. Even if you could develop the Hurd as a set of patches, they would never make it into the "standard Linux kernel". (Curious use of the word "standard", BTW.)
The rift of Hurd vs Linux is like vi vs emacs. Vi and Hurd are meant to be a small tools designed to work in conjunction with other small specialised tools, the whole being greater than the sum of the parts. Emacs and Linux are meant to be "all features under one roof".
Actually, that's a good way of looking at your question. Asking "can't you just implement these features in Linux?" is like asking "why do you need all those POSIX commands like diff(1); can't you just implement that in Emacs?" The answer is "yes", but would you want to?
sub f{($f)=@_;print"$f(q{$f});";}f(q{sub f{($f)=@_;print"$f(q{$f});";}f});
A few people have mentioned trying to merge Linux with the Hurd. For many reasons, this probably won't happen, and would probably detract from some of the advantages the Hurd's design offers.For example, Neal Walfield mentions in the interview that there's a fellow who's succeeded by himself at porting substantially the Hurd to the PowerPC architecture. He took OSFMach from the MkLinux project, modified slightly the four core servers and libc, and had a system capable of running bash, fileutils, and I think some other standard apps. This feat confirms the portability of the Hurd's design, which might not be as easily accomplished with the Linux kernel. I don't know Linux's internal arrangement very well, but I have read comments of Linus's to the effect that kernel development shouldn't be easy. While writing Hurd servers or an implementation of Mach isn't particularly easy, it looks as though the portability and modularity promises of the microkernel advocates may be borne out. In addition, at least one fellow has succeeded at running Lites, the BSD single-server, alongside the Hurd on a machine running Mach. In principle it should be possible to run the MkLinux single server in a similar way atop Mach, perhaps concurrently with the Hurd. This would be similar, according to the Hurd's developers in a recent list discussion, to the virtual server capabilities discussed last week someplace
The Hurd accomplishes this while remaining POSIX compliance, sufficient to make the user experience indistinguishable from standard *nixes. At first my biggest disappointment with the Hurd was that nothing much seemed different. All the standard utilities were there, I got X working (though I don't use kde or gnome -- just windowmaker), and found myself somewhat surprised that most of what I need to do I can get done with my GNU/Hurd machine. This seems to have been accomplished by about ten or so kernel developers plus maybe fifty application porters over a long time; naturally if the user and developer bases were larger, things would be farther along.
My GNU/Hurd system is, however, slow. I haven't done any careful tests, but it feels sluggish at times. File access and network operations are fairly slow, similar operations are noticeably faster with Linux. There's a lot of driver support missing [e.g. no sound
Anyway, it's not quite there yet, but things are coming along, both feature- and performance-wise. It's worth trying out, if you've got a spare pc with a gigabyte of disk or so.
What are you talking about? The nVidia kernel drivers are released under the GPL, and they work just fine on any 2.4 kernel.
Interested in open source engine management for your Subaru?
Its true; betting on Moore's law has been proven to be a winning strategy. Ask Bill Gates.
It makes me think of Windows 95. When I first installed it on a 486/33, it seemed huge, bloated and slow. If I run it now on a PIII/800, it seems to be fast, lean, stripped down and almost elegant. I guess context is important.
therefore they are bad.
Right, right.
As others have noted, there's no way for a microkernel to be as speedy at flipping bits around as a monolithic kernel, copying between address spaces and everything. Apple attempts to mitigate some of those costs by keeping all their Mach threads in one address space, IIRC, but even with that speed up there's still some overhead.
But that doesn't mean that microkernels suck.
Apache doesn't serve static pages as fast as other web servers. It doesn't serve dynamic content as fast as some servers. But people use Apache for other reasons, things like configurability, extensibility, and support. And because the only thing you *can't* do with an Apache module is make babies.
Microkernels are interesting to computer scientists because they allow abstraction in the kernel, and God only knows that there's no word that computer scientists like more than "abstraction". Microkernels, for all there faults, are just plain prettier. And as research continues into microkernels and how to mitigate their many flaws, there might come a time when the extra processing they require might be worth it. Maybe all the abstraction and objectiness will be worth it to some system designer in the future.
At some point, there may be people who will be willing to trade some latency and throughput for extensibility and configurability in their kernels. They might be willing to trade some clock cycles for the ease with which they can implement different security policies, ala HURD. The point is that not everyone needs the same kernel, and not everyone needs the same kind of kernel.
And competition is good for them both. The HURD developers are incouraged to speed up the kernel, thanks to Linux. Linux kernel hackers will eventually desire some of the design niceties of the HURD kernel, they just won't admit it on the LKML.
Outside of a dog, a book is a man's best friend. Inside a dog, its too dark to read.
"Blahblah is to slow" arguments are lame. Microkernels are too slow. Java is too slow. 3D graphics are too slow. GUIs are too slow. Virtual memory is too slow. Accessing files over a network is too slow. Calling the OS instead of directly banging the hardware is too slow. *yawn*
Upgrade your 386SX to a new Athlon, dude. (Or better yet, a dual Athlon -- one less context switch ;-). Then nothing is too slow anymore. You can run CPU-bound stuff continuously at 87.5% utilization and your computer will still be just as fast as what you had 6 years ago, which was already overkill.
500 clock cycles, with 2 billion clock cycles per second, that works out to... a good-looking excuse to blame things on the next time you get fragged in Quake.
What's really funny is that you'd dare to say that something with a slight performance decrease has a "very little chance of surviving on Intel-based architectures." And yet just last week, I saw someone at my office spending way too much time, struggling to copy a bunch of files with MS Windows' explorer shell. I guess Windows has very little chance of surviving too. Unless .. wait .. unless maybe people don't care? Could it be?!?
As copyright owner of this comment, I authorize everyone to defeat any technological measure which limits access to it.
I'm not quite sure that's true. Win95 seemed bloated and slow on the ancient 486/25 I first used it on
Best explanation I can come up with is that there hasn't been any increase in processor speed in the last 5 years. I'm convinced that they hit a wall around the 386 or so, and have simply been rebranding the same chips every year or so, trusting that we'll convince ourselves that things really are going faster.
How can we continue to believe in a just universe and freedom to eat crackers if we have no ale?
Hi all;
;)
I think that micro and monolithic kernels each have their place. For my PC, though a monolithic kernel probably meets my needs best. Also, I will be referring to monolithic kernels as "monokernels" even if this is not technically correct
Microkernels beat out monokernels often when it comes to Really Big Servers and Supercomputers, in part because SMP is much more difficult to do on monokernel designs I suspect that this is why UNICOS/mk is a Microkernel (of course, it is from Cray).
As Linus once said "Mantra: `Everything is a stream of bytes'. Repeat until enlightened." In other words, any obsessiveness that gets in the way of moving streams of bytes around extremely efficiently is not good architecture. Message passing (and separate address spaces for kernel "server" modules) fall into this category.
Exactly, but your considerations for that 64-proc supercomputer or mainframe are different than your considerations for your 1 proc workstation, aren't they? Microkernel may be more efficient for the former, but monokernels are more efficient for te latter.
LedgerSMB: Open source Accounting/ERP
The HURD isn't popular yet because
- It's still slow
- It's still buggy
- It's POSIX compliant in theory, but not as used by the real world
- Hardly anyone is working on it
- There hasn't been an official release in 4 years (that's 83 years in Internet time)
- There is no prospect of a date for any further official releases
OTOH, the HURD is cool because:- It's a microkernel
- It has granular security
Except for the last three bad points, the HURD sounds alot like the Windows NT kernel. In fact, the biggest difference could be bad point #4 (since #5 and #6 flow from #4).Maybe instead of reinventing the wheel, they should just use the NT kernel with the GNU runtime tools and release GNU/NT.
It is sort of absurd to point to a half dozen largly experimental systems and point out that Hurd's features can be found in them. (Why does C++ exist? Algol68, PL/1 and Simula have all the features C++ does.)
The Hurd is aimed at being a Free, POSIX, usable system, unlike BeOS and QNX (not Free), Minix (a teaching system), Eros (a experimental system) and Plan9 (not Free, and not POSIX.) As for L4, the article points out that the Hurd can be ported to L4, it's just that nobody has put in the elbow grease yet.
flamer...
/will/ load non-GPL compat modules, but it will mark the kernel as tainted.
insmod
read the man page for insmod.
I use Friend/Foe + mod-point modifiers as a karma/reputation system.
If it is possible to implement the Hurd with any underlying microkernel, isn't it essentially possible to provide Hurd functionality with a macro-kernel as well? Layer your Hurd-servers around the macro-kernel as user space apps, disable much of the macro-kernel functionality, still suffer from not doing stuff in kernel space. This is possible isn't it? Wouldn't this allow for the "enhanced" security model which is at the heart of the Hurd? In other words, why does one have to re-invent the wheel to accomplish what the Hurd does, as well as work along side of existing systems, and not require a "compatibility" layer? The interviewer admits the Hurd suffers from a scalability standpoint. Also, isn't it possible to provide the security contexts the Hurd offers through projects for the Linux kernel, such as what NSA is working on with SE-Linux and SIDs? (known in the article as "tokens to perform specific tasks".) This is perhaps a different solution to the problem, but the result appears to be the same...
Russian Russian Russian RussianDollSig DollSig DollSig DollSig
A point I've not seen anyone make is that the Hurd scales better than Linux ever will, and I don't mean in terms of hardware, I mean in terms of development.
Vast wars already take place about what should or should not be in the Linux kernel. The rate of progress on all of these fronts together is approaching glacial. There is just no easy way for a small team to keep a grip on a monolithic kernel that is being pulled in different directions by different developers. It has taken years for a journaling filesystem to be accepted into the kernel.
The Hurd neatly sidesteps all of these issues by not requiring any of these things to be in the kernel. Useful functionality like reiserfs and tux can be developed and released without any requirement for them to be "accepted" by any controlling group. At the moment, the only way to do something similar with Linux is to release a patch. This makes Linux much more susceptible to forking.
In case anyone disagrees with me on the forking issue, consider that Linux already has a very high profile fork: RedHat. RH have been distributing a forked kernel for a long time. Admittedly they only patch the kernel in minor ways, but nonetheless they maintain an alternative version. This is because if their views on what they want out of the OS differ from the kernel admins they have no choice but to fork.
But a RedHat version of GNU/Hurd could be composed of whatever OS servers they choose without any patching. They could mix and match the kernel distribution in much the same way as they currently mix and match the userspace distribution. Because most of the Hurd kernel *is* a userspace distribution.
The thing is, Linus does care about speed difference. A lot.
And I'm sorry, but the basic GNU software set is a disaster. I speak from personal experience. I've been hassling with their bugs for years. Ten or so years ago it was hassling with the official port of GNU source control to DOS -- done by somebody who didn't understand FAT filesystem semantics. Last year I had to sweat blood to deal with the reference counting bug in glibc. This bug was eventually fixed -- after glibc maintenance moved from GNU to Red Hat. What can you say about a team that takes so long to fix such a basic bug? Aside from the fact that it has Lots of Really Great Ideas?
Face it, GNU has "succeeded" only because you need it to do anything useful with the Linux kernel.
Well, of course not. Hurd as been around much longer, if you count its Project Mach origins. If Hurd had had its act together back when LT was a grad student, he probably never would have bothered to write the Linux kernel. Proably just as well...Look, if Hurd is so wonderful and important, then you should want something serious to happen with it. That is not going to happen if all its supporters just stand around saying "cool!" And it's certainly not going to happen if nobody asks why this project has been chasing its own tail for so long.
How is HURD that different? What benifet does it provide the user that hasn't been seen elsewhere? HURD is, at best, a research OS. It has neither the mindshare nor the technological advances to become mainstream. If you compare it with other research OSs, its not very interesting even then. Sure everyone has a right to work on what they think is interesting, but people also have the right to call a particular project pointless and redundant.
A deep unwavering belief is a sure sign you're missing something...
Not really. Even if you have an L4 HURD that is complete and stable, do you really have anything more than a stable, secure UNIX system? Those are a dime a dozen these days! The microkernel will not make it super-stable or super-secure, and competing OSs (which have the maturity of code that marks a stable/secure system) will be ahead of it in these respect for at least a decade or more. There is nothing in the HURD to justify its existance; it provides nothing to the world that doesn't already exist. Maybe that's fine for a research project, but IIRC, research projects are supposed to try new things!
A deep unwavering belief is a sure sign you're missing something...
I've noticed this too.
Fortunately, I have been wearing a tinfoil hat which reflects the government mind-control ray away!
Conformity is the jailer of freedom and enemy of growth. -JFK
But the kernel is the operating system!
A Government Is a Body of People, Usually Notably Ungoverned
How does Hurd compare to xMach?
A Government Is a Body of People, Usually Notably Ungoverned