GNU/Hurd Delayed To Fix Disk Size, Serial I/O Limitations

gregger writes "This Infoworld article indicates that the GNU/Hurd is still waiting to stampede. Evidently they have to switch from the GNU Mach implementation they're using now to OSKit's Mach which will help them support faster serial I/O and larger hard discs. Currently GNU/Hurd will only support somewhere between 1 to 2 GB partitions."

Comments

whoops

[seanw]

The release of a production version of the free GNU operating system (OS) has been delayed beyond the end of the year, as the current development version of the system does not support large disk partitions and high speed serial I/O (input-output), according to Richard Stallman

is it just me, or does it sound like they had it all ready to ship, date planned and everything, and then someone pointed out that it was lacking some major I/O features/performance, and the developers collectively slapped their foreheads and went "oh shit, yeah, we kinda forgot about that one."

like, all this took them by surprise? sucks to forget to implement a couple crucial features, eh?

Thank god they're fixing partition size

[metalhed77]

Does anyone here know why they let the partition size issue languish for so long? Hell, I've had files larger than 1GB (and not porn! go figure). Hard disks have been at the 10 GB mark for years, where it really doesnt' make sense to have 10 partitions. I wish richard luck. On another note, does anyone know how HURD benchmarks against linux?

Re:Thank god they're fixing partition size

[caseyc]

I have to wonder if GNU/Hurd developers have been partitioning their HD's with so many 1gb partitions up until now. And, at what point did it occur to them that there was a better way to do it?

Re:Thank god they're fixing partition size

[Permission Denied]

The ext2fs server actually mmap()s the partition

This is completely nuts.

I can appreciate that the filesystem driver is a userland process - this means I can write a "filesystem" as a normal userland process (eg, make some things have a filesystem-like interface, so you can do interesting things with databases like make /etc/passwd a directory). This is a cool idea.

However, using mmaping an entire partition is just crazy. This is poor design. What were they thinking?

Are they trying to avoid the system call overhead for seek() calls? This is the only reason I can think of that someone would do this - when you read/write/seek, you have to do a system call for seek(), but that comes "for free" when you mmap() because you specify the address. This would only be a problem with HURD-like systems, because there is no overhead if the filesystem driver runs in kernel space along with whatever other subsystems it needs to use.

I know everyone hates backseat designers, but I'd like to know if the following approach has been considered:

Make two new system calls, say "readaddr" and "writeaddr". These work like read and write, except that you also specify an offset (perhaps with a "whence" field like lseek). This saves you the overhead of calling seek all the time by combining the operations into one system call. This might be useful for other things as well, but I would imagine a filesystem driver is one speed-critical piece of code that does a lot of jumping around. Actually, I just remembered that there already are calls for this: pread/pwrite.

Another possible approach: using mmap is nice, but mapping in an entire partition is fubar. Why not map in specific parts of the partition as they become needed? Eg, keep superblock mapped right away from startup, and map in other parts as they are needed. Seems a bit complex, but could be done more easily with another layer of abstraction (some library which keeps a hash or something of mmap()ed bits and provides a nice interface for filesystem drivers). I'm guessing mmap was used as this transfers certain operations (eg, cache management) "deeper" into the "kernel" and avoids code duplication.

But anyway, mmaping an entire partition is really nuts. They're not getting any sympathy from me.

Re:Delayed???

[gmack]

I actually watched Stallman speak in Montreal recently. One interesting tidbit was that he still seems dumbfounded about the fact that the Linux kernel beat them into production even though one of the advantages of microkernel is supposed to be ease of design and the fact that mach had half of the work done already.

So, Linus was rfight?

[Royster]

From Open Sources: Voices from the Open Source Revolution

So at the time I started work on Linux in 1991, people assumed portability would come from a microkernel approach. You see, this was sort of the research darling at the time for computer scientists. However, I am a pragmatic person, and at the time I felt that microkernels (a) were experimental, (b) were obviously more complex than monolithic Kernels, and (c) executed notably slower than monolithic kernels. Speed matters a lot in a real-world operating system, and so a lot of the research dollars at the time were spent on examining optimization for microkernels to make it so they could run as fast as a normal kernel. The funny thing is if you actually read those papers, you find that, while the researchers were applying their optimizational tricks on a microkernel, in fact those same tricks could just as easily be applied to traditional kernels to accelerate their execution.

Re:Tha HURD

[SpankTech3000]

I would really like to know if this project would even be relevant to the FSF if everybody just agreed to call it GNU/Linux and not just Linux. Stallman is pushing this thing because he *hates* Linux, not because he loves free software.

My $0.02 (-$12.02 after depreciation).

Apologies Not Accepted

[Anonymous Coward]

The Hurd project is another example of volunteers for Free Software.
It might one day benefit you directly, in the same sense that
the GNU project has benefited you in the past, throughout our life.

Show some respect to the developers of Hurd. Fart their way?
Unless you are a major kernel developer from a competing OS,
you are in no position to speak this way. It is not even a joke.

( I hope you are not a typical Slashdot reader, for it
will be a disgrace if are about to read such postings on this site.)

Re:Tha HURD

[elmegil]

Given the common whining rhetoric complaining that they don't get enough credit for providing most of the "unix" utilities in Linux (while they do deserve credit, whining about it is not the answer), you'd think they'd be in a bigger hurry to do it themselves and do it "right". After all, Linus took what they claimed to want to do all those years ago, and did it himself in a lot less time and a lot more successfully (at least so far). That's not a ringing endorsement of the gnu/fsf ideology....

Remind me again....

[jone1941]

I thought the beauty of a Mach microkernel is that it is only in charge of passing messages...if there were hard drive/serial io issues wouldn't that be a limitation of the disk io server? I'm sure someone out there has an answer for this...in fact I'm sure that if I looked hard enough I could find a good explanation of this. However, given that this project has been going on for so long, and it is probably the most idealistic of all of the GNU projects, shouldn't they have gotten the mach piece right before they started everything else? Let the reasonable responses begin...

Re:Tha HURD

[Pseudonym]

I think you might be confusing yourself.

A monolithic kernel has no fewer layers than a modern microkernel OS. Say you want to write to a file. You make a system call, which goes through the system call interface layer, which then goes to the read() system call implementation, which then goes to the disk subsystem, which then goes to the disk driver.

"But microkernel OSes have to copy stuff around between subsystems", I hear you say. Well that's true under Linux too. At the very least you have to copy the data from user space to kernel space. Compare this with a modern microkernel system where the data is copied straight between your user space and the disk server's user space. Same number of layers, same amount of copying.

Having said, that I agree with you that Mach can't possibly be as fast as, say, Linux, which is why I carefully said this applied to "modern" microkernel systems, such as QNX. So my hopes aren't high for the Hurd, at least in its current incarnation. However, modern microkernel systems which run as fast as modern monolithic kernel systems are here today and they work.

Re:Tha HURD

[David_W]

It's also worth noting that Mach was tried in the marketplace as well...where is it today?

MacOS X?

do you actually use it?

[pitc]

I've got the hurd installed... and although I almost never use it, being subscribed to the mailing lists offers me insight into kernel development.

Linux was created years before I knew of its existance... and now that I'm interested in understanding how it works, it seems like I missed the boat...

Re:The Hurd

[paladin_tom]

The thing is, though, that they're still talking about it, and haven't been able to actually do anything with it.

Haven't been able to do anything with it? Buddy, I've got a GNU/Hurd box running at my university. It's not vapourware. And it's rapidly making progress.

don't expect me to get all hot and bothered because the design is different in some exciting but arcane way.

I don't call microkernel architecture "arcane." I'm not satisfied with the long-term potential of monolithic kernels, hence I support the Hurd.

Actually, IBM is getting all hot and bothered by this stuff. They're financing the development of SawMill, a multi-server version of Linux running on top of the L4 microkernel. This stuff is the future.

Re:GNU/Hurd

[amccall]

The Intel guys got Linux compiling on their compiler several months ago. Look it up. The patch was small, and it should already be in development kernels.

So, replacement for gcc, replacement for libc(bsd/other libc's mentioned), replacement for all other standard unix utilities.

As a side note several other OS projects use the GNU compiler to create their binaries. Should the be called GNU/Projects as well, even though they use few if any other GNU components? And I thought GNU and the FSF was about freedom....

Re:depend on what you mean by "right"

[Ian Bicking]

I have no idea whether the GNU/Hurd does or does not address them, and even if it does, it is 15 year old technology.

The Hurd isn't really a good microkernel -- it's not really a microkernel at all, but a bunch of services built ontop of a microkernel (Mach). Of course, the microkernel is essential to the actual operation, and the services have been written with a specific micokernel in mind... but it's not unreasonable to consider the Hurd running on a different (better, more advanced, faster) microkernel. People in the Hurd community have talked about just this, though of course no one has actually done the hard work of converting it.

But sadly, my impression of what the Hurd has shown, is that just because something is userspace doesn't mean its easy to debug. It seems like code accessibility -- even for original developers -- has not been very good. I think it's in the same way that threaded programming is much harder to debug... a complex set of interworking services is even worse.

And while microkernels allow a certain level of modularity, it really should be possible to achieve a great deal of modularity in a monolithic kernel as well -- just not in as safe a manner. I don't know that safety is the difficult part of Linux development. Well... I'm not entirely clear on what is the difficult part, I've never tried to program on the kernel. Probably an issue of factoring -- when refactoring needs to occur across module boundries (for whatever reason) it requires different developers to communicate and agree on things (which is where the overload is occuring). But that same problem will exist in a microkernel -- only the refactoring will be occuring between processes. That's not a big difference.

Maybe with enough thoughtfulness you can refactor everything in the Right Way, so that interfaces are entirely stable and development can occur without as much interdependence. That's not impossible -- there's a lot of experience from Linux and elsewhere to learn from. But I don't think that is related to monolithic or microkernel design.

Good Architecture, Bad Community

[multiview]

I really love Hurd if you take hurd just as the bunch of source files, but I can't stand hurd if you view the Hurd project as a developer community. Most of these guys waste their time with flaming you because you are dare to try booting GNU/Hurd within proprietary software (VMWare in that case).

Instead of giving you straight answer ignoring the sematical inaccuracies in a newbies questions, they spend the largest part of the discussion on explaining things you already know.

I tried to join the hurd project serveral times, but before I've been able to contribute, I've been turn of by the ignorance and fanaticism of that community, therefor..

Hurd will fail, because the Hurd community fails to attract people.

Innovations Vs Practical system

[Anonymous Coward]

The arguments on both sides seem to revolve around:
1. This is good innovative stuff!!!
2. Linux is already out and mature there so why bother.

Both sides have good points, however they are on opposite sides of the Market/Technology continuum.
Back around the time that Unix was gaining 'market' acceptance, there were quite a few >Really nicemay have an advantage, is that while it supports the feature set of Unix, and therefore keeps the Unix compatibility people happy, it can be extended modularly, because it is not architecturally limited.
Where Hurd may lose, is that there are already established ways to do Unix on commodity systems. And if you think that's not an important factor, ask yourself why there are so many Microsoft systems on desktops.

The truth is probably in the middle. There will be a niche 'market' for Hurd. Whether it's a big or small niche depend on what it has to offer, the pain level of getting what it has to offer, and whether or not the sort of development community build up to support it the way it did for Linux.

Re:Delayed???

[Zeinfeld]

I actually watched Stallman speak in Montreal recently. One interesting tidbit was that he still seems dumbfounded about the fact that the Linux kernel beat them into production even though one of the advantages of microkernel is supposed to be ease of design and the fact that mach had half of the work done already.

I have to think that the Hurd is a case of following the fashion rather than evaluating the microkernel technology on its merits.

There are lots of folk out there who will blather on at great length about the merits of kernel design for absolutely no other reason than they think it makes them look clever.

I have not done anything at the O/S level since writing one ten years ago (unless you count the Web as an O/S). At that time Mach was flavor of the month because OSF and Next had used it as the basis of their operating system and Cutler had used a lot of the concepts of Mach to design his follow on operating system to VMS. Then Rashid joined Cutler at Microsoft in a very high profile move.

So yes microkernels were flavor of the month ten years ago. However the reason why they were flavor of the month had more to do with the politics and problems at the time.

OSF was trying to build a kernel quickly to compete with System V. Microsoft was building Windows NT to get to market as fast as possible. Microkernels were touted as the equivalent of RISC in CPUs, a design that allowed for shorter development time and hence faster to market.

Microsoft had another issue, they wanted to be able to emulate other O/S. In particular Posix so they could sell in the federal market. They also wanted to be able to migrate VMS to run on WNT at a later date as a subsystem. This is actually in the works now and will take place when HP transitions from Alpha to Itanium on the high end server line. One of the reasons Microsoft was keen to do this is that Cutler and his principal staff had left Dec after Dec cancelled the Prism project, Cutler's objective stated at the time was to make Dec have to pay for the O/S they could have had for free. At the time Dec was bigger than Microsoft.

There are advantages to microkernels, but the NT design has not been pure microkernel for some time. In order to get acceptable performance on early hardware they had to allow the display drivers to run in kernel mode.

The problem that I think will prevent HURD ever working is that to build a real O/S you have to really understand the reasons behind the principles you follow and break them when necessary. RMS is unfortunately a prisoner of many dogmatic beliefs which once fixed he simply will not abandon regardless of the evidence.

Linus may or may not have known what he was doing when he had the argument with Andy Tannenbaum, but he made the right decision. Andy has written a lot of good books that are widely used as text books, I don't know if people like Cutler, Rashid, Hoare and Co would rate him as being in the front rank. It is the same situation in most fields, everyone has heard of Bruce Schneier, fewer have heard of Ron Rivest and only people in the field tend to know names like Paul Kocher (SSL 3.0, the one that works), Butler Lampson (ACLS, lotsa stuff), Clark (end to end principle), Bellovin (firewalls), Schiller (IETF Security Area director).

Oftopic: Mark Goldston, CEO of United Online (Juno/blue light) is a clueless dweeb, he just tried to tell Mark Haynes on CNBC that cable modem router boxes are not a threat to his business as few people can afford them... Not only are WiFi cable routers $100 at frys they will be built into the cable modems soon. So either he is uninformed (unlikely) or another lying CEO.

Glad you noticed!!!

[3seas]

How many commenting would have never noticed any delay had they not read it here?

Regardless of any negativity being expressed here towards the efforts of the Hurd Developers or the goal, this is a project that needs to be done.

I have no doubt that had Linux not come along, there would have been more man power and efforts put into the Hurd these past years.

Of course Linux was a distraction for many, yet it was also NOT a distructive distraction. Alot of GNU and GPL software has been developed and put into use. Enough So that, as we all know, MS has taken notice and has even launched a competitive campagin against not just Open Sourse, Linux and GNU, but with a focus on the GPL.

What software there has been made to run on Linux, can and probably already has been ported to run on the Hurd....(except for a few packages that just don't make sence to port as they deal with monolithic kernel issuse that don't exist in the Hurd). The count of software packages ported is in the thousands.

Even the drivers written for Linux are usable on the Hurd.

All of that porting and compatability was/is alot of work for which the Hurd Development team has done. So there has been energies going into alot more than just the hurd core.

Perhaps the really good part of all this is that MS probably doesn't have a clue as to what to expect of developer who will develop applications for the Hurd, to take advantage of the hurd. And it should be understood that the hurd opens the door up a lot more for development innovations.

So what will you have when the Hurd is officially publicly released... production version...??

You will have what appears to be no or very little different than using Linux. On the surface. But under the hood.... It's a more versatile, stable and in sum of.... overall more powerful in ability to bring about advancements.

There are quite a few other OS's being developed today, under the open source idea. And there is nothing about the Hurd that says you cannot attach a personal choice smart user space interface OS to the Hurd. Integrating it to benefit from the security of the Hurd, the GNU number crunching software already written, etc... thru the IPC of the Hurd....

I like the idea of plugging my personal Smart Interface OS into a hurd system for such benefits. A 3" CD, a smart card, or some yet to be developed re-writable device I can take with me.

RMS

[thoolihan]

I'm sure I will get flamed on /. for this one, but since 98% of the comments are along the lines of "down with RMS", I have to say this.

At some point you have to decide if you are going to go along with the pithy flames or do real research. It's not popular, but it reveals the truth. If not, go to the next comment, this isn't for you.

From a proctical standpoint, I understand the "Linux" side of the argument. However, people make that argument with statemnt like... "Don't do drugs, you'll end up like the Hurd peopl" - LT. RMS makes his argument respectfully on the GNU website and encourages people to use GNU/Linux. On the GNU site, he says the easiest and best way to start using free software is to go get a GNU/Linux distro. Personally, I respect people who make their arguments with facts instead of one-liners. If you buy things because they sound like a good quick answer, then you start going for things like "trusted computing".

Finally, since this is a discussion of the HURD kernel: I think people should find this interesting. The GNU tools we are already familiar with are going to get a microkernel. Merit arguments aside, there are a lot of people who choose/like microkernels (apple, *BSD). Also, it's a kernel project that offers a ton of work to be done. After all, 1GB partitions is a sign that there is a long way to go. Entry level kernel hacking on a system that has a LONG way to go is easier than "even though you've never kernel hacked, figure out how to save a few cycles with this kernel module that has been working for five years". Also, keep in mind, the HURD has one major advantage over the Linux kernel. There is not a one man bottle neck.

Personally, I like the linux kernel and use several Gentoo systems, and some OpenBSD. But I always welcome another choice in software and look forward to seeing the HURD in a more usable state.


There is a fine line between picking your battles and cowardice

Isn't all this just posturing and rhetoric?

[Real World Stuff]

I believe Microsoft has been doing this (software) for years, and has made a very profitable business. It is hard to argue with a model that can defeat the U.S. government, and have 40 billion dollars in the bank. Sure it is not free like your siter after a cuple of beers, but business drives the economy.

OTOH something to think about

" The FSF is also modifying the GNU General Public License (GPL), though the fundamental principles will remain unchanged, according to Stallman. "We have looked at, for example, adding a clause that explicitly states that you give a patent license when you redistribute the software," Stallman added. "