GNU/Hurd Delayed To Fix Disk Size, Serial I/O Limitations

gregger writes "This Infoworld article indicates that the GNU/Hurd is still waiting to stampede. Evidently they have to switch from the GNU Mach implementation they're using now to OSKit's Mach which will help them support faster serial I/O and larger hard discs. Currently GNU/Hurd will only support somewhere between 1 to 2 GB partitions."

Comments

Re:GNU/Hurd

[Anonymous Coward]

GNU *is* GNU/Linux. Except for this tiny itty-bitty kernel that some guy wrote, which we only took 19 years to get around to doing.

-RMS

Hurd delayed?

[Anonymous Coward]

I'm shocked! SHOCKED!

cool!

[Anonymous Coward]

I hope it will be able to run the new Mosaic software. Have you guys seen that? It's like Gopher but with you can add pictures, change the font size, etc.

Re:cool!

[paladin_tom]

This isn't the right way to measure the "goodness" of a system. The Hurd has concepts that are actually innovative.

If you're going to say that the Hurd sucks because it doesn't support some piece of hardware or software, then *damn*, Linux really sucks... and it did even more so at version 0.2. Gee, what am I doing... where's a Windows box? Win 98 must obviously be superior to all these Free/Open Source systems, with all the hardware and software it supports.

whoops

[seanw]

The release of a production version of the free GNU operating system (OS) has been delayed beyond the end of the year, as the current development version of the system does not support large disk partitions and high speed serial I/O (input-output), according to Richard Stallman

is it just me, or does it sound like they had it all ready to ship, date planned and everything, and then someone pointed out that it was lacking some major I/O features/performance, and the developers collectively slapped their foreheads and went "oh shit, yeah, we kinda forgot about that one."

like, all this took them by surprise? sucks to forget to implement a couple crucial features, eh?

19 years

[Anonymous Coward]

GNU/Hurd. 19 years in the making, and worth every minute of it.

Finally the world will have a politically correct OS that works just like other Unices have for decades.

Thank god they're fixing partition size

[metalhed77]

Does anyone here know why they let the partition size issue languish for so long? Hell, I've had files larger than 1GB (and not porn! go figure). Hard disks have been at the 10 GB mark for years, where it really doesnt' make sense to have 10 partitions. I wish richard luck. On another note, does anyone know how HURD benchmarks against linux?

Re:Thank god they're fixing partition size

[defile]

It's a really hard problem to fix.

The ext2fs implementation is actually a userland process which takes a partition as an argument and attaches a translator to the mountpoint. The translator's job is to send requests under this namespace to this server.

The ext2fs server actually mmap()s the partition containing the filesystem and performs all operations as if it were a contiguous block of memory. Unfortunately the ext2fs driver, since it's a userland process, can only address 2GB of memory (the kernel often takes half). Adding a heap and a stack leaves mmap() with about 1GB to safely play with.

Eliminating this limitation would mean using either a 64-bit architecture, or using a read/write/lseek interface instead of mmap, which may mean totally throwing out the ext2fs server as-is. Perhaps they weren't concerned with the limitation because they thought everyone would be using 64-bit architectures by now?

I'm not sure if there are any other reasons other than the filesystem servers using mmap() for the limitation.

Re:Thank god they're fixing partition size

[paladin_tom]

Does anyone here know why they let the partition size issue languish for so long?

The partition size is limited because the Hurd maps the entire disk partition into main memory, and the 32-bit architecture of current Intel processors limits the size of a virtual address space to 2^32 bytes, hence the limitation. Changing the Hurd to do things differently isn't exactly a one-weekend patch.

On another note, once we go the 64-bit processors, we'll see a much larger virtual address space (double it's current size 32 times), and hence a much higher cap on the partition size (assuming no fix).

On another note, does anyone know how HURD benchmarks against linux?

This really isn't the right question to ask: remember that the Hurd is at version 0.2, and that "Premature optimization is the root of all evil." No new Free/Open Source kernel is going to ship and be immediately as fast and full-featured as Linux... things just don't work that way.

What's important is that the Hurd represents new OS technology... and that's more important that any current lack of performance or drivers.

Re:Thank god they're fixing partition size

[Some Dumbass...]

Hell, I've had files larger than 1GB (and not porn! go figure).

Hint: "man logrotate"

Re:Thank god they're fixing partition size

[Permission Denied]

The ext2fs server actually mmap()s the partition

This is completely nuts.

I can appreciate that the filesystem driver is a userland process - this means I can write a "filesystem" as a normal userland process (eg, make some things have a filesystem-like interface, so you can do interesting things with databases like make /etc/passwd a directory). This is a cool idea.

However, using mmaping an entire partition is just crazy. This is poor design. What were they thinking?

Are they trying to avoid the system call overhead for seek() calls? This is the only reason I can think of that someone would do this - when you read/write/seek, you have to do a system call for seek(), but that comes "for free" when you mmap() because you specify the address. This would only be a problem with HURD-like systems, because there is no overhead if the filesystem driver runs in kernel space along with whatever other subsystems it needs to use.

I know everyone hates backseat designers, but I'd like to know if the following approach has been considered:

Make two new system calls, say "readaddr" and "writeaddr". These work like read and write, except that you also specify an offset (perhaps with a "whence" field like lseek). This saves you the overhead of calling seek all the time by combining the operations into one system call. This might be useful for other things as well, but I would imagine a filesystem driver is one speed-critical piece of code that does a lot of jumping around. Actually, I just remembered that there already are calls for this: pread/pwrite.

Another possible approach: using mmap is nice, but mapping in an entire partition is fubar. Why not map in specific parts of the partition as they become needed? Eg, keep superblock mapped right away from startup, and map in other parts as they are needed. Seems a bit complex, but could be done more easily with another layer of abstraction (some library which keeps a hash or something of mmap()ed bits and provides a nice interface for filesystem drivers). I'm guessing mmap was used as this transfers certain operations (eg, cache management) "deeper" into the "kernel" and avoids code duplication.

But anyway, mmaping an entire partition is really nuts. They're not getting any sympathy from me.

Re:Thank god they're fixing partition size

[Olivier Galibert]

No, what they're trying to do is offload the cache management to the virtual memory manager. With mmap() backed by the partition itself, the VM can read and write the pages transparently w.r.t the ext2 server. With read/write/lseek, you have to do actual memory management. Last time I looked, there was no interface for collaboration between the VM and the servers for cache management.

And this kind of cache management is horribly hard in a monolithic kernel for a start. Look how long 2.4 took before the VM behaviour was considered decent (2.4.16 iirc). A decently fast distributed one is even worse to design.

OG.

Systems work

[peterb]

Is harder than most people seem to think it is.

That being said, I think the Hurd is pretty much a solution in search of a problem. Who cares? And why? The FreeBSD kernel does everything Hurd purports to want to be able to do, and is more mature, stable, and feature-complete. The same could probably be said of the Linux kernel.

Does that mean the Hurd guys should stop what they're doing? Of course not. Writing operating systems is fun.

It does, however, probably mean that the stuff they're doing isn't really news.

What is HURD?

[randomErr]

http://www.gnu.org/software/hurd/hurd.html

GNU HURD is a slimmer re-write of the UNIX kernel that is completely OOP.

Here's a cut and paste from the homepage:

The Hurd is not the most advanced kernel known to the planet (yet), but it does have a number of enticing features:

it's free software
Anybody can use, modify, and redistribute it under the terms of the GNU General Public License (GPL).
it's compatible
The Hurd provides a familiar programming and user environment. For all intents and purposes, the Hurd is a modern Unix-like kernel. The Hurd uses the GNU C Library, whose development closely tracks standards such as ANSI/ISO, BSD, POSIX, Single Unix, SVID, and X/Open.
it's built to survive
Unlike other popular kernel software, the Hurd has an object-oriented structure that allows it to evolve without compromising its design. This structure will help the Hurd undergo major redesign and modifications without having to be entirely rewritten.
it's scalable
The Hurd implementation is aggressively multithreaded so that it runs efficiently on both single processors and symmetric multiprocessors. The Hurd interfaces are designed to allow transparent network clusters (collectives), although this feature has not yet been implemented.
it's extensible
The Hurd is an attractive platform for learning how to become a kernel hacker or for implementing new ideas in kernel technology. Every part of the system is designed to be modified and extended.
it's stable
It is possible to develop and test new Hurd kernel components without rebooting the machine (not even accidentally). Running your own kernel components doesn't interfere with other users, and so no special system privileges are required. The mechanism for kernel extensions is secure by design: it is impossible to impose your changes upon other users unless they authorize them or you are the system administrator.
it exists
The Hurd is real software that works Right Now. It is not a research project or a proposal. You don't have to wait at all before you can start using and developing it.

Re:What is HURD?

[Eil]

This structure will help the Hurd undergo major redesign and modifications without having to be entirely rewritten.

"...but we might change out the whole kernel from time to time when things aren't looking so good."

Re:Delayed???

[gmack]

I actually watched Stallman speak in Montreal recently. One interesting tidbit was that he still seems dumbfounded about the fact that the Linux kernel beat them into production even though one of the advantages of microkernel is supposed to be ease of design and the fact that mach had half of the work done already.

So, Linus was rfight?

[Royster]

From Open Sources: Voices from the Open Source Revolution

So at the time I started work on Linux in 1991, people assumed portability would come from a microkernel approach. You see, this was sort of the research darling at the time for computer scientists. However, I am a pragmatic person, and at the time I felt that microkernels (a) were experimental, (b) were obviously more complex than monolithic Kernels, and (c) executed notably slower than monolithic kernels. Speed matters a lot in a real-world operating system, and so a lot of the research dollars at the time were spent on examining optimization for microkernels to make it so they could run as fast as a normal kernel. The funny thing is if you actually read those papers, you find that, while the researchers were applying their optimizational tricks on a microkernel, in fact those same tricks could just as easily be applied to traditional kernels to accelerate their execution.

Re:So, Linus was rfight?

[Animats]

As it turns out, microkernels aren't all that portable, because they need close ties to the hardware to support all the fast interprocess communication that they do. But because they're small, the fact that they contain CPU-dependent code isn't as much of a limitation. Porting is more work per line of code, but there aren't as many lines to port.

Look at QNX and L4 as examples of fast microkernels. About all the kernel does is manage memory, interprocess communication, and task switching. Everything else is in user space, where it's easier to debug and can't mess up as much when it breaks.

In addition, if you're serious about security, a system where only the microkernel is trusted is the only thing that has any hope of working. In a microkernel system, the kernel tends not to change much over time, once it's working. New functionality is all in user space. You're not patching holes forever, like we are now.

You do take a performance hit, but in a world where Java, Perl, and XML are used for production work, it's tiny by comparison.

Relevancy:

[Wakko Warner]

It's being built so that GNU weenies will finally be able to claim a 100% PURE GNU OPERATING SYSTEM. This will gain them fame, fortune, and, their primary objective:

Mad hoes.

Yes. You see, the GNU HURD project is just a front. These guys are just looking for a little lovin'. I, for one, will be downloading and running HURD 1.0 as soon as it's released, to support the libidos of these great, visionary men.

- A.P.

Re:Tha HURD

[Istealmymusic]

No, Stallman is pushing this thing because its modular and relies on the Mach microkernel and the Flux OSKit library, in stark contrast to Linux's monolithic kernel design. Not to say monolithic kernels are bad, but microkernels do have their advantages, and both GNU/Hurd and GNU/Linux will each have their respective pros and cons.

Re:Just more Vaporware?

[MissMyNewton]

Given all the comments I've been reading it's seems to me like this is nothing more than Vaporware.

Maybe not. But right now it's clearly meant to be Hurd and not seen. ;-)

Re:Tha HURD

[rgmoore]

I'd assume that they're working on the HURD because they think that it's interesting, fun, and/or a good learning experience. Not all Free Software development has to take place with the goal of taking over the world. A lot of it, as ESR points out, is done to satisfy the interests of the individual authors. That's why there are a million projects on Freshmeat that are essentially clones of the same basic project- mp3 player frontends, database systems to catalog CD collections, etc. Individual programmers write them for their own personal reasons and then provide them for anyone else who wants a copy.

Delayed?

[grub]

I want a new slashdot poll:

Which long awaited project will be the first to become reality?
a) Duke Nukem Forever
b) SMP for OpenBSD
c) GNU/Hurd
d) The second coming of Jeebus

Re:Delayed???

[Servo]

The biggest issue is that Stallman is an idealist. Torvalds just wanted a working Unix-line OS.

Re:GNU/Hurd

[joe_bruin]

when me and richard m. stallman (the m stands for 'merryweather', did you know that?) started GNU/hurd back in 1908, we were out to replace the closed-internals of the international business machines (ibm) automatic punch card tabulator, which was at use at the time in the department of the census (where me and rich were summer interns). those machines had a 2mm steel case sealed with canadian metric square screws (wherever you call them, please don't correct me). since nobody had any metric screwdrivers at the time, much less square ones, we had no access to the internal cogs and wheels of the tabulator. we definitely did not want to punch through the casing, because that would void our warranty and service contract, and we would have to contract ibm to build us a second tabulator (which cost nearly 200 american dollars, and took 7 months to assemble).

when it (frequently) broke down, we had to call an ibm machinist to come open the case for us and oil the flywheel or unjam the transverse flying arm on the card-feeder. as you can imagine, this seemed hardly the ideal solution, because usually all it needed was a little bit of work that me and rich could easily perform (even through we were not trained calculating machine operators).
long story short, we starting working on the GNU/hurd tabulator. the centerpiece to our system was the pipelined card loader, which could load the next punchcard while the calculating engine was stilll churning on the previous card. we had also designed the system so that you could have dual loading mechanisms, so that one would always be running if the other jammed. rich always insisted that we should publish the blueprints for our machine, so that other people in our tabulation club could also build similar machines, and help us with the design. to me the whole idea sounded a bit bolshevik, but richard seemed intent to follow through with it, and i didn't mind so much. honestly, i didn't believe he would ever be able to publish anything, given that his handwriting was quite terrible (although he was working on a new type of typewriter, the electro-macs so that he would be somewhat more legible).

5 years later, when i was conscripted to join the great war in europe, we had a nearly complete tabulator in hand. we had solved nearly all the problems of page clipping and bending that were present in our earlier builds, and our machine could run at a rate of well over 70 cards per minute (compared to the ibm's 42). however, we never completed the loader fully, and the latest model i saw could only hold 3 cards on the loading queue, making it much less than useful (however promising).

i've lost contact with rich during the war years. i had always assumed he's been killed in action. anyway, i'm glad to see he's still going strong with our GNU/hurd tabulator, and wish him well on it. hopefully it will be done before my great grandkids graduate college.

Re:Tha HURD

[Uller-RM]

*shrug* You and I may like speed... but for the average user microkernels offer more advantages.

You and I are running servers and gaming systems; we want pure performance, and don't mind rebooting a couple of times or recompiling a kernel to change hardware or upgrade drivers. In contrast, my mom has trouble right-clicking My Computer and choosing Properties to get a driver list. For her, a layered driver system that can dynamically load and unload drivers as needed and layer itself against instability is a MAJOR plus, even if it's not for me.

Just because I favor speed over robustness doesn't mean either is intrinsically better - it just means my needs run that way. And, being a programmer, chances are that my needs represent a very tiny fraction of the computer users out there.

I guess this makes sense

[happystink]

They better make sure that Hurd supports hard drives up to 20 terabytes or so, since that'll be about the average size by the time Hurd ever gets done.

The dark side.. er. half.. um portion of penguins.

[Art Popp]

The best reason for HURD: "Because they want it that way."

No one should have to justify what they want to build to you or anyone. Free software is not about the GPL. It's about freedoms. If these people want to build the most paradigmatically pure kernel ever conceived of, I think that's great.

If they want to turn an architecturally useful chunk of marble into a useless statue of some kid named David. That's great too.

When I enter a bunch of keywords into freshmeat and pick over the results, I occasionally ask myself, "What was this guy thinking?" Others with that same list ask that same question, but about different projects. It's the fact that we are free to combine conceptual purity, modifiability, stability, speed, and dozens of other engineering trade-offs in exactly the manner that we think is "right" that makes picking through Freshmeat like picking through a box of Dark Chocolates.

Oddly, the same rule applies. If you don't like a particular chocolate, don't eat it; don't whine about it; just pick a different one

I wish Mr. Stallman the fewest alpha particles and the best of luck in his noble pursuit.

Re:Will they have to change the name?

[paladin_tom]

Actually, when distinguishing between implementations of the Hurd on different microkernels, we ususally say Hurd/Mach versus Hurd/L4. This is consistent with GNU/Linux and GNU/Hurd, ie. HigherLayer/LowerLayer.

OSKit-Mach is just a superior implementation of Mach, so we'd still call it Hurd/Mach if we needed to make a distinction.

Real improvements to the Hurd will come with the eventual port to the microkernel which is much superior to Mach (and the answer to current anti-microkernel FUD).

Is this really a surprise anyway?

[mosch]

What's the next big headline, 'Flying Cars That Turn Into Briefcases Not Available Yet'?

Re:The Hurd

[paladin_tom]

Bang on the money.

If I might add to what you've pointed out, RMS has explained that the motivation for continuing development of the Hurd is that it has the potential to be something much more powerful.

Which leads to what really bothers me about the Slashdot crowd's reaction to the Hurd. Lot's of people I know criticize Free/Open Source Software just rips stuff off, and doesn't innovate. Well, the Hurd is one of the most innovative Free Software projects around. These guys were talking about buiding a multi-server OS back at the beginning of the 90s.

Come on, once the Hurd is finished, GNU/Hurd will be years ahead of GNU/Linux, Windows NT, or Mac OSX. The only other OS I know of that's as theoretically-advanced as GNU/Hurd is QNX another multi-server.

This is cool stuff. Unfortunately, it seems that most people just want to complain, "Oh, does it have the drivers for XXXXX? No. Then it's useless." Grow up - the value of an operating system isn't defined by what hardware it runs on. That's much easier to change than the fundamental architecture of the system.

In 1985...

[ttfkam]

microkernels were the rage. HURD answered the call and started work. Now, almost 20 years later, MIT pulls the rug out with exokernels. So will we wait until 2020 to get a working model of that too?

God bless HURD for trying to advance the state of the art and improve upon the dated UNIX model, but sheesh! I wish HURD were ready for prime time. I really do. But a working model with caveats (Linux, OSX, *BSD) will always be better than a better model that's mostly theoretical in the real world.

That said, no one's paying the HURD developers. If it gos their nads, have at it. RMS needs to relax and realize that it is little more than a research experiment and not the second coming.

Re:The Hurd

[paladin_tom]

People can use GNU/Linux. You don't use the kernel (as an end-user), you use your applications.

I think we've got a best-of-both worlds situation here: in the meantime, we've got a very good monolithic kernel (Linux), and we've got a nice multi-server in the works (Hurd), for when the time comes when monolithic kernels just can't cut it anymore.

And regarding your statement, "It doesn't matter if you are theorectially advanced.", that's a load of BS. If no one is innovating, technology stagnates. What we're seeing here is the price of innovation. And if Free/Open Source isn't willing to do this, then we'll deserve the criticism that we're just ripping off proprietary software.

Re:GNU/Hurd

[Twirlip of the Mists]

Oh, dude, you pretty much stuck your foot in it here. As evidence that Torvalds gave Stallman permission to use the name "GNU/Linux," you cited this article. Did you read it? Here's the salient portion.

HY: About the GNU/Linux argument; have you talked with Richard Stallman about this?

Linus: rms asked me if I minded the name before starting to use it, and I said "go ahead".

Sounds good, right? But the thing is, it didn't stop there. Torvalds goes on:

I didn't think it would explode into the large discussion it resulted in, and I also thought that rms would only use it for the specific release of Linux that the FSF was working on rather than "every" Linux system.

I never felt that the naming issue was all that important, but I was obviously wrong judging by how many people felt very strongly about it. So these days I just tell people to call it just plain "Linux" and nothing more.

(Emphasis mine.)

So Torvalds, who has exclusive control over the name "Linux," as used to describe computer operating system software, initially granted permission for the use of the name "GNU/Linux," but he and Stallman had different ideas of how that variation on the "Linux" trademark was to be used. In this interview, Torvalds makes it clear that he does not approve of the use of the mark "GNU/Linux" to describe "'every' Linux system." Consequently, the name "GNU/Linux" is an infringement on Torvalds's trademark.

If you want to call the operating system "Foonix," you're free to do so. You can call it whatever you want-- to the extent that you don't violate anybody's copyright. But you can't dilute or otherwise distort the trademark "Linux" without permission, which Torvalds explicitly denied in that interview.

Thanks for pointing out that article. It makes the issue even more crystal-clear to me.

If you can't say something nice...

[Ageless]

[this space intentionally left blank]

None of you so called geeks get it.

[slashdot_commentator]

HURD is not the operating system choice of "hackers" or slashdotters. Hackers want to run computer applications (reliably and speedily). That is not what HURD is about. Its the utopian platform for computer science geeks; people who want to go beyond the current paradigm of UNIX, classic sequential computing, etc. . By abstracting the ukernel to a couple of critical operations (time slicing, memory allocation, and IPC), and moving every other operation to user mode, you have a tool that can be used to implement new concepts in computer operating systems.

Its not an alternative to Linux. Its an orange to Linux's apple. It will suck as an alternative to Linux. It will run slower than Linux (especially if they stick with Mach). It will not run more stablely than Linux (given its increased complexity). It may be a better platform for multiple CPU configurations, be we won't know that for sure until its ukernel design is complete, and an implementation of HURD actually proves it to be faster. Very few people will want to port useful packages to HURD; they'll go to Linux for reliability and performance. HURD's purpose is not a platform to run applications. Its a platform for computer science research.

That is the reason why I do not wish death on HURD and rejoice when there is good news for it. It does not really compete with Linux for mindshare. If it proves to be a superior platform for MP processing, only then will it have a mundane use.

I have massive contempt for its project management. Its currently looking like the OS that will never get released. And it does not deserve a serious look until it gets a quality ukernel, like L4 (which itself is unfinished). MACH will not cut it, or its UKS(?) version.

RMS

[thoolihan]

I'm sure I will get flamed on /. for this one, but since 98% of the comments are along the lines of "down with RMS", I have to say this.

At some point you have to decide if you are going to go along with the pithy flames or do real research. It's not popular, but it reveals the truth. If not, go to the next comment, this isn't for you.

From a proctical standpoint, I understand the "Linux" side of the argument. However, people make that argument with statemnt like... "Don't do drugs, you'll end up like the Hurd peopl" - LT. RMS makes his argument respectfully on the GNU website and encourages people to use GNU/Linux. On the GNU site, he says the easiest and best way to start using free software is to go get a GNU/Linux distro. Personally, I respect people who make their arguments with facts instead of one-liners. If you buy things because they sound like a good quick answer, then you start going for things like "trusted computing".

Finally, since this is a discussion of the HURD kernel: I think people should find this interesting. The GNU tools we are already familiar with are going to get a microkernel. Merit arguments aside, there are a lot of people who choose/like microkernels (apple, *BSD). Also, it's a kernel project that offers a ton of work to be done. After all, 1GB partitions is a sign that there is a long way to go. Entry level kernel hacking on a system that has a LONG way to go is easier than "even though you've never kernel hacked, figure out how to save a few cycles with this kernel module that has been working for five years". Also, keep in mind, the HURD has one major advantage over the Linux kernel. There is not a one man bottle neck.

Personally, I like the linux kernel and use several Gentoo systems, and some OpenBSD. But I always welcome another choice in software and look forward to seeing the HURD in a more usable state.


There is a fine line between picking your battles and cowardice