Interview With WOLK Creator Marc-Christian Peterse

Jeremy Andrews writes "KernelTrap has spoken with Marc-Christian Petersen, who originated the WOLK project in March of 2002. WOLK is the Working Overloaded Linux Kernel, a large set of nearly 450 useful patches applied against the current stable 2.4 Linux kernel tree. The project has recently expanded to offer a second 'secure' patchset, this one against the older stable 2.2 tree. In this interview, Marc-Christian Petersen tells the history behind WOLK and discusses many of the patches included."

Comments

Re:What about the newbies

[CanadaDave]

It's so easy to compile the kernel from source. Redhat has a nice PDF tutorial about this somewhere on their website. Sorry I can't get the link for your right now. It is actually a chapter in their user manual I think. I learned how to do this when I was a newbie and it wasn't too bad...even for a newbie.

Production servers

[cpeterso]

Actually, I disagree. I have found the WOLK kernels to contain a lot of the fixes and features we needed all in one convenient package. Of course, I stress tested the WOLK servers before putting them into the production server room. I would highly recommend anyone that is curious in the WOLK kernels to use them in a production environment.

Re:Production servers

[jaoswald]

I'm not a kernel aficionado, but the one that gives me the willies is the Compressed RAM caching. That sounds like a gimmicky fix for "too cheap to buy real RAM."

That and the quoted emphasis on MP3/audio performance seem like this package is not aimed at real production situations, but personal workstation.

I realize that these features can be managed individually, but then what is the advantage over managing these by oneself?

you can test all the patches.

[7-Vodka]

It's a very quick way to test all the patches. Then only apply the ones with features you want to your production kernel.

(First) fork?

[netsharc]

Is this the first big "unofficial" fork of the Linux kernel? We've had different trees, but those have been maintained by people who are very close to kernel-development (I only know of AC, but I belive there are more), but this tree seems to have come out of nowhere? I hope this isn't the event that marks the beginning of Linux following of the old Unix history.

I guess that's the idea of a modular and open-source kernel, you can add things you want, remove things you don't want, but somehow adding patches from "outsiders" make me feel I'm not running a *real* Linux system - The way Linus Intended(TM).

OTOH, My LFS system is unique, with changes that make it different to standard Linux systems - including patches in the kernel - and I love the thing, so I guess there's no need to feel "guilt" over it.

Re:(First) fork?

[SmegTheLight]

First ? Nope - Before Wolk was Folk (the Functionally Overloaded Linux Kernel).

Fork ? Nope, just some some guy with the help of the few, to bring to the unwashed masses the kernel that the people with the time to hunt down and find patches have.

It's a patch set to the main kernel - Just with a crap load of really good stuff added on. Stuff that is out there, in the great open wastes of mailing lists.

ps.. WOLK is a great topping for your LFS cake.

Things I'd like to see in the kernel...

[Bollie]

This is just my personal wishlist:

1) A standard hardware acceleration layer for 2D and 3D cards, something we can ask the NVidia people to add to their drivers and code equivalents for other cards.
2) Wine intergration. Routing Win32 messages through the kernel would be kinda nice.
3) Java acceleration. Hooks for some standard Java functions: this would help a lot in some specific embedded situations.
4) ACL support for ports and stuff (like the security patches).
5) A standard "driver package" format containing the kernel module, user-mode tools and installation instructions for binary only (yecc) drivers. (One driver fits all distros!)

I've been working with Linux-based systems since '97 and I have to say, it's just getting better and better. I'm sure a lot of the above is would actually not be good in most kernels, but since one of Linux's strong points is scalability, I'd really like to see Linux take on the desktop, handheld and server market!

Re:Things I'd like to see in the kernel...

[Pemdas]

All of these are IMHO, of course...

1) A standard hardware acceleration layer for 2D and 3D cards, something we can ask the NVidia people to add to their drivers and code equivalents for other cards.

I agree with this, and hope that the future holds running XFree86 (or Berlin or whatever) on top of the standard Framebuffer/DRM interface. It's already possible to run XFree86 on the framebuffer driver, but the system needs to have kinks ironed out; most of that work is happening on non-x86 ports, though.

2) Wine intergration. Routing Win32 messages through the kernel would be kinda nice. 3) Java acceleration. Hooks for some standard Java functions: this would help a lot in some specific embedded situations.

No. Just no. There's no significant gain to be had from putting any of that in the kernel. Furthermore, this is userspace stuff. The fact that the kernel has good stuff in it doesn't mean everything should go into the kernel...

4) ACL support for ports and stuff (like the security patches).

I've mixed feelings about this. It seems that the current security model is fine for the typical user, even though ACLs are really wonderful for larger servers with more nebulous administration structures. Overall, though, I think you're right, this stuff should get in eventually.

5) A standard "driver package" format containing the kernel module, user-mode tools and installation instructions for binary only (yecc) drivers. (One driver fits all distros!)

This is a complete don't care for me. If it comes with a binary-only module, I don't buy it. I'm still running a Matrox G400 at home as a result. Generally, I think kernel developer sentiment is turning more and more negative towards binary-only drivers--I wouldn't expect the community to do much, if anything, to make it easier for such developers.

Re:Things I'd like to see in the kernel...

[1010011010]

> > Wine intergration.
> this is userspace stuff

Heck, it's "userspace" stuff on Windows NT/2000/XP, as well. CSRSS (Client-Server Runtime Subsystem) is the Win32 operating environment server, and is a native NT application. It talks directly to the NT kernel and exports the Win32 API. "Windows applications" talk to CSRSS. Windows NT/2000/XP is a lot like "Wine running on VMS."

Re:Things I'd like to see in the kernel...

[Some Dumbass...]

1) A standard hardware acceleration layer for 2D and 3D cards, something we can ask the NVidia people to add to their drivers and code equivalents for other cards.

The problem with this is that then misbehaving 2D and 3D programs can mess with your kernel. There's already a "standard hardware acceleration layer" for 3D called DRI, the Direct Rendering Infrastructure (I'm guessing you know that, but some people may not). I personally have hard locked-up my system and even corrupted my hard drive by compiling and testing buggy code against the DRI-accelerated version of Mesa. I'm not even sure I was running as root at the time! (If you allow non-root users to use DRI for 3D graphics by seting "Mode 0666" in the "DRI" section of XF86config-4, can't they do the same kind of damage?) For this reason, I would argue that kernel-level graphics acceleration is kind of dangerous. Perhaps coding DRI differently could prevent these problems, I don't know (possibly at the cost of performance?)

For 3D, I think that having kernel-level acceleration is inevitable. You just can't get good performance even on the fastest video cards without some sort of kernel-level rendering interface. Even the non-DRI nVidia drivers use a kernel module (And sure enough, I've hard locked-up my Linux box using both DRI-based drivers and nVidia's drivers, for different video cards.) Even "good" performance is, of course, not really sufficient. Fancier and more detailed 3D rendering is going to push video cards to the limit for years to come. So for now, we're going to need all the performance we can get, even if that means risky kernel-level 3D-acceleration.

For 2D, on the other hand, who needs the extra performance? Everything 2D that I do is already more than fast enough even without kernel-level acceleration. I say just let X be the 2D standard for Linux. It's already becoming the de-facto standard, as SVGAlib, GGI and other 2D rendering libraries seem to be getting used less and less. And even if it's not as fast as it could be, X must be safer than using the kernel framebuffer or some other kernel-level 2D.

Re:Things I'd like to see in the kernel...

[oever]

5) A standard "driver package" format containing the kernel module, user-mode tools and installation instructions for binary only (yecc) drivers. (One driver fits all distros!)

This is a complete don't care for me. If it comes with a binary-only module, I don't buy it. I'm still running a Matrox G400 at home as a result. Generally, I think kernel developer sentiment is turning more and more negative towards binary-only drivers--I wouldn't expect the community to do much, if anything, to make it easier for such developers.

I agree. A better solution would be to have a mechanism in place that allows a driver module to be compiled from source easily, without the need of having a previous compiled kernel. Maybe this mechanism exists, I don't know. I've never seen it though.

Ideal scenario:
A new device is being developed. The manufacturer writes a driver for all relevant kernel versions and mails it to Linus. At some point after this, the drivers shows up in the relevant kernels. Now the new hardware hits the market with a source-only driver and a compile script that figures out which kernel is present on the system for the people who do not run the newest kernel.

If a scenario like this is formalized, it will give hardware manufacturers more incentive to write drivers for linux, since it will be easier to garantee that the hardware will work with a stable kernel.

Re:Things I'd like to see in the kernel...

[bzzzt]

For 3D, I think that having kernel-level acceleration is inevitable.

It's not required. The utah-glx project which provided 3d support for the XFree86 3.x series did not need a kernel driver. The DRI kernel driver is needed for properly storing the video card state when switching tasks to allow multiple programs to make use of the video card. The real 3d stuff is still in the XFree driver.

Re:Not for production?

[WanderingGhost]

WOLK is not for production use, but the -secure 2.2 kernel is (I've asked Marc and he later included this in his announcement to lkml).

I'm using it on a server and it works great.

Catching up with Macintosh of 1996

[yerricde]

Compressed caching is the introduction of a new level into the virtual memory hierarchy. Specifically, RAM is used to store both an uncompressed cache of pages in their 'natural' encoding, and a compressed cache of pages in some compressed format. By using RAM to store some number of compressed pages, the effective size of RAM is increased, and this way the number of page faults that must be serviced by very slow hard disks is decreased.

This is exactly the technique that Connectix's "RAM Doubler", a replacement for the Macintosh System 7 virtual memory manager, used way back in 1996. I wonder if Connectix has a patent on it.

SuperMount has the ability to access your cd's/floppies on the fly without need to mount / umount them every time.

Mac OS has automounted removable media since 1984.

It's good to see that Linux is progressing as a kernel for a workstation OS. But even its major proponents admit that it has some catching up to do. WOLK is a step in the right direction.

Re:sweet

[jonadab]

> One of the patches talkaout in the interview was supermount.
> Does anyone know why this is not in the main kernel.

Didn't know it wasn't... (Guess you know which distro I use.)

> This is a must have feature for linux on the desktop.

Agreed. _Especially_ for expansion into the non-geek
end-user segment of the desktop market (the largest
segment).

> It has been included in distros like mandrake for a long
> time, so it should be pretty stable.

It's been stable in my experience.

Re:What about the newbies

[mindslip]

Download the linux-2.4.18-WOLK3.x-fullkernel.tar.bz2 file into /usr/src

type:

tar jxvf linux...whatever...tar.bz2

to unzip it

type:

mv linux-2.4.18-WOLKwhatever linux

to rename the directory it creates to "linux", so other stuff you might build can use it.

type:

cd linux
make menuconfig dep bzImage modules modules_install

Play with all the features! Make sure in the first menu item, you enable "experimental features".

Then if you don't die with an "error 1" or something similar, run Linuxconf, goto

Boot --> Lilo --> Add a kernel I've just compiled

and play!

Whatever you do, MAKE SURE you don't overwrite your current (*working*!) lilo/kernel entry! Use a different name.

I've relied on WOLK for a lot of neat drivers and speed/reliability fixes I just can't get if I try and patch the bare kernel myself.

WOLK is the most valuable project out there to the enterprise... it *REALLY* makes Linux kick butt when it comes to server-room type hardware. Hats off to everyone involved.

mindslip