Slashdot Mirror

KernelTrap is running a story on an interview Alan Cox gave at LinuxUser & Developer Expo 2003 in Birmingham, U.K. A summary of Alan's talk is also available.

What happened on thanksgiving?

Thanksgiving 2002 saw the 2.4.20 data corruption on umount kernel released; thanksgiving 2001 saw the famous 2.4.15 "greased turkey" data corruption on umount kernel released.

There have been other kernels with problems, but it seems that the data corruption bugs tend to arrive with thanksgiving.

[disclaimer: I'm a Drupal contributor]
Drupal is a CMS which doesn't use the OO features of PHP, but has nonetheless an OO design: for example all content is a "node", and you can "subclass" a node getting a story, an image, a forum topic etc.
It uses hooks so it can be expanded easily; it has both themes a-la *nuke and templates. Of course it has a good user management system.
The core is maintained by few people (not me) in a very strict, almost maniacal ;) cleanliness.
It's fast and powers sites like Kerneltrap which sometimes is on /. home and withstands the load. I use Drupal to power different sites, from personal blogs to corporate intranets, and plan to use it as a base for other completely different projects.
It also has a very active community.
Of course it has its own faults and deficiencies, but we are working to fix them.
Here is a recent review of Drupal.

How is BeOS cleaner than KDE or GNOME? That's an easy one.

A) EVERYthing is multi-threaded, so the BeOS GUI remains responsive at ALL times. Xfree, on the other hand, runs in a single process, and all X apps running must share a timeslice with X, which is neither guaranteed nor, in my experience, acceptable - hence the nasty GUI latencies of X. This is one of the biggest things keeping me away from Linux right now. Of course, unless you've used BeOS, you're probably not even aware that you're being killed with the death of a thousand cuts.

B) There's no X server!! Bonuses here include low latency, low overhead (yes, X is a pig) and no window manager process running on top of X running on top of the kernel. In my experience Linux is like a chain of dominoes - just about anything can crash X, so all the programs vanish too, and I've lost my work. Whoops. BeOS isn't so fragile.

C)BeOS boots in less time than KDE or GNOME take to load. On my laptop, it takes less than half that time.

Internal problems? Maybe, but BeOS never had X! IMO that's reason enough to like it.

BeOS was designed to be the fastest, most responsive desktop OS available. BeOS boots in ten seconds or less on my old hardware - I'd like to see Linux try that. Also, the GUI is about twenty times more responsive than Linux could ever be, by design - and resource usage is a scant fraction of Linux w/X +KDE (which barely runs on my machine). Keep your Linux servers, but try YellowTab when it hits town.

http://www.kerneltrap.org/node.php?id=628

Have fun!

Red Hat Linux 9 Technical Changes (or when the RELEASE-NOTES are just not enough) by Dax.Kelson@GuruLabs.com
Copyright 2003 Guru Labs, L.C.
Intro Over the past eight years or so, I've been excited each time a new version of Red Hat Linux gets released. During the past few years, people have even been writing reviews of each release. As a general rule, I've been dissatisfied by the superficialities, inaccuracies, and irrelevancies in the reviews often times performed by someone who does not have intimate knowledge of Red Hat Linux. A systems administrator needs an in-depth review that covers relative to the previous release:

• Architectural & behavioral changes
• Installer changes
• Changes to included software packages

Normally, with each new release of Red Hat Linux, someone here at Guru Labs combs through it looking for the above changes to update the Guru Labs Linux courses. This time it was my turn, and I decided to simultaneously write a technical review for the system administrators out there. I hope that the results are satisfactory.

Abbreviation notes:
RHL = Red Hat Linux
RH = Red Hat Inc.
Architectural & behavioral changes There were many changes between RHL7.3 and 8.0, for example, the use of root=LABEL=/ in the /boot/grub/grub.conf file, the replacement of Xconfigurator with the redhat-config-xfree86 program, and the new dhclient DHCP client daemon. There are not nearly as many behavioral changes from RHL8.0 to RHL9, yet the ones that exist are significant. Kernel 2.4.20-8 The kernel in RHL8.0 was based on the 2.4.18 kernel. Despite the name, the RHL 2.4.20-8 kernel is based on 2.4.20 plus bug fixes identified up through 2.4.21-pre4-ac4. During the past couple years, the RHL kernels have included back ported functionality from development kernels that has proven stable. The new RHL9 kernel is no exception. Major changes since RHL8.0 include:

• Addition of Native POSIX Thread Library (NPTL) for standards based threading support with impressive performance. This is definitely a nice addition, however, I anticipate that sys admins who add patches on-top-of the RHL kernel from 3rd party (UML, FreeSWAN, etc) sources will have a more difficult time getting the patches to apply and work cleanly. Presumably when the 2.6 kernel comes out, the divergence of the RHL kernel will drop substantially.
• • Certain applications using the old LinuxThreads API in a certain manner may no longer work (was that vague enough?)
  • In particular if using Java, update to the latest version from Sun at:
  • http://java.sun.com/j2se/1.4.1/download.html
  • The WIN32 API translation software, WINE, suffers from this problem. Proper fixes are in the works, however, workarounds exist.
  • Installing and running Oracle 9i R2 has major issues since it includes two different older embedded Java JVMs that don't work with NPTL. The solution is stick with RHL8.0 or the officially supported Red Hat Linux AS edition.

ACPI support appeared in a beta (as well as in a 8.0 beta), but was removed for the final shipping kernel.

Filesystem ACL and EA support appeared in the betas, but was pulled for the final shipping kernel. I was really looking forward to ACLs and EAs support in RHL (Solaris had support since 2.5.1), maybe an errata kernel will re-add the feature.

• To see what software specifically supports

• Architectural & behavioral changes
• Installer changes
• Changes to included software packages

Normally, with each new release of Red Hat Linux, someone here at Guru Labs combs through it looking for the above changes to update the Guru Labs Linux courses. This time it was my turn, and I decided to simultaneously write a technical review for the system administrators out there. I hope that the results are satisfactory.

Abbreviation notes:
RHL = Red Hat Linux
RH = Red Hat Inc.
Architectural & behavioral changes There were many changes between RHL7.3 and 8.0, for example, the use of root=LABEL=/ in the /boot/grub/grub.conf file, the replacement of Xconfigurator with the redhat-config-xfree86 program, and the new dhclient DHCP client daemon. There are not nearly as many behavioral changes from RHL8.0 to RHL9, yet the ones that exist are significant. Kernel 2.4.20-8 The kernel in RHL8.0 was based on the 2.4.18 kernel. Despite the name, the RHL 2.4.20-8 kernel is based on 2.4.20 plus bug fixes identified up through 2.4.21-pre4-ac4. During the past couple years, the RHL kernels have included back ported functionality from development kernels that has proven stable. The new RHL9 kernel is no exception. Major changes since RHL8.0 include:

• Addition of Native POSIX Thread Library (NPTL) for standards based threading support with impressive performance. This is definitely a nice addition, however, I anticipate that sys admins who add patches on-top-of the RHL kernel from 3rd party (UML, FreeSWAN, etc) sources will have a more difficult time getting the patches to apply and work cleanly. Presumably when the 2.6 kernel comes out, the divergence of the RHL kernel will drop substantially.
• • Certain applications using the old LinuxThreads API in a certain manner may no longer work (was that vague enough?)
  • In particular if using Java, update to the latest version from Sun at:
  • http://java.sun.com/j2se/1.4.1/download.html
  • The WIN32 API translation software, WINE, suffers from this problem. Proper fixes are in the works, however, workarounds exist.
  • Installing and running Oracle 9i R2 has major issues since it includes two different older embedded Java JVMs that don't work with NPTL. The solution is stick with RHL8.0 or the officially supported Red Hat Linux AS edition.

ACPI support appeared in a beta (as well as in a 8.0 beta), but was removed for the final shipping kernel.

Filesystem ACL and EA support appeared in the betas, but was pulled for the final shipping kernel. I was really looking forward to ACLs and EAs support in RHL (Solaris had support since 2.5.1), maybe an errata kernel will re-add the feature.

• To see what software specifically supports ACLs and EA

An anonymous writer at kerneltrap.org provided this link for a working exploit:
http://isec.pl/cliph/isec-ptrace-kmod-exploit.c

Look at the patch. There's no fancy algorithm, nothing compared to Dijkstra's work at all.

Sure, but that isn't really the type, it's a description of what the variable does (it points somewhere in a file). The same way unix file descriptors are often named "fd" (and yes an fd does allow you to do a lot more with it than an fp/fptr) ... the type is int but anyone who encodes that into the name is on some serious drugs, IMO.

One could attach 'i' to every variable that's an int. But that's stupid, the point to prefixing variables is to carry around type information that the compiler doesn't check. Examples for types that C compilers don't check for... odd integers, even negative doubles, strings, strings that contain non-aphlanumerical chars, etc.

Many bugs in the open source world related to people not understanding how variables are being used. Of course anyone can read the type info. But it is hard to understand how the variable should work. A perfect example of this problem has come up recently.

from kernel trap on 2.5 and volume management:

At the end of his message, Linus commented:

"PS: NOTE - I'm not going to merge either EVMS or LVM2 right now as things stand. I'm not using any kind of volume management personally, so I just don't have the background or inclination to walk through the patches and make that kind of decision. My non-scientific opinion is that it looks like the EVMS code is going to be merged, but .."

...which resulted in a lengthy debate on the virtues and pitfalls of each of these volume managers.

Yes Microsoft Corp makes very buggy software. It is true.

Here's the canonical one: http://www.kerneltrap.org/node-592.html
It's exempted from the slashdotting message.

The static copy of the story dosen't indicate if the I/O scheduler blocks on all filesystems, or separately on each one? Ie if I have a read on filesystem A, would another write on filesystem B be affected?

http://www.bluesnews.com/
http://www.shacknews.com/
http://www.slashdot.org/
http://www.linuxgames.com/
http://www.icculus.org/
http://www.flipcode.com/
http://www.google.com/
http://www.gouranga.com/
http://curmudgeon.linuxgames.com/
http://icculus.org/fingerdigest.html
http://kerneltrap.org/

No doubt this will be buried into the mass of similar posts before long, but it is a decent format for listing where people generally go...

I have a definite list and surprisingly an order too (anyone else do this compulsively?)

1.) CNN
2.) LinuxToday
3.) OSnews
4.) KernelTrap
5.) Yahoo! Mail - Only including this because it's in my list.
6.) ExtremeTech
7.) AnandTech
8.) Tom's Hardware
9.) 2cpu
10.) Slashdot - Last because it takes the longest.

Hmm, come to think of it I have some wierd habits while surfing too. When I'm traveling my path of websites, I picture them on a 2d plane with distance in between. CNN on the left, linuxtoday in the lower middle, etc. Anyone else do this?

The GCC developers are on it already.

Now, is it really surprising that a third party compiler is not as upto date as a compiler from the CPU manufacturer?

Now go wipe your face. It's covered with Scott McNealy's jism.

GOTO dead?
Not exactly .

The contest benchmark might be what you are looking for. It tests system responsiveness by running kernel compiles under different kinds of load.

Still based on kernel compiles, granted, but at least it tries to measure responsiveness. Been used heavily to benchmark recent kernels - check Kernel Trap for results.

The Linux scheduling latency page of Andrew Morton might be useful as well. Alas, kernel patches tend to work on x86 first before PPC..

Why isn't Sistina's LVM making it into the kernel? SUSE has been including it as standard in their distribution for some time.

Why isn't Sistina's LVM making it into the kernel? SUSE has been including it as standard in their distribution for some time.

In addition, the EVMS team then recognized the implication of this decision vis-a-vis the inclusion of EVMS in Linus' tree in the near future, and decided that a significant rewrite of some of their code was in order.

"As many of you may know by now, the 2.5 kernel feature freeze has come and gone, and it seems clear that the EVMS kernel driver is not going to be included. With this in mind, we have decided to rework the EVMS user-space administration tools (the Engine) to work with existing drivers currently in the kernel, including (but not necessarily limited to) device mapper and MD."

This announcement was met with TONS of positive praise on lkml: for the actual technical decision, for the mature and pleasant manner in which it was handled, and for the public policy of removing duplication of kernel code in general, simplifying the MD/device mapper code specifically.

Finally, Alan Cox stated about 2.4:

"I plan to try and push LVM2 to Marcelo after the next release. Whether he will take it I don't know. Obviously its good to have the ability to move back nicely to older kernels."

In addition, the EVMS team then recognized the implication of this decision vis-a-vis the inclusion of EVMS in Linus' tree in the near future, and decided that a significant rewrite of some of their code was in order.

"As many of you may know by now, the 2.5 kernel feature freeze has come and gone, and it seems clear that the EVMS kernel driver is not going to be included. With this in mind, we have decided to rework the EVMS user-space administration tools (the Engine) to work with existing drivers currently in the kernel, including (but not necessarily limited to) device mapper and MD."

This announcement was met with TONS of positive praise on lkml: for the actual technical decision, for the mature and pleasant manner in which it was handled, and for the public policy of removing duplication of kernel code in general, simplifying the MD/device mapper code specifically.

Finally, Alan Cox stated about 2.4:

"I plan to try and push LVM2 to Marcelo after the next release. Whether he will take it I don't know. Obviously its good to have the ability to move back nicely to older kernels."

Microkernels have their pros and cons, with their main con being slower speed due to the greater need for message passing. The open source nature of Linux allows it to have a monolithic design with many microkernel-like properties.

With that said, GNU/HURD still has a lot going for it. Whereas Linux is essentially a UNIX clone (and there's nothing wrong with that), GNU/HURD is very different. Remember, GNU means "GNU's Not UNIX". There is a great overview of HURD given at KernelTrap.

Someone should start a site that covers long term issues, rather than the week by week stuff I've found on the web... or maybe someone has, and I'm just too out of the loop....

KernelTrap.

I was actually waiting for this drive, but mainly because of this story and others about Sony's dirty tricks with DRM, I'm waiting for another drive.
I don't care about copying CDs or DVDs, but I do care when my system hangs when I want to listen to a CD/see a DVD while (in between) working...

Is anyone aware of drives like this from other vendors?

ad 1.) In this interview with pf developer Daniel Hartmeier he talks a bit about performance.

In this interview with Robert Love in July, he predicted 18 months before 2.6 gets released(that would make the release early in 2004).

I'm more inclined to go with Robert Love's estimate considering 2.4's late release.

Offtopic : Hey, my story submission got accepted! :) Now that's a first.

Don't buy Sony CD-Drives.

in case the link is dead, apparently trying to access (copy-protected) CD's the wrong way with a Sony CD drive will cause a kernel oops. Possibly/Probably a flaw on purpose built-in by Sony.

(grrr sorry about double post, that'll teach me to hit "submit" instead of "preview")

The "throttle" module you refer to is a mechanism to detect surges in traffic, and to allow other modules to thus intelligently and automatically react. For example, the "statistics" module will automatically stop some of its logging and statistical analysis during heavy loads. In any case, this is new to Drupal, to be released with 4.1 (otherwise currently available in the development version from CVS). I hope in time other modules will utilize this logic. (I.E., themes could automatically become simpler under heavy loads, fancy features requiring lots of CPU could be temporarily disabled or reduced, etc) For more information read the help page.

I have written another module, called "filecache" which does what you have described. That is, it simply caches pages to the filesystem. If Slashdot results in MySQL choking, the site will continue to function thanks to the static cache files, automatically resuming dynamic generation after the rush is past and MySQL is able to respond to queries again. The module and patches to Drupal can be viewed here. I'm adding some more features to this module, but the core is done and seems to work quite nicely. (Additionally, with pages cached to the file-system I'm seeing a significant performance boost)

Drupal is an excellent engine, very modular in design and easy to work with. More, it's backed by a team of friendly and helpful developers. I made the switch after performance issues with PHP-Nuke, and I've not looked back. And it just keeps getting better...

-Jeremy

You can find some screenshots at KernelTrap: http://kerneltrap.org/node.php?id=404

may be found here.

-rimdo

It's not 2.4, but it gives you an idea about how things work.

Thanks for the KernelTrap mirror.

FYI, according to the OpenBSD site it's "Theo de Raadt", not "Theo DeRaadt".

Don't believe me? Check this user's posting history, Theo's personal homepage, interviews, or mailing list posts.

I remember reading an interview with Andrew Morton -- read it here -- in which he goes into great lengths about low latency and Robert Love's kernel preemption patches.

To cut the story short, he feels that kernel preemption is the proper way forward because his low latency patch is based on manually inserting points into loops which can be interrupted. This brings more noticable improvements, but is more manual work. Andrew says that a combination of the two approaches is possible, with a lock-break mechanism implemented into later versions of the kernel-preempt patch, which does essentially the same thing as the low-latency patch.

Now I'm certainly no expert on this :-) but it seems that low latency is, in a way, incorporated into the latest kernel preemption code (cause it's no longer a patch). FWIW.

To me, the most critical thing in the Linux market right now is the lack of good software courses, books and software itself. Without good software and an owner who understands programming, a hobby computer is wasted. Will quality software be written for the Linux market?

Almost a year ago, Alan Cox and myself, expecting the linux market to expand, hired Marcelo Tosatti to maintain Linux 2.4. The the initial work took only two months, the three of us have spent most of our lives documenting, improving and adding features to Linux. Now we have reiserfs, ext3, a robust VM, UML, and the 2.5 development tree. The value of the computer time we have used exceeds $40,000,000.

The feedback we have gotten from the thousands of people who say they are using Linux has all been positive. Two surprising things are apparent, however, 1) Most of these "users" never bought Linux (less than 10% of all computer owners have bought Linux), and 2) The amount of royalties we have received from sales to hobbyists makes the time spent on GNU/Linux worth less than $2 an hour.

Why is this? As the majority of users must be aware, most of you steal your software. Hardware must be paid for, but software is something to share. Who cares if the people who worked on it get paid?

Is this fair? One thing you don't do by stealing software is get back at Berkeley for some problem you may have had. Berkeley doesn't make money selling software. The royalty paid to us, the manual, the CD's and the overhead make it a break-even operation. One thing you do do is prevent good software from being written. Who can afford to do professional work for nothing? What hobbyist can put 10-man years into programming, finding all bugs, documenting his product and distribute for free? The fact is, no one besides us has invested a lot of money in Linux software. We have written 3 stable kernels, and are writing Linux-2.5, but there is very little incentive to make this software available to Linux users. Most directly, the thing you do is theft.

What about the guys who re-sell Linux, such as linuxmall.com, aren't they making money on hobby software? Yes, but those who have been reported to us may lose in the end. They are the ones who give
Linux users a bad name, and should be kicked out of any club meeting they show up at.

I would appreciate letters from any one who wants to pay up, or has a suggestion or comment. Just write to me at:

3940 Freedom Circle
Santa Clara, CA 95054 USA

Nothing would please me more than being able to hire ten programmers and deluge the Linux market with good software.

Linus Torvalds
Transmeta Corporation

Can't we at least cite those sites whose word-for-word text is reused by /. for announcement here??

Linux: UML Merged Into 2.5

IIS can use named pipes to connect with a local MS SQL server, which gives you better throughput and multiplexed reads. Apache/MySQL/Postgresql can't touch it. However, the recent dirty buffer patches help somewhat. I expect linux will overtake IIS wrt this soon.

Kernel Traffic
KernelTrap

"Serial ATA - Bummer. I realize this is new. But I get the feeling Serial ATA is going to be huge, especially for lower end servers. Finally getting real hot plug support and a setup that'll make things easier on the HW RAID vendors (I can't wait to see a Serial ATA card from 3-Ware!) I would hope this would be flagged as something to be merged into 2.6 as soon as its possible, even if marked experimental."

As an AC pointed out on the kerneltrap link:
--- from www.serialata.org --- Q2: Previous efforts to transition to a serial bus were not successful. Why do you believe that Serial ATA will be successful? A2: Serial ATA is a drop-in solution in that it is compatible with today's software

So, I'll bet that it'll be handled at the hardware level (at least preliminarily) in Linux using the base chipset drivers that are already in place. Sounds like it should be trivial to tack on a quickie patch just to recognize the hdd (albeit w/o optimizations) in the 2.6 branch, but the number of lines of code I've written in the kernel can be counted on an armless man's fingers, so what do I know.

Seems like everyone and their mother has been waiting for Serial ATA, so I doubt they'll leave us completely out in the cold...

# echo 0 > /proc/sys/net/ipv4/tcp_ecn

I can't take the credit for this discovery. I learned about it while sifting through the comments to this very article. Also, check this link for more info on ECN.

You're not disappointed with his dismissal of OPN. You're disappointed with my dismissal which I posted to kerneltrap a good 24 hours ago. Nevertheless, I'm willing to let it slip by as a well-moderated post is more likely to get the message through. Still, sad that people have to copy-and-paste articles word-for-word :-(

I'm also disappointed to see people like you are still supporting Levin as I have seen him insult and hurt the feelings of innocent developers trying to use "OPN for what it is intended for" for no apparent reason. The abuse of power can't just be shrugged off; the issue must be addressed -- and the best way of doing so is by providing a community-driven alternative.

-- Richard Osborne

In a long-expected announcement, lead HURD developer Roland McGrath admitted, "We are no longer ... developing ... GNU Mach".

This comes as no surprise. The gartner group predicted a .8 probability that GNU would close up shop within the next 5 years, citing widescale adoption of Windows 2k, as well as competing free licenses such as BSD, artistic, and public domain.

I expect GNU CEO Richard Stalin to resign under heavy pressure within the next 2 months.

I just read the news at kerneltrap -- GNU HURD is officially dead.

According to lead developer Roland McGrath, Mach is the foundation of HURD, so it's future looks bleak.

Long live linux!

Well, a bit of both of what you suggested. But I should note that I never said it doesn't already exist. My comments were directed at the views expressed in the rebuttal, NOT towards the current state of software.

When I said balance is necessary, I mostly meant that there be both open source and closed source programs out there. With regards to this, my point was that the author of the rebuttal, at times, seems to be advocating 100% GPL'd software. That's NOT balance.

As for a blend of the extremes as you called them, some balance there in the form of more software being released somewhere in the middle, such as the licsencing scheme used by BitMover for BitKeeper (described in this interview) would be nice.

With this point, I think more balance would be achieved through a more co-operative existance between open and closed source. Of course, I can't expect we'll ever see Microsoft being part of such an initiative.

I think Larry stated his opinion about this here

The other question is: would it be so difficult to produce a bk-compatible openBK? Don't think so either. If the community continues to adopt bk at this rate, sooner or alter, someone will come out with an openBK for sure. Welcome to the wonderful world of OpenSource!

If "the community" had produced anything better than CVS, bit keeper wouldn't exist, and Linus/Linux would be using it. Welcome to the wonderful world of OpenSource!

there has been quite a lot of talk lately about kernel development processes, patches getting dropped, etc. I think it's all terribly overblown. The people who aren't being heard (and who aren't even bothering to comment) are the _users_ of that system - the developers. We're all just rolling our eyes and waiting for it to stop. The current system could be more efficient, but it mostly works OK; it is very unlikely to change and anything like a kernel fork is hugely improbable, even if Linus gets bored of it all and decides to do something else.