Slashdot Mirror
Missing Kernel Patches
BlueEar writes: "There is an interesting, short story posted on the Gentoo Linux site. It talks about kernel patches created by Linux distributors that while publically available never get submitted. It even gives an example of one 'no brainer' patch that has been sitting over a year, without being incorporated into the 2.4.x distribution. The article ends with an appeal to Linux community to keep those patches flowing to Marcelo."
What most people don't realize that someone who puts together these releases like Linus or Marcelo is by no means omniscient. The kernel is a huge piece of work, and no one person can know what's happening in every corner of the thing. Most of Marcelo's time goes to merging patches, so he surely cannot go around the net looking for what ever fixes some distributor might have used or even checking out how come some fix that was circulating around before was never submitted to him.
What's nice is that nowadays there seem to be a couple of "patch harvesters" on the lkml who create their own releases (Alan Cox is now one of these people!) and persistently keep submitting forgotten patches.
Many of the distributor's fixes are ad hoc kludges that are designed to quickly making the thing *work*, ignoring elegance and maintainability... even when they do fix things, in the long run we don't want to take all of them into the kernel.
Based on that sample patch they gave it seems that an unpatched system allocates one more page of memory than it actually uses. Sure it's not nice in terms of resource use but it's hardly going to affect the operation of the kernel.
Obviously with the number of people (especially "power" users) who run the "generic" kernel any critical flaws are going to get uncovered and patched. I think these kind of issues, that directly affect the stability of the kernel are more important than "clean up" type patches this article describes.
Obviously they're nice to have, but it's hardly a priority when there are bigger fish to fry.
Yeah, that's what Marcelo needs, every clueless dweeb bombing them with endless copies of "this rmap vm is so 1337 why dont j00 include it in 2.4.19!"
An example of why a particular patch might not be accepted, even though it seems like a "no-brainer", is because it would be for too specific a purpose. It might optimize the kernel for one particular application, at the expense of others. One of the best things about Linux is that it is general-purpose: suitable for everything from palmtops and embedded systems to servers and enterprise applications.
A patch to aggressively cache the disk in memory, for instance, might be good for servers but not for embedded systems. So, I could understand how a patch would be rejected in this case.
As an example, a company I once worked for made many minor changes to the Linux kernel. Since Linux is GPL, I made a webpage publishing these changes, and unlike the company, my webpage is still in existence!
Splash Open Source Page
These changes would be too narrow in focus to apply to the Linux kernel for everybody, so we never submitted them.
Dr. Demento On The 'Net!
I am wondering if the distributors themselves don't have too much interest in offering patches upstream, not only with the kernel. Commercial distros have a chance to become "pseudo-proprietary" this way.
I think this is a rather childish behavior and use Debian instead.
This sig is a true statement, but I cannot prove it.
Actually the pre-patched code seems to be reserving one LESS page than is actually needed, and forgetting to reserve the last page required.
Admittedly this can't be giving that bad an effect, as it would have been fixed in the main kernel but it looks like it could make the system go BOOM !
"Free software as in beer, copy protection as in racket" - Telsa Gwynne
The same thing sometimes happens to the BSDs, where a bug will be fixed (usually in Open) and nobody gets around to integrating the same fix into the others.
It seems to me that much of this could be automated... for each patch which gets added into the xBSD source tree, compare the contexts to the yBSD and zBSD source trees and alert a human if it looks like there's a match.
But for this to be effective, I think that patches would have to have labels attached, since it's really only bug fixes for which this is necessary.
Tarsnap: Online backups for the truly paranoid
> why doesn't someone start a Linux code review project ???
Then, what is http://kerneljanitors.org, mentionned at the end of the article?
Out of all the people involved with this on the planet, not one person could be assigned the task of doing this sort of sweeping up? Lots of busy folk out there, certainly, but those people were found to do the major stuff in the first place... And please, save the "well why don't you do it, smarty man?" responses for someone that sort of backwards logic will work on, thanks, I'm just making an observation, not an accusation.
BytesTemplar.com
"Out of all the people who moan that these sort of issues should be fixed by someone else isn't there someone who could be ordered to do this in their own free time instead of having fun to fix this for small minded whingers like me?"
Yes, fortunately the kerneljanitors project does this. And I think they do it out for altruistic reasons rather than because someone assigned them to.
This is not backwards logic. I am not suggesting you do it. I am suggesting you stop whining about there being noone else to do it when you can't be bothered either.
Carpe Daemon
There is an interesting, short story posted on the Gentoo Linux site.
No, there's a short article posted on the Gentoo Linux site. A ``short story'' is a form of fiction. (Not that anyone at Slashdot cares, but some of us can't help tilting at windmills.)
--Jim
And was Linus "too young and inexperienced" when he totally botched the 2.2.x series?? (arount 2.2.9 IIRC)
What kind of quality assurance is that
It isn't. Read the licence. It's called the GNU GPL. I'm surprised you haven't come across it before. There is no "quality assurance" If you need someone to hold your hand, or you are using Linux for production purposes, go and talk to Redhat or another distributor who will provide the quality assurance you seem to want. They test all their kernels with test suites and simulated production workloads.
-- MartinG To mail me: echo kewyjlcxyzvjfxbqwh | tr bcefhjklqvwxyz
Careful.
:wq
Besides, you operate under the assumption that I haven't contributed anything to the kernel, which would be wrong.
I think the assumption he operates on is that you can't be bothered to go through and submit patches, as you were complaining that someone should to do it.
It's not an issue that people aren't working on the kernel enough, it's that there are too many mad scientists, and not enough henchmen.
could be that whoever produced the patch (Mandrake in this case) got tired of having to submit it over and over, only to have it ignored bye (for example) Linus. i'm not complaining here, but i think at least part of the solution to this "problem" relates to how the patches are handled by the maintainers.
Acts@core.mailboks.com Acrux@core.mailboks.com Adam@core.mailboks.com Adar@core.mailboks.com Ada@core.mailboks.com
You've never used Solaris have you?
The only thing I've seen that was more patched than Solaris were some of Novell's products.
I'll stick with Linux, thanks.
No, this is not a troll. Hear me out. This is an example how this work can be done using a good tool. I use Visual Sourcesafe here as an example, but any tool with the same functionality described below will do:
Visual Sourcesafe has the ability to merge back changes automatically in branch B from branch A when they have the same parent.
Say, you have the kernel v2.4.10. You branch off another project from it, call it v2.5.0. When you fix a bug in 2.4.11, you can merge it back into 2.5.0 without a hassle, it can be automated or you can do a visible merge when there are conflicts. The other way around also does work. So you can do this even further: branch of a prerelease 2.4.11-pre branch and a 2.4.11 branche from the 2.4.10 branch. Create fixes in 2.4.11-pre, merge them back into 2.4.11 after testing and when you're done, release 2.4.11 and get rid of 2.4.11-pre.
This is inside a versioncontrol system, you don't have to hassle around with a lot of files you have to merge by hand which will increase the risk for errors.
Of course, Visual Sourcesafe is just 'a' tool, you could use another which has the same functionality and is perhaps Open Source (I don't know of any but I'm sure others will). Doing this job by hand TODAY is erm... not understanding why we have computers in the first place. That's right: to serve mankind.
Never underestimate the relief of true separation of Religion and State.
Marcelo is certainly well aware of the existance of many patches that never get included in the main kernel tree, as he maintains Conectiva's kernel package which contains a large amount of vendor patches. He certainly has his reasons for not including the patches to the official kernel -- it certainly would make his life much easier if he reduced the number of vendor patches in Conectiva's tree applying some of these to the main tree. Marcelo is being very conservative regarding the 2.4 tree, and I believe that's the way it should be, considering it's a "stable" kernel.
The article is concerned with patches that big Linux-distros apply to their kernels. The kernels they put in their distributions, not special purpose kernels. Redhat (and other Linux-distributors too i suppose) do extensive testing on those kernels before they get included with their distributions. So if they find a bug and patch it, or if they find that a patch has issues in testing (and leave it out) it would benefit the whole Linux-Community (themselves too, since they would have fewer patches to manage) if that information somehow made it back to the kernel-maintainers.
--
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
... because once their patches are included they wouldn't have to maintain them themselves. So i don't see, how it could be a waste of time to send obvious patches in, or alert the kernel-maintainer of problems with recent patches that came up in their testing.
--
"By the way if anyone here is in advertising or marketing... kill yourself." -- Bill Hicks
I'd tend to agree, but it easy to say "accept everything" good disregarding who and how it will be maintained. I think that bug fixes beign dropped are a problem, patches not beign included DEPENDS. If they can't maintain it, Linux stalls and we all lose. So the problem is, as Linux said (and i didn't see it inmediately) sending the patch to the people that actually maintain the stuff beign patched. Just my 0 cents (i am broke)
unfinished: (adj.)
Actually I commit every patch I make to KDE into CVS right away, unless it's something that simply doesn't make sense for everyone (like changing a default setting to match Red Hat Linux, or making stuff run through consolehelper which isn't avaliable on many other OSes (even other Linux distributions)).
This message is provided under the terms outlined at http://www.bero.org/terms.html
Another possible explanation is that there are a small group of whiny people who are tired of Linus controlling the linux development process just because he happens to have invented it in the first place, and are waging a propaganda campaign to replace him with a committee that will rubber-stamp every ill-concieved patch submitted from anywhere on the internet, with little or no review. Goodness knows, any time any random undergrad or script kiddie changes a few lines of code in the linux kernel, it must be an incredible improvement that we cannot live without. What does Linus know about kernel development, anyway?
I was running at 80x34 first. I had to maximize the terminal to 129x44 to be able to read it. I doubt your terminal is even close to 1024x768 characters...
--
The Cap is nigh. Time to get a fresh new account.
As a maintainer of a package which is distributed via many linux and *BSD distributions, I'd like to complain on the behalf of software authors everywhere. The linux distributions are nutoriously bad about applying patches to their rpms (say) but never submitting them back to the authors of the package themselves. The BSD distributions are just as bad. The infamous FreeBSD port tree also frequently houses patches that never make their way back to developers.
I'm not sure how this could ever be considered a good thing, as the project authors must spend time searching through distribution source releases looking for patches, which takes time. The distributions must continually apply their patch to a changing source tree (and I'm sure it'll eventually break and need reworking), so they loose time as well. This is one case where communication really could be a very positive thing.
sigh... It's about time I went to search for patches again...
The next site to slashdot will be ready soon, but subscribers can beat the rush and start slashdotting it early!
Interestingly, however, much of this stuff can be achieved with LCLint, which if I recall correctly has a mechanism which can be used to extend it to statically check almost anything. Now that sounds like a good idea (tip: if running lclint for the first time on a program, use the -weak command-line option!)
If vendors have semi-proprietary systems by virtue of applying patches that aren't making into the mainstream ...
...
And if one wants to ensure that one is running the most stable, but well-patched system
Then who has it - Redhat, Debian, Mandrake, etc.?
Or is this even a fair comparison? And should one make this comparison when planning a Linux install?
To celebrate the occasion of my 1000th post, I will post no more forever on Slashdot. Goodbye.
That would be a neat idea in your Linux distribution - why don't you go and do it?
For now, the "real" Linux is a benevolent dictatorship of Linus for Linus (and sometimes not all that benevolent either :). I'm not saying this is a bad thing, it just means that what you get is exactly what Linus wants, either because he likes the idea or because people he trusts talked him around to liking it.
I'm not sure he could maintain that oversight if people were voting in changes all the time. Although I could see a system where various patches get votes, and then the winner(s) are given a chunk of Linus precious time for him to consider them. The big stumbling block in the Linux kernel process at the moment is not lack of patches, it's lack of time for Linus to consider them and integrate them.
Your right to not believe: Americans United for Separation of Church and
From Kerneltrap's wonderful interview with Andrew Morton:
The above article should be required reading for those following/concerned about kernel development.
What good is a stable tree if all vendors have to apply 500 patches to it for it to be useful?
Exactly.
Mod this A.C. up!
It is extremely difficult to be proprietary when you are bound by the GPL. If your referring to Red Hat's using Rick's VM, there would be no stopping you from nabbing a
I also use Debian and must tell you that they make changes to the kernel. That is good, however. It just isn't practicle for a distro to try and update to the latest kernel. Plus if you like me, the first thing you do on any distro is nab a tarball from ftp.kernel.org.
This is one of the problems Ive always had with the GPL (although I use it on my code; I think everything the GPL provides else outweighs this minor issue). A clause that requires one who makes public modifications to GPLed source should be required to, at the very least, make a good effort to contact the original author/maintainer and supply him with the modifications. Now, RMS is right that having this as an absolute requirement is bad if the original maintainer disappears, it would suddenly lock people out of using their code. Therefore, one should merely be required to contact the email or website credited in the copy of the source they are attempting to modify if the address is dead, the requirement is nullified.
Liberty in your lifetime
What kind of quality assurance is that
It isn't. Read the licence. It's called the GNU GPL. I'm surprised you haven't come across it before. There is no "quality assurance" If you need someone to hold your hand, or you are using Linux for production purposes, go and talk to Redhat or another distributor who will provide the quality assurance you seem to want. They test all their kernels with test suites and simulated production workloads.
Open source software is no excuse for a lack of QA. There are people who want to use Linux for serious work, there should be a -stable that is truly stable. That's the point of concurrently having a 2.4 and a 2.5 release. Red Hat's job is to create a distro from various components and test them together. They can't be expected to transform an unstable kernel into a stable one. Their job is hard enough as it is (keeping track of hunderds of packages is not easy).
Of course there is always a compromise between stability and (fast) progress, the *BSD's are far more focussed on stability. IMHO Linux is focussing far too much on progress, creating great instability. The newest developements belong in the unstable releases for the adventurous to use and test (new VM's for instance). Kernels in the 2.4.x tree should be well-tested before they are released into the open.
The Drowned and the Saved - Primo Levi
I just wanted to mention that Gentoo is the coolest distribution that I've ever tried. It has quite a time consuming install process because everything is compiled from scratch; however, that's the power behind the distro. _EVERYTHING_ is compiled specifically for your hardware, and you specify global compiler optimisations that you want applied to each and every package. The package manager, portage, is based on the FreeBSD port system, but it's rewritten in Python with many added features (i.e., better handling of dependencies, fine-grained package management, "fake" (OpenBSD-style) installs, safe unmerging, system profiles, virtual packages, config file management, etc...). It has the ease of Debian's apt mixed with the better performance of custom compiled binaries, and let me tell you, it flies! It includes custom patched kernels with the preemptive, scheduler, XFS, and many other features already patched in! Running Gentoo and Win2k Pro dual boot on my machine, I can tell you that Gentoo (w/ KDE2) is noticably faster and more responsive, and I never thought I'd say that about X under Linux, but it's true! If you haven't done so, try Gentoo today!
A musician without the RIAA, is like a fish without a bicycle.
It's not an issue that people aren't working on the kernel enough, it's that there are too many mad scientists, and not enough henchmen.
:)
Actually, this is almost exactly what I was trying to say.
BytesTemplar.com
Bless you, sir. *sniffle*
:)
BytesTemplar.com
It seems like it would be trivial for vendors to maintain their patches in their own BitKeeper repository. If done consistently across vendors, it would allow the kernel maintainers to merge patches into the standard distribution with minimal effort.
Moreover, this would probably make it easier for anybody to track different sets of patches. Imagine being able to use an SCM tool to help minimize the pain of tracking patches through several kernel revs. Many of us do this on a daily basis anyways and would love to see such tools used properly in the open source community.
I've used CVS and VSS, plus some own made tools but these were never up to par with what other tools could offer. The mention of sourcesafe was indeed as an example. I know VSS isn't made for very large projects, even microsoft uses a different system internally afaik, but the functionality it has (i.e. the branching/merging) is IMHO what should be used in Linux development/management.
Never underestimate the relief of true separation of Religion and State.
...And no, I'm not trolling.
/.).
People talk about the exchange of ideas between the BSDs and Linux, and I think that a core group like FreeBSD's would be a great idea for the Linux world.
It seems like we are running into more and more scaling issues with the people behind Linux than with Linux itself. This is no fault of theirs. Linux is too big a project for a "the buck stops here" kind of person like Linus.
Obviously, Linux is Linus's brainchild, and he can do whatever he likes with it (yes, I know the GPL allows forking, but think of how a kernel fork would be recieved on
I don't believe that Linux can attain the kind of consistency (and that is not the goal anyway) of FreeBSD or NetBSD, I think they might be able to fix some of the kernel patching and architecting problems if an elected core team could work on this.
-Peter
. Penguins Surely Ca
Does using CVS as compared to the lkml method of submitting patches make a difference in terms of quality/stability of code and speed in which bugs get fixed?
I figure since theres been so much speculation, its probably better to ask somebody whos likely to have done both.
---
Silence is consent.