Linus Does Not Scale

EmilEifrem writes: "Seems like everybody's getting more and more frustrated by Linus' (in-) ability to handle patches. Rob Landley just wrote an "RFC on Penguin Patch Management" wherein he proposes a "Penguin Patch Lieutenant" system that he believes would scale better. The full discussion can be found on the Linux kernel mailing list. Linus seems to dislike it, as usual, source code maintenance tools/organization are for wimps!, but a lot of others find it a good idea. Anyway, it's a very good read."

Comments

Growing pains

[Multiple Sanchez]

Lots of corporate/management types have the negative impression of Linux as an OS that has no professional control over kernel development. It's seen as a souped-up hotrod modified in the garage that runs like a dream but could fall apart at any minute. Hell, even a lot of BSD people see it this way. If (*if*) a goal of the Linux community is to gain wider acceptance or be taken more seriously, one way to do that might be to give more than one person final say over the kernel.

Yeah, it's Linus' baby -- but once IBM is advertising your OS during the superbowl, it might be ok to expand the development structure a little bit.

Linus is, as is often the case, right

[gaj]

Linus's vision of the kernel dev process is really the right answer. Why can't more people see that? By using a non-hierarchial network of maintainers, the kernel effort can scale indefinately.

The basic premise he makes is one that many developers seem to miss. To quote Linus:

Basic premise: development is done by humans

From that, he goes on to point out that people tend to have a small group of people they work well with and trust; perhaps 5 - 10 people. So, in essence, if Linus appoints 5 - 10 maintainers of major subsystems (which he has), and each of them has 5 - 10 people the trust to maintain specific aspects of their subsystem, then there is no problem.

We're not there yet. But that is the only direction I can see that will work, long term. It just takes time for the appropriate people to move into their spots; trust takes time to build, and the structure has taken time to modularize enough to allow this to work.

And, though I would prefer to see it in use just to make the maintainers' jobs easier, CVS will do nothing to solve the problem. It is a tool, not a process. We need a process. Actually we have a process, it just isn't fully implemented yet.

No, strike that. We have two processes. The first, used by the bulk of the kernel hackers, is not fully implemented yet. The second, used by a minority of the kernel hackers and a large part of the pretenders, is simple: whine alot, and, when that doesn't work, whine some more.

Linus says it much better, and in his own unique ... er ...

Idium, sir?

Yes, in his own, unique idium.

Re:No, no, no...

[Shotgun]

He then goes on to say that the solution to the problem of the scalability of one maintainer is to partition the different subsystems of the kernel to such an extent that there would be precious few patches that actually require a knowledge of the entire kernel source.

Which is the proper way to design a complex system anyway. Partition the system and then present well defined interfaces, versus making a complex, monolithic system more complex and monolithic.

Linus is correct. CVS just gives you an extremely efficient filing cabinet. Without a someone to file things properly, the filing cabinet is no better than a shoebox. Running a CVS tree makes you the equivalent of a secretary. People will just come by, throw things on your desk, and yell "File this" as they run home for the day. Linux chooses to be an engineer. He refuses to just accept what it thrown at him. He wants explanations and reasoning as to WHY a patch should be included. Not using CVS slows the development system down so that it is comprehensible by a single person. The kernel isn't just a collection of source files collected in a cabinet. It's a fine masterpiece of artwork on Linus' desk.

CVS can work, look at FreeBSD

[Chris_Keene]

A lot of people have said CVS could not work, agreeing with Linus. Yet FreeBSD uses a CVS tree with multiple committers, and has had no serious problems that I am aware of. Infact, I would say they have had less problems than linux has had in the last few years.

cjk
Toothpaste likes you

Re:Is Linux bigger than Linus?

[renehollan]

The problems with CVS is that it doesn't handle directories and branch merges very well. The former is a shortcoming of CVS and the latter makes it painful for Linus (or anyone for that matter) to maintain a private branch. It's doable, but ugly.

Of course, I was spoiled with Clearcase when I was at Teradyne, but, at $1000 a seat, it isn't exactly cheap (and is dangerous in the wrong hands). Working at a startup now, and living with CVS, has tought me that it doesn't provide the kind of isolation, yet, tracking that Linus needs.

However, perhaps all is not lost. If Linus doesn't like CVS, let him not build out of CVS, but still provide an incomming CVS repository for certain trusted leutenants that filter patches. This provides the following:

1. A history of patches received.

2. A mechanism for delivering those patches to testers.

3. A source Linus can diff against his private tree.

Of course, when Linus makes changes outside the CVS repository, it will be out of sync, but the suggested Patch Penguin can be charged with syncing it up with Linus' latest changes. This way, Linus can generate his own patches from CVS and examine them at will, or on the urging of leutenants.

Another orthogonal approach to the ckernel into separately maintainable parts. There is no excuse for bundling every device driver under the sun when one has loadable modules. Device driver patches would then go to the device driver maintainer (in the same way, and for the same reason ,that Linus does not get Apache patches), offering some relief. Some core device drivers might remain Linus' responsibility, but he'd be more free to concentrate on architectural rather than implementation issues.

Yes, this would fork the kernel, in so much as there would be different bundled device driver sets. I see this no worse than the variety of existing distros, though I would like some central mirrored repository of what drivers were available.

Re:The Linux Fund?

[MisterBlister]

What makes you think he wants to?

I'm fairly positive that if he did want to he could find a Linux-related company to back him on this, despite the fact that many are in the crapper (RedHat would take him, IBM would probably pay him a full salary just to work on Linux, etc.)

Maybe he has interests outside of Linux? Ever even consider that?

Neither did Moses

[rjamestaylor]

That Linus doesn't scale reminds me of a passage from Exodus where Moses' father-in-law came and saw his success and also observed Moses judging between the people of Israel from morning to evening. Here's the passage:

It came about the next day that Moses sat to judge the people, and the people stood about Moses from the morning until the evening.
Now when Moses' father-in-law saw all that he was doing for the people, he said, ""What is this thing that you are doing for the people? Why do you alone sit as judge and all the people stand about you from morning until evening?''
Moses said to his father-in-law, "Because the people come to me to inquire of God." "When they have a dispute, it comes to me, and I judge between a man and his neighbor and make known the statutes of God and His laws.''
Moses' father-in-law said to him, ""The thing that you are doing is not good."
"You will surely wear out, both yourself and these people who are with you, for the task is too heavy for you; you cannot do it alone."
"Now listen to me: I will give you counsel, and God be with you. You be the people's representative before God, and you bring the disputes to God, then teach them the statutes and the laws, and make known to them the way in which they are to walk and the work they are to do."
"Furthermore, you shall select out of all the people able men who fear God, men of truth, those who hate dishonest gain; and you shall place these over them as leaders of thousands, of hundreds, of fifties and of tens."
"Let them judge the people at all times; and let it be that every major dispute they will bring to you, but every minor dispute they themselves will judge. So it will be easier for you, and they will bear the burden with you."
"If you do this thing and God so commands you, then you will be able to endure, and all these people also will go to their place in peace.''24 So Moses listened to his father-in-law and did all that he had said.
Moses chose able men out of all Israel and made them heads over the people, leaders of thousands, of hundreds, of fifties and of tens.
They judged the people at all times; the difficult dispute they would bring to Moses, but every minor dispute they themselves would judge.

Here's the Linux version:

It came about the next day that Linus sat to judge the patches, and the patches stood about Linus from the morning until the evening.
Now when Rob Landley saw all that he was doing for the patches, he said, ""What is this thing that you are doing for the patches? Why do you alone sit as judge and all the patches stand about you from morning until evening?''
Linus said toRob Landley, "Because the people send patches to me to fix the Kernel." "When they have a patch, it comes to me, and I judge between the patches and make known the statutes of the Kernel and its laws.''
Rob Landley said to him, ""The thing that you are doing is not good."
"You will surely wear out, both yourself and these people who are with you, for the task is too heavy for you; you cannot do it alone."
"Now listen to me: I will give you counsel, and lucky evolution be with you. You be the people's representative before the kernel, and you bring the patches to the Kernel, then teach them the statutes and the laws, and make known to them the way in which they are to code and the work they are to do."
"Furthermore, you shall select out of all the people able men who fear the Kernel, men of truth, those who hate dishonest gain (proprietary software); and you shall place these over them as leaders of thousands, of hundreds, of fifties and of tens."
"Let them judge the patches at all times; and let it be that every major patch they will bring to you, but every minor patch they themselves will judge. So it will be easier for you, and they will bear the burden with you."
"If you do this thing and lucky evolution so commands you, then you will be able to endure, and all these people also will go to their place in peace.'' So Linus listened to Rob Landley and did all that he had said.
Linus chose able men out of all LKML and made them heads over the people, leaders of thousands, of hundreds, of fifties and of tens.
They judged the patches at all times; the difficult patch they would bring to Linus, but every minor patch they themselves would judge.