The Evolution of Linux

Taiko writes: "Kerneltrap.org has posted some of the more interesting messages from a recent kernel mailing list discussion. It started with a post on proper indentation, but turned into something a bit more. There are some posts by Linus and Alan Cox about the nature of design, computer science, Linux development, evolution, and more. Quite interesting and funny."

Comments

Great stuff!

[Prop]

I enjoy reading Linus' thoughts so much.

All around him, people try to make him or Linux more than it really is - and invariably, Linus brings it down a notch and puts it in perspective

It's amazing that this guy gets constantly hero-worshiped, his baby created billion dollars of wealth (at one point, at least), and yet just keeps his feet firmly planted

Compare that to the clowns that get high and mighty because they rUleZ at Quake, or on some IRC channel ... The geek community could learn a LOT from trying to emulate Linus' behaviour.

Re:Great stuff!

[akc]

Linus claims that Linux has no guided direction and developes purely through evolution and luck. First off, I would view this as highly insulting if I was a major player like IBM, or even someone who has only minimally contributed. He is basically saying that these people are as useful as a random code generator. Even more importantly, his statement is not true.

I don't think he was saying that at all. In IBMs case they have a direction they are pushing and develop code in that area, but at the same time there is a vast array of people taking it in different directions. The net result is unplanned (as opposed to random).

The analogy to selective breeding is wrong. Yes, we can speed up evolution through selective breeding, but we are only changing minor traits. Sure, you can breed a dog with long hair, a short snout, and good temperment, but what if you want to breed a dog with feathers, or a fifth leg? At the very best, that would take an incredible amount of time. The better solution is to research and apply genetics. Lets apply that to the kernel... we can either let the scsi mid layer slowly evolve into something useful, or we can sit down and give it a good design phase and have something that works in a much shorter period of time.

Firstly, the lifecycle time of the kernel is down to a few days instead of years - secondly things do evolve - just look at the progress of the VM (either one). First attempt didn't get it quite right, so then there are some patches and things get a bit better, but something else is bust (etc etc). This seems quite close to the breeding approach (but is only one of a number of parallel directions for the kernel).

Windows does not succeed because of evolution and a deep gene pool. Windows succeeds because of 1) marketing, 2) aggressive business tatics against competitors, and 3) it's not so buggy that it's totally unusable.

I don't think Windows succeeded because of evolution either - it was a major mutation which occurred at a certain lucky point in history and wiped out most of its competitors. [Sure marketing and business tactics helped - but the real winner was the GUI interface (against DOS) and the fact that apple didn't open up their hardware whereas IBM did]. Don't we have something like this in the natural world? [My brain is addled with the thought of the mutant in Asimov's Foundation series]. The big question though is - in the long term will it continue to evolve fast enough to keep up when pressed with alternative species (like linux and the speed with which it is evolving!)

Re:Great stuff!

[Mr Z]

The main point you missed, though, is that Linus is right at the macro level -- there is no overall design process for Linux as a system or an overall direction for Linux.

At the kernel subsystem level, there's plenty of design, and plenty of goals, and plenty of localized direction. In the filesystem space, there was a lot of buzz around journalling filesystems. In the MM department, we had something more akin to controlled chaos... :-) And yes, the SCSI layer could use some actual careful design work.

There was no overarching goal "We must optimize for market X" that drove any of this. Sure, some people want to run Linux on huge machines, and so they want journalling. Other people want to shove Linux into wristwatches and PDAs, and so they instead want to focus on memory footprint. And still others care about interrupt latency over throughput. So, each little care-about niche as their own little projects that pull Linux in lots of different directions at the macro level. Each individual project is very directed, and some have significant design work. But none of it is directed from On High as part of the Grand Plan for The System.

--Joe

Too much back patting..

[LordOfYourPants]

In all seriousness, would this article have been given a second glance if Linus wasn't involved? If I were to post a message saying "Hey, my friends and I were discussing the meaning of life after arguing about pencils, check out the log," I doubt a single editor on slashdot would have given it a 2nd glance. What kind of sick twist on celebrity worship is this?

Ecosystem biased against small players?

[cygnusx]

Reading this lkml thread, I had the distinct feeling that you could replace Sun with Apple in Linus' posts and much of it would still be true.

You heard them above. Sun is basically inbreeding. That tends to be good
to bring out specific characteristics of a breed...

Following that thread, can I now propose Linus' Law:

Any software system with a large enough user base can rely on the accumulated experience of its users to add features, and also picking ideas from smaller systems now and then (at a very low incremental effort).

Corollary. The onus is on the smaller players to come up with new features to distinguish themselves from the masses -- but ultimately it's no-win for them because their *really useful* ideas will be subsumed into more popular systems anyway ... only a matter of time.

I need sleep and I'm quite possibly not thinking straight, but am I right in thinking this would create enormous pressures for specialized players like Sun and Apple (and Be, as they found out) in the long term?

If that is the case, where does that leave the "small is beautiful" rule? Does it mutate to "small is beautiful, provided you are part of a *big* idea that has incredible amounts of 'traction'"?

Re:Ecosystem biased against small players?

[Ami Ganguli]

I'm not sure the number of users is important so much as the number of developers/contributers. Or if Linus is correct, the number of developers with different agendas.

In fact the whole debate starts to sound a bit like ESR's Cathedral & Bazaar.

Re:science and engineering

[the eric conspiracy]

Science provides the tools that engineers use to build stuff.

Science provides a lot of dandy tools. Engineers like tools.

Engineers would be useless without science to provide new raw materials.

Baloney. We (engineers) were building all sorts of impressive stuff long before the invention of science. Check out the Great Pyramid and Yu the Great.

I don't agree completely with Linus

[shinji1911]

According to Rik:

Biological selection does nothing except removing the weak ones, it cannot automatically create systems which work well.

In short, I believe the biological selection is just that, selection. The creation of stuff will need some direction.

And I have to nod vigorously to that. Even taking the model of accelerated evolution through human breeding of species: you direct two animals together to breed. You don't just let the Ps, the F1s, the F2s, etc. just all wander around in a pen, have a sniper sitting on a post shooting the ones you don't want, and hoping the rest go at it...

crap!!!

[koekepeer]

bollocks, no way this is insightful!!!

you don't understand the concepts of evolutiuon, and neither does mr. van riel.

biological selection (actually, the terminology is "natural selection") does not work by weeding out the weak ones. natural selection favours the multiplication of successfull ones (ie 'survival of the fittest').

the argument you (and rik van riel) are using, is essentially the same as most creationists use: mutation can only break down and not build up.

this is wrong. read some darwin before you comment on this stuff please.
regards,

meneer de koekepeer

Powerful words

[ftobin]

I must say, the following quote from Linus from the article is one that strikes me with fear and awe:

Try to prove me wrong.

When someone with the prestige that Linus has says something as powerful as this, I cannot help but feel that this topic is something that he is absolutely passionate about, much in the same way Stallman is passionate about Free Software. Linus doesn't seem like the type of person to use this sort of phrase on a whim; like he says, "I'm deadly serious".

Re:Heh

[barneyfoo]

You sound pretty paranoid.

Honestly you shouldn't be too worried. The shit _hasn't_ hit the fan, and 2.4.16 is ROCK solid. _Yes_, 2.4 took a long time to stabilize. It's there now, after the Van Riel vm was tossed aside. So lets cut the crap and call a spade a spade: 1) Linus is not a stable release maintainer. If linus puts out a kernel it needs to be tested. Linus does not put out release candidates. Only a fool would use a product that has been released without prior testing. 2) 2.4 took so long to stabilize because of the mistaken beleif that a BSD style VM was best for linux. 2.4 doesn't have the infrastructure to handle it (reverse memory mapping, etc). 3) 2.4.16 is a fucking great kernel. Except for a few possible bugs (the source tree is 149MB uncompressed!) I know of no problems whatsoever. 4) 2.5.x is already starting off with a bang. the new block/io layer should kick major ass, along with all the other enhancements planned. 5) Quit your whining. The sky isn't falling, alan cox isn't retiring to the hills to become a hermit, and linus torvalds knows what the fuck he's doing.