Slashdot Mirror
The Evolution of Linux
Taiko writes: "Kerneltrap.org has posted some of the more interesting messages from a recent kernel mailing list discussion. It started with a post on proper indentation, but turned into something a bit more. There are some posts by Linus and Alan Cox about the nature of design, computer science, Linux development, evolution, and more. Quite interesting and funny."
The role of Linux in the history of computer science will turn out to be that Linux kept the Open Source model _open_ on the inevitable pathway to Technological Singularity.
Take for example the latest hot Linux gadget, the Sharp SL-5000D Zaurus PDA for Developers which runs both Linux and Java, and is therefore an appealing platform for the further development of Mind.JAVA Artificial Intelligence in the Linux environment -- everyman's last great hope of avoiding a catastrophic Microsoft take-over of the 'Net.
The world owes a lot to Linus Torvalds, Richard Stallman, Eric S. Raymond, Tim Berners-Lee and the countless other heroes of the Open Source futurity either posting here on SlashDot or toiling messianically away in obscurity.
I enjoy reading Linus' thoughts so much.
All around him, people try to make him or Linux more than it really is - and invariably, Linus brings it down a notch and puts it in perspective
It's amazing that this guy gets constantly hero-worshiped, his baby created billion dollars of wealth (at one point, at least), and yet just keeps his feet firmly planted
Compare that to the clowns that get high and mighty because they rUleZ at Quake, or on some IRC channel ... The geek community could learn a LOT from trying to emulate Linus' behaviour.
In all seriousness, would this article have been given a second glance if Linus wasn't involved? If I were to post a message saying "Hey, my friends and I were discussing the meaning of life after arguing about pencils, check out the log," I doubt a single editor on slashdot would have given it a 2nd glance. What kind of sick twist on celebrity worship is this?
Following that thread, can I now propose Linus' Law:
Any software system with a large enough user base can rely on the accumulated experience of its users to add features, and also picking ideas from smaller systems now and then (at a very low incremental effort).
Corollary. The onus is on the smaller players to come up with new features to distinguish themselves from the masses -- but ultimately it's no-win for them because their *really useful* ideas will be subsumed into more popular systems anyway
I need sleep and I'm quite possibly not thinking straight, but am I right in thinking this would create enormous pressures for specialized players like Sun and Apple (and Be, as they found out) in the long term?
If that is the case, where does that leave the "small is beautiful" rule? Does it mutate to "small is beautiful, provided you are part of a *big* idea that has incredible amounts of 'traction'"?
"Survival" is a very clear term in biology, it means
being able to keep yourself alive.
What does survival mean in software terms? Does
it mean that you make the most money (Microsoft?),
that you get to have the most users?, that you
endure in time and get written in textbooks?,
that you show clear technical superiority?
I think that any of these can be taken as
proof of "survival" of a software project, yet
the fact that MS-DOS lasted extremely long and
became extremely popular cannot possibly
mean that it is something we want to copy or
admire.
An argument that I would happily accept is that
evolution exists in linux-world as the result
of survival of different linux
ideas/implementations (e.g. new VM, new
low-latency etc) in the linux user subspace.
Now, the linux users space is a group of
technically aware people (?!) and evolution
of different linux variants in that space
can be said to be constructive in a technical
sense, thus producing real progress.
This process cannot universally guarantee
software quality (from a purely technical
standpoint)
P.
Science provides the tools that engineers use to build stuff.
Science provides a lot of dandy tools. Engineers like tools.
Engineers would be useless without science to provide new raw materials.
Baloney. We (engineers) were building all sorts of impressive stuff long before the invention of science. Check out the Great Pyramid and Yu the Great.
According to Rik:
Biological selection does nothing except removing the weak ones, it cannot automatically create systems which work well.
In short, I believe the biological selection is just that, selection. The creation of stuff will need some direction.
And I have to nod vigorously to that. Even taking the model of accelerated evolution through human breeding of species: you direct two animals together to breed. You don't just let the Ps, the F1s, the F2s, etc. just all wander around in a pen, have a sniper sitting on a post shooting the ones you don't want, and hoping the rest go at it...
The people claiming evolution to be a process to slow for software development seems to miss out on an important point. Measurement of evolution speed cannot be carried out in years. Evolution must be measured in lifecycles. The number of lifecycles needed for a program/snippet to evolve is about 1-20 lifecycles (releases) and by multiplying this with the time it takes for one lifecycle to complete you've got an approximat value of how fast computer programs evolve.
Another important point is that in this evolution - tough on som level about "survival of the fittest" - there is a certain level of continious "trial and error". This is in fact the way most programming - and learning - is done and this is done through the lifecycle. In real life, DNA can't remember actions carried on by their owners.
Look a monkey!
and there are better suited OSes for large amounts of CPUs
what is nice is precisely to have an OS that is scalable enough to be adapted to anything, and Linux is better at that game than any other OS.
But it's not the same kernel that runs on a SMP server than on a PDA. Not at all the same. That was the point of Linus, somehow: Linux is better because everybody has a need, and try to adapt it to that need, and as a result Linux becomes better.
Anyway, i think the conclusion will come by itself very soon, as for the fate of Java: for now, it's a Sun only thing. But the gcc team and others are working a lot on it. So, we will soon see who was right.
bollocks, no way this is insightful!!!
you don't understand the concepts of evolutiuon, and neither does mr. van riel.
biological selection (actually, the terminology is "natural selection") does not work by weeding out the weak ones. natural selection favours the multiplication of successfull ones (ie 'survival of the fittest').
the argument you (and rik van riel) are using, is essentially the same as most creationists use: mutation can only break down and not build up.
this is wrong. read some darwin before you comment on this stuff please.
regards,
meneer de koekepeer
Oh well...
the problem with that witty finsk is that he appaprently was forced to endure a few real bad CS classes back in Helsinki.
He's wrong, of course. Whatever works in Linux works because at some point somebody did some serious thinking before starting to spew out code. Planning data structures. Maybe even read about how others tackled the problem.
Thats called Design. In a few areas Linux serously lacks design. and it shows.
f.
Bridge building has also had tens of thousands of years of trial and error which surely helps. Another thing is competition: There's none in bridge building AFTER you've got the contract. Nobody is going to build another bridge next to it to see if they can make a better one.
When discussing the need for proper 'scientific' design, Alan Cox said:
"Engineering does not require science. Science helps a lot but people built perfectly good brick walls long before they knew why cement works."
To me, this seems to be a very poor analogy. The fact is that before the widespread use of maths and materials science in structual engineering ('building a good wall'), structual engineering didn't really exist at all - there were just builders and designers. 'Engineering' only really began when the science was added; before it was an art or trade. As for building a 'perfectly good wall', yes, the walls did indeed usually stand up; but:
a) Not always. Take the case of medaeval cathedrals. In order to stop the weight of the roof pushing the walls apart, the walls had flying buttresses built for support; however in some cases the buttresses actually were so big that they collapsed the wall in the other direction!
b) Not very efficiently. Due to the builders being unable to optimise their design, buildings were often very wasteful of materials in design.
c) How many medaeval skyscrapers were there? You just can't build many of todays huge structures without 'sciencey' engineering.
All in all, I think Alan would have been better advised not to compare it to building a wall; the problem is more that an operating system has such wide scope and enormous complexity (due to different areas of code affecting each other), as well as being flexible enough to change over time, that it isn't feasible to desgin the whole system as you would a dam or skyscraper.
Chris Cunningham
I wonder if Linux would have been the phenomena it is today if Linus had been brought up in evolution-bashing Kansas...
Yes, but he'd be saying "Linux isn't designed, the reason it works is because we have God on our side. We have lots of changes coming from people interested in pulling the project in different directions, including me, but He guides us to accept only the best final outcome. Look at Sun, they're doomed to failure because they try to follow their own narrow path instead of putting faith in the Lord..." etc. etc.
I have tremendous respect for Linux & co. with regards to software development, and it is always nice to see people who are not philosophers (or biologists) discussing philosophy (and biology).
However with respect to their opinions on philosophy (and biology), they are, as a previous poster commented, quite undergraduate. Actually I might be inclined to say worse about them, as I am self educated beyond High School, and I am aware of a much broader world of philosophy (and biology) than they seem to be.
Actually, it reminds me of nothing so much as Alan Cox' posturing on the DMCA, where my opinion was that people who do not understand such issues at all should refrain from making lawyerly or political comments in a broader public form where they have respect that is not meritted for the comments they are making.
While it is nice to see these people expressing interest in broader topics, I feel that they should keep their public discussions to the issues of which they have some understanding, namely software development. All that can come of their ponderings otherwise is to spread their ignorance further than they already have.
Even Slashdot wants to hide some things
Yeah, well, maybe if Linux used a microkernel, they wouldn't have these problems.
Why do so many people read Linus comments and try to simplify them further missing the point completely? Obviously When Linus says that the Kernel evolves through "sheer luck", he is not trying to say that the changes made to Linux were not intentional. Each individual believes his change is good and necessary, and many are... Linux and his support staff, like Alan, are there to audit the changes and ensure that the ones that matter get in. In the end, however, each change is not what moves Linux forward at the pace it does, it's the fact that moving all of the bits around finds "lucky" combinations that create sparks of genius which create new intentional changes that make some of those earlier changes, that at the time seemed like the most important, to become irrelevant in the light of the newest revelation.
Linux has gotten to it's current popularity levels (20% -40% of all server installations) because it's reliable and useful in the field. Statements like "You own linux as much as Linus does" might be academically interesting in a slashdot debate or a GNU checklist but the implications can radically affect Linux's future adoption.
I'm only trolling a little bit, just to make the point that the Linux kernel development model *will* be very different 2 years from now compared to what we're used to with the 2.0/2.2/early 2.4 series. That may or may not be a good thing, depending on how it works out. I'm unwilling to fallback on ESR dogma that open source solves all problems -- there's some real, substantial organizational questions which have been recently opened and have yet to play themselves out (this was the underlying impetus for the thread on l-k.)
I must say, the following quote from Linus from the article is one that strikes me with fear and awe:
When someone with the prestige that Linus has says something as powerful as this, I cannot help but feel that this topic is something that he is absolutely passionate about, much in the same way Stallman is passionate about Free Software. Linus doesn't seem like the type of person to use this sort of phrase on a whim; like he says, "I'm deadly serious".
You sound pretty paranoid.
Honestly you shouldn't be too worried. The shit _hasn't_ hit the fan, and 2.4.16 is ROCK solid. _Yes_, 2.4 took a long time to stabilize. It's there now, after the Van Riel vm was tossed aside. So lets cut the crap and call a spade a spade: 1) Linus is not a stable release maintainer. If linus puts out a kernel it needs to be tested. Linus does not put out release candidates. Only a fool would use a product that has been released without prior testing. 2) 2.4 took so long to stabilize because of the mistaken beleif that a BSD style VM was best for linux. 2.4 doesn't have the infrastructure to handle it (reverse memory mapping, etc). 3) 2.4.16 is a fucking great kernel. Except for a few possible bugs (the source tree is 149MB uncompressed!) I know of no problems whatsoever. 4) 2.5.x is already starting off with a bang. the new block/io layer should kick major ass, along with all the other enhancements planned. 5) Quit your whining. The sky isn't falling, alan cox isn't retiring to the hills to become a hermit, and linus torvalds knows what the fuck he's doing.
I think this whole discussion is a matter of perspective.
Of course Linus can say that Linux was not designed, when he is viewing it from this time. Im shure he didnt think the same way back when he was starting.
I mean, as a student, any geek here can tell you that, had he taken a task such as building a UNIX clone, he wouldve had at least some premises on which to base the creation of the software. Namely:
A) Architecture: i386
B) Version that we are cloning: In the spirit of POSIX and as close to it as I can.
C) It runs on my hardware with the "stuff" I can understand (say, the Minix filesystem which was the only fs he had acces to in terms of code)
Of course, most software projects run into changes of requirements and Linux was no exception, so the list of premises may grow, shrink and change but it doesnt mean that we cannot make a list and trace every element to someone's necessities.
So, here we run into the perspective part. Is this list something all kernhackers run and check to make their stuff, or is it a way of analizing the Linux's kernel evolution in time?
We can look back and say Linux was not designed to be what it is today. And we can say that what it is today is the result of evolution, the evolution of its design.
So I guess I agree whith the spirit of what Linus is saying. He is merely pointing out that he has no control over Linux in any way other than allowing or rejecting changes. And then, not even that (ask the TurboLinux guys or the USSS).
Now, is it good that requirements grow out of your hands? In a commercial environment it shure is bad since it messes up your planning and, ultimatley, you get less money because you didnt plan for this. But under the Open Source model of development? No, its not bad because it doesnt cost anyone.
Hell, I make money with Linux but I know for shure the kernel hackers arent doing anything so that I can have a nice working system. They do it because THEY want to have a nice working system. So if they mess up I may suffer a bit, but theyll suffer the most because its their money in the table.
I think if one understands this, one can make this piece of software into some dough,
Okay, im drifting now. The point is that Linux can be analyzed in terms of how has it evolved into what it is today. Noone can argue that it isnt a good production system for many situations so noone can say it's evolution has been a faliure.
So this whole argument about creationism Vs. evolutionism is a matter of the way you look at it. From one point you can see that Linux has had an evolution from its first design premises but that it does not lack a list of guidelines that constitute a design. From another point you can say that its a system that has evolved and that the fact that you can list some set of design premises does not mean the whole system HAS been designed to do what it does now.
Its a useless discussion anyway. Linux will keep doing what its doing no matter what we discuss here.
Alex
NO SIG
And I submit to you that design is inherent to evolution.
Evolution, in my view, is a process comprised of two cycling stages, as others have pointed out. Mutation is a random process, as random events cause (perhaps a number of) individuals in a species to develop a new trait. Selection is a process of deciding which "mutants" are able to reproduce and propagate.
In biology, is there decision-making in mutation? Depends on what kind of mutation. If a gamma ray snips a DNA molecule, there's no decision made there - it just happens. But decisions can affect mutation. DNA researchers and biologists create mutants in labs everyday. And as a society, we've accepted a technologically advanced quality of life that we know affects our environment and in turn affects us. What goes around comes around.
Decision making takes a more active role in selection and propagation. In anthropology, we measure evolutionary success generally by the number of viable offspring produced by the variation. That means that a successful variation of a species in a world of scarce resources (such as food and useful time) manages and allocates its resources in such a way that it is able to have more children than other variations of the species and thus have more influence on the future direction of the species. Successful management requires successful decision-making. Just try to manage without making a decision and you'll see. It doesn't matter if radioactive spiders turn whole packs of dogs into super-intelligent beings able to telepathically move fire-hydrants and build solid-gold toilets to drink out of - if those dogs decide to spend their time doing that and never have any puppies, they're an evolutionary dead-end. This is actually an issue that's been discussed in Anthro...people we see as being more more successful in our society are having fewer kids than less successful people...anyhow, we see that decision-making (and thus, design) is not mutually exclusive to evolution and in fact plays a large role in it.
In software, mutation could be described as a change to either the source code of a software "component" or the configuration of a collection of software components. Any such modification is a mutation of software, whether intentional or not. Most changes in software, for good or ill, are intentional. Some are caused by gamma rays hitting storage devices and flipping bits, but more are done on purpose as an act that serves some purpose (bug-killing, optimization, etc.) So there is a decision there to serve the purpose via change. There's also a decision to either let a modification stand (because it serves the purpose, or because reversing the change is not worth it), or to revert to the pre-modification state. The decision is there even if it's only to ignore the issue. Decision making and, by extension, design is present in the selection of software changes. You cannot separate design from software evolution, because you cannot separate the evaluation and decision making process from the software development process. Doing such a thing about amount to putting a million monkeys on a million consoles banging away and hoping Linux 3.0 magically results. Statistically it could happen, but animal control would have a cow.
Linus originally decided to go with Rik's VM code for 2.4, then later switched to Andrea's code. Neither move was decided by a coin toss. Evolution? Yes. Design? Yes. It's both, and why can't it be both?
I'll finish by quoting from "Modern C++ Design" by Andrei Alexandrescu (page 4):
-- John Truong