> What concerns me with BSD (not particularly
> FreeBSD) is that each distro forks the kernel
> and maintains only that fork. This is not unity.
In more than ten years of free BSDs this has actually happened only two times:
The NetBSD -> OpenBSD fork
That one basically happened, because the NetBSD people weren't very comfortable with the way Theo dealt with the users.
The FreeBSD -> Dragonfly fork
This is because, Mathew Dillon was not agreeing to the new roadmap for the FreeBSD 5 line and wanted to go a different way. Why should anybody try to stop him?
All BSDs still share a lot of common ground and code and idea exchanges happen:
NetBSD and OpenBSD have the same VM Subsystem (UVM), UFS2 changes have been ported from FreeBSD to NetBSD, rcorder(8) made its way into FreeBSD from NetBSD, OpenSSH is used almost everywhere and so on.
Dragonfly -- e.g. is a test baloon -- try out something new, without affecting, what's already working. To tell the truth, I think this is the best scientific way of doing something.
Why are there no Linux forks? I don't know, but there are many reasons that there should, I can only speculate, why there aren't any:
Forking is uncool, it means to have an opinion, that is against the masses of other developers. And you know that some Linux developers aren't the nicest people around, when you criticize their work
Forking is a major hazzle: The development is decentralized, coordinating common subsystems and other parts of two different, but similar systems would be a hazzle. Besides: You're out of the mainstream's "Linux is the next best thing(tm)" hype and that does not really comply with typical Linux developer philosophy, which is rather "do what's needed, so that the thing works" then "do what's probably right"
Besides, one must tell, that there are forks:
SuSE ships an maintains it's own kernel, so does Redhat and heaven knows how many companies, that distribute an embedded version of Linux somewhere.
But I do not really want to know, how many killed by (some stupid) policy like the Philips-USB-CAM support. How many drivers are out there, that worked with Linux 2.2 but won't any more, because something has changed in the vanilla kernel, but the driver never make in in there, so it just died? I really don't want to know...
Yes, and why do you think Linux has a good leader?
Quite interesting, I would never say that. You should perhaps take a reading turn on lkml archives with an eye on how the established developers base (those with some credit) have a tendency to put down developers with new ideas (and even code), just because those don't fit into their world view. Nobody has ever forked Linux from that so far and the problems still remain unsolved.
The model would work absolutely fine, if there would be no such phenomenons as inertia or development costs and if there would be an unlimited amount of people with technical expertise and the motivation to change things to the better even against the current system.
I do not know, where the idea, that an evolutionary approach to something will leed to an optimal result. This just is not true: In evolution the first solution that works and does not cause any substantial (which really means lethal here) problems, will find widespread use and will eventually even phase out other better, but latter solutions. The human knees and teeth are an example or is the combined digestal and respiratory throat part with most mammals. None of these are absolutely lethal or cause problems with reproduction, so they just stay put where they are, because they are there and they work reasonably well.
What was the reason, that prevented some other company at pushing out something better, but different than Windows? Welcome to the point.
But I have the strange fear that you will continue not to try to understand my argument, but rather counter-argument based on the same assumption, that I actually put into doubt.
Well, then Java obviously isn't cross-platform either: just look at sun.* and com.apple.*! What frightening inconsistency between different platforms and implementations!
I do not know, if you have read the official standard, but neither "sun.*", "com.apple.*" nor "com.microsoft.*" are part of it. The java standard only defines "java.*", "javax.*" and uses in some parts "org.omg.*" and "org.ietf.*". Nothing else is even mention there as standardized. Sun is not Microsoft and has a long history of defining open, platform-independent standards, just look at what probably 90% of Unix people are currently mounting their server side file-systems over.
You're just making the same tired, old argument that it takes a big, centrally managed process to set good standards.
You are probably getting me wrong there.
I just wanted to say, that neither approach actually solves the problem, that there is the requirement to some kind of leadership. In Linux development, this post is currently filled in more or less good way by Linus Torvalds. But the typical open source programmer simply ignores the question how some kind of leadership can and should be established. And most people even neglect the fact, that is is necessary. No problem has yet been solved by simply ignoring it and Adam Smith's theorem, that everybody can pull into his favorite direction and the net result is still benefit for the community also ignores some very important points in social theory.
You don't understand the bikeshed story if you think the solution to the problem it describes (which is a real problem) is to institute heavy-handed top-down decision making.
I know of no way to solve the bikeshed problem. I only said, that -- unfortunately -- it is also not solved by a bazaar like approach. heavy-handed top-down decision making (as you call it, I would not call the right kind of decision maker that way) does solve the problem for a designated leadership, it does not solve the problem on how to get the right kind of leadership. Plato himself did not answer the question on how to determine on who is the right guy to be in the decision-making position and the question is still unanswered.
All of the hardware detection problems are solved simply in other existing systems (Open/NetBSD). Why does everyone want a new, complicated solution to a problem that already been solved by someone else?
The answer is simple: This is Linux.
The explanation is a longer one, of course: Linux has a very long tradition of solving things, that have already been solved, a different way.
The scheduler, the vm (all versions), the driver interface (if you will call it such) have all been reimplementations of things, that have already been done. And most of the Linux reimplementations also reimplemented and gradually removed the errors the predecessors have made in their course of development. On such perspective Linux is in some terms a step-by-step view on the developers learning how to do something. And sometimes they learn something, somebody else already knew for a long time and sometimes they just get it wrong again.
It probably is a great deal because of developers wanting to leave their fingerprints, doing things their way and not giving up their freedom. Whatever the reasons behind this are, the traditional Linux way of doing things is to do things different. Guess, why Linux 2.6 drivers aren't IOKit compatible...
It works very well: most major open source OS'es implement POSIX and other common standards and systems like Perl, Python, Tcl, and Gnome run on dozens of different platforms. The APIs and functionality of those systems has been exceptionally stable. Sun Java is pitiful in comparison.
Of all these standards as you cite them, only Gnome has actually originiated from the community or a bazaar-like design approach.
POSIX has existed long before Open Source was such a heavily, community centred approach to software engineering. And to tell the truth: No operating system currently complies to POSIX. Partly this is, because POSIX is insufficient, partly this is, because there are things in POSIX nobody cares about to implement the POSIX-way (like realtime).
Perl, Python and Tcl are all designs of a very small group. Perl is still under the wings of Larry Wall, Python is in most places still Guido van Rossums principal work and I don't really want to know about the origins of Tcl.
There is a number huge of systems out there, were the community approach has utterly failed. There is the Linux DDI, there is the ever-changing glibc and there are things like/proc, that I do not really want to loose a word about.
If the involved programmers are so humble to step down and concile their own ideas and interests with that of the project, things get on very well.
If they don't do that, things get inconsistent and the design structure runs away. You may be able to hide this fact on the surface, but it's still there.
Full respect for the Gnome people. They do a very good job of keeping their API stable and sane, although some of their principal design decisions may not have been the optimal ones. That is, what I call good software engineering practice.
But very often, that does not work. When there is no one to step up and make a design decision or when the one is simply ignored or flamed to death by the people involved, things go wrong. And that's a problem, open source does not solve, but rather possibly increase, since everybody regardless of qualification will try to put down his fingerprint. Parkinson's bikeshed story was true in 1950 and it is still true today; neither the system in which it surfaces nor the people have changed.
"igb" is correct; in fact, some cathedrals have never been finished, even though they are quite useful and beautiful! Antonio Gaudi's La Sagrada Familia Cathedral in Barcelona is perhaps the perfect example of a fantastic structure that is taking centuries to construct!
A cathedral is typically being built by many architects of course. This is at least true by the fact that most of the architects did not live long enough to survive the completion of the building.
Normally although the successor of an architect has shared or at least continued the construction along the ideas and visions of his former colleague.
But in Europe there are also cathedrals, that have been built by a series of architects where one did not share the ideas and vision of the previous one. These normally have a longer building time (things were destroyed and rebuilt or changed) or a very inconsistent look. But I do not think, ESR has enough history knowledge to know about this;-).
The term
normally is of course important here, since there are cathedrals, that have different styles combined within them and are still very beautiful
Open-Source on the other hand generally tries to built a cathedral without an architect. Interestingly it works -- somehow, most of the time;-).
Linux works without purity because it's not designed to be pure. It's designed to be taken apart and reoutfittied as necessary.
The interesting thing is probably, that most of the issues, that arise when using Linux are fundamental, that is they are high top (you may call it deep bottom depending on viewpoint) design problems.
There are only some of those points where Programmer Joe with no clue has implemented something he did not understand in a variation of C that nobody else understands. These are rare, but they exist.
You will find the real problems at locations like the in the generic device-driver infrastructur, the USB bus-to-driver interface or even the VM-subsystem that is to closely tied to the rest of the kernel. Things constantly change there, they are not stable and nobody seems to be able to agree on how they are done correctly. The well-known bikeshed is calling in its tribute here.
The Bazaar approach seems very incapable to define standards, i.e. to reach an agreement on certain voluntary restrictions to their own freedom for the better of the project.
There are two ways to solve this problems:
The Lockeian (after John Locke) way, where everybody gives up a piece of his personal freedom for the public good.
The Hobbesian (after Thomas Hobbes) way, where you force everybody to comply to the rules.
Since the first way seems not to work well within most parts of the open source community (neither does it with any type of pure free market economy), Sun uses the second with the Java programming language specification. That may cause a major uproar, everytime it is discussed, but it results in the specification to clean, stable and actually looking to be made by someone having a remote clue of what he is doing.
The real way to go would be for everybody to learn, that it is necessary to abandon certain freedoms to reach a higher goals... for example to let a design to be done by a (small) comittee;-). But that surely is not true right now, so what are the options left?
There is a difference, between "the tools that are going to be developed" and "the tools that should be developed from a rational point of view".
A developer is basically somebody, that closes the semantic gap between the idea as it is presend in the human mind thinks and the code that has to be interpreted by the machine. A good tool is a tool, the reduces the burden of this process, so that more complex ideas can be realized.
Of course, not everybody has yet understood this simple fact, e.g. I frequently call him Linus "writing kernel code should not be simple" Torvalds in casual Linux design problem discussions.
But yet more problematic is the fact, that we have not really (yet?) understood, how the process works. Do you know how a program to be is initialially represented in your mind, before you start decomposing it into modules and even smaller parts drawing from learned knowledge and books which algorithms to apply and which techniques to use? I surely don't... the same way I do not actually know, how my brain composes a series of strange markings on a surface in to legible text.
Yes, neuro-psychology has made progress in understanding the inner workings of the human brain, but they are far from knowing how it really works.
But the worst part: Software designers and exspecially programmers have a long history of ignoring scientific progress, not only in their own area of expertise, but even more concerning other sciences.
All of the discoveries in natural language programm have been deliberatly ignored by newer languages such as C#. And languages like PHP and Perl are even inconsistent from a pure computer scientists point of view.
There are things, I would like to see:
easy design of self-learning user-interfaces
real natural language programming
globalized object stores
...
but most of these things are just not going to happen.
Either because somebody is to traditionalist about something or simply because clean, simple, standardized interfaces are definitely not what you get with such a heavily divers and varying area like current in-the-field programming.
Every Anti-Linux argument I have ever had (and some of them I have also started;-)) has -- at the some point of the discussion -- always directed onto two items by the Pro-Linux part:
- Linux ist faster - Linux is the only operating system that
will prevail
The first one is an argument, that many great people in computer science have found fundamentally broken for other types of software. The same guys, that state, that Microsoft has made a big design mistake by moving the graphics drivers from user space back into kernel space promote the exactly same design mistakes in the Linux kernel.
This is about the same argument, that people have made, that mono executes faster than.NET. The interesting thing here is, that mono does not implement the slightes security check throughout the complete runtime environment. Solaris may be slow (on = 4 CPUs), but that comes from a huge amount of checking code and locking.
L4Hurd is predicted to about 10% slower than Linux for a typical workload. I do not think, that this is an unacceptable price to pay for subsystems that do not compromise the whole kernel if they contain a buffer overflow.
The other argument is more subtle. It says two things: Linux is the best (which is surely not true _right now_) and that everything else will at some time be obsolete, when Linux has finally caught up in features and gotten better that the competition.
That's a nice one. It means "We want freedom, but we want it our way", very similar the ubiquitous call of standards. When developing the new driver interface for the 2.6 kernel series, the linux kernel developers had the choice between using two very good, already existing oo-driver interfaces (freebsd kclasses and darwin IOKit). They rather choose to implement their own version, incompatible to everything else and to improve it over time on their own. And they believe, implementing an interface badly first and improving it over time is good.
In my opinion, this way of thinking is fundamentally broken. An interface is a defintion of the way modules interact with one another. If it needs to be changed, that is always a big problem. Everybody depending on that interface will need to change his code. The argument from the Linux developers than is "We do not care to change our code and we do not care about everybody else's". In other words: "If you took trust in our interfaces you are a fool, give us your source and we will (probably, if we are in the right mood) fix it for you".
This is not distributed development and it surely it not freedom for anybody else than the "core" developers.
Slashdot Mirror
> FreeBSD) is that each distro forks the kernel
> and maintains only that fork. This is not unity.
In more than ten years of free BSDs this has actually happened only two times:
That one basically happened, because the NetBSD people weren't very comfortable with the way Theo dealt with the users.
This is because, Mathew Dillon was not agreeing to the new roadmap for the FreeBSD 5 line and wanted to go a different way. Why should anybody try to stop him?
All BSDs still share a lot of common ground and code and idea exchanges happen:
NetBSD and OpenBSD have the same VM Subsystem (UVM), UFS2 changes have been ported from FreeBSD to NetBSD, rcorder(8) made its way into FreeBSD from NetBSD, OpenSSH is used almost everywhere and so on.
Dragonfly -- e.g. is a test baloon -- try out something new, without affecting, what's already working. To tell the truth, I think this is the best scientific way of doing something.
Why are there no Linux forks? I don't know, but there are many reasons that there should, I can only speculate, why there aren't any:
Besides, one must tell, that there are forks: SuSE ships an maintains it's own kernel, so does Redhat and heaven knows how many companies, that distribute an embedded version of Linux somewhere.
But I do not really want to know, how many killed by (some stupid) policy like the Philips-USB-CAM support. How many drivers are out there, that worked with Linux 2.2 but won't any more, because something has changed in the vanilla kernel, but the driver never make in in there, so it just died? I really don't want to know ...
Yes, and why do you think Linux has a good leader?
Quite interesting, I would never say that. You should perhaps take a reading turn on lkml archives with an eye on how the established developers base (those with some credit) have a tendency to put down developers with new ideas (and even code), just because those don't fit into their world view. Nobody has ever forked Linux from that so far and the problems still remain unsolved.
The model would work absolutely fine, if there would be no such phenomenons as inertia or development costs and if there would be an unlimited amount of people with technical expertise and the motivation to change things to the better even against the current system.
I do not know, where the idea, that an evolutionary approach to something will leed to an optimal result. This just is not true: In evolution the first solution that works and does not cause any substantial (which really means lethal here) problems, will find widespread use and will eventually even phase out other better, but latter solutions. The human knees and teeth are an example or is the combined digestal and respiratory throat part with most mammals. None of these are absolutely lethal or cause problems with reproduction, so they just stay put where they are, because they are there and they work reasonably well.
What was the reason, that prevented some other company at pushing out something better, but different than Windows? Welcome to the point.
But I have the strange fear that you will continue not to try to understand my argument, but rather counter-argument based on the same assumption, that I actually put into doubt.
Well, then Java obviously isn't cross-platform either: just look at sun.* and com.apple.*! What frightening inconsistency between different platforms and implementations!
I do not know, if you have read the official standard, but neither "sun.*", "com.apple.*" nor "com.microsoft.*" are part of it. The java standard only defines "java.*", "javax.*" and uses in some parts "org.omg.*" and "org.ietf.*". Nothing else is even mention there as standardized. Sun is not Microsoft and has a long history of defining open, platform-independent standards, just look at what probably 90% of Unix people are currently mounting their server side file-systems over.
You're just making the same tired, old argument that it takes a big, centrally managed process to set good standards.
You are probably getting me wrong there.
I just wanted to say, that neither approach actually solves the problem, that there is the requirement to some kind of leadership. In Linux development, this post is currently filled in more or less good way by Linus Torvalds. But the typical open source programmer simply ignores the question how some kind of leadership can and should be established. And most people even neglect the fact, that is is necessary. No problem has yet been solved by simply ignoring it and Adam Smith's theorem, that everybody can pull into his favorite direction and the net result is still benefit for the community also ignores some very important points in social theory.
You don't understand the bikeshed story if you think the solution to the problem it describes (which is a real problem) is to institute heavy-handed top-down decision making.
I know of no way to solve the bikeshed problem. I only said, that -- unfortunately -- it is also not solved by a bazaar like approach. heavy-handed top-down decision making (as you call it, I would not call the right kind of decision maker that way) does solve the problem for a designated leadership, it does not solve the problem on how to get the right kind of leadership. Plato himself did not answer the question on how to determine on who is the right guy to be in the decision-making position and the question is still unanswered.
All of the hardware detection problems are solved simply in other existing systems (Open/NetBSD). Why does everyone want a new, complicated solution to a problem that already been solved by someone else?
The answer is simple: This is Linux.
The explanation is a longer one, of course: Linux has a very long tradition of solving things, that have already been solved, a different way.
The scheduler, the vm (all versions), the driver interface (if you will call it such) have all been reimplementations of things, that have already been done. And most of the Linux reimplementations also reimplemented and gradually removed the errors the predecessors have made in their course of development. On such perspective Linux is in some terms a step-by-step view on the developers learning how to do something. And sometimes they learn something, somebody else already knew for a long time and sometimes they just get it wrong again.
It probably is a great deal because of developers wanting to leave their fingerprints, doing things their way and not giving up their freedom. Whatever the reasons behind this are, the traditional Linux way of doing things is to do things different. Guess, why Linux 2.6 drivers aren't IOKit compatible ...
Of all these standards as you cite them, only Gnome has actually originiated from the community or a bazaar-like design approach.
There is a number huge of systems out there, were the community approach has utterly failed. There is the Linux DDI, there is the ever-changing glibc and there are things like /proc, that I do not really want to loose a word about.
If the involved programmers are so humble to step down and concile their own ideas and interests with that of the project, things get on very well. If they don't do that, things get inconsistent and the design structure runs away. You may be able to hide this fact on the surface, but it's still there.
Full respect for the Gnome people. They do a very good job of keeping their API stable and sane, although some of their principal design decisions may not have been the optimal ones. That is, what I call good software engineering practice.
But very often, that does not work. When there is no one to step up and make a design decision or when the one is simply ignored or flamed to death by the people involved, things go wrong. And that's a problem, open source does not solve, but rather possibly increase, since everybody regardless of qualification will try to put down his fingerprint. Parkinson's bikeshed story was true in 1950 and it is still true today; neither the system in which it surfaces nor the people have changed.
"igb" is correct; in fact, some cathedrals have never been finished, even though they are quite useful and beautiful! Antonio Gaudi's La Sagrada Familia Cathedral in Barcelona is perhaps the perfect example of a fantastic structure that is taking centuries to construct!
Brooks has some interesting points on this in The Mythical Man Month:
A cathedral is typically being built by many architects of course. This is at least true by the fact that most of the architects did not live long enough to survive the completion of the building. Normally although the successor of an architect has shared or at least continued the construction along the ideas and visions of his former colleague.
But in Europe there are also cathedrals, that have been built by a series of architects where one did not share the ideas and vision of the previous one. These normally have a longer building time (things were destroyed and rebuilt or changed) or a very inconsistent look. But I do not think, ESR has enough history knowledge to know about this ;-).
Open-Source on the other hand generally tries to built a cathedral without an architect. Interestingly it works -- somehow, most of the time ;-).
Linux works without purity because it's not designed to be pure. It's designed to be taken apart and reoutfittied as necessary.
The interesting thing is probably, that most of the issues, that arise when using Linux are fundamental, that is they are high top (you may call it deep bottom depending on viewpoint) design problems.
There are only some of those points where Programmer Joe with no clue has implemented something he did not understand in a variation of C that nobody else understands. These are rare, but they exist.
You will find the real problems at locations like the in the generic device-driver infrastructur, the USB bus-to-driver interface or even the VM-subsystem that is to closely tied to the rest of the kernel. Things constantly change there, they are not stable and nobody seems to be able to agree on how they are done correctly. The well-known bikeshed is calling in its tribute here.
The Bazaar approach seems very incapable to define standards, i.e. to reach an agreement on certain voluntary restrictions to their own freedom for the better of the project.
There are two ways to solve this problems:
Since the first way seems not to work well within most parts of the open source community (neither does it with any type of pure free market economy), Sun uses the second with the Java programming language specification. That may cause a major uproar, everytime it is discussed, but it results in the specification to clean, stable and actually looking to be made by someone having a remote clue of what he is doing.
The real way to go would be for everybody to learn, that it is necessary to abandon certain freedoms to reach a higher goals ... for example to let a design to be done by a (small) comittee ;-). But that surely is not true right now, so what are the options left?
A developer is basically somebody, that closes the semantic gap between the idea as it is presend in the human mind thinks and the code that has to be interpreted by the machine. A good tool is a tool, the reduces the burden of this process, so that more complex ideas can be realized.
Of course, not everybody has yet understood this simple fact, e.g. I frequently call him Linus "writing kernel code should not be simple" Torvalds in casual Linux design problem discussions.
But yet more problematic is the fact, that we have not really (yet?) understood, how the process works. Do you know how a program to be is initialially represented in your mind, before you start decomposing it into modules and even smaller parts drawing from learned knowledge and books which algorithms to apply and which techniques to use? I surely don't ... the same way I do not actually know, how my brain composes a series of strange markings on a surface in to legible text.
Yes, neuro-psychology has made progress in understanding the inner workings of the human brain, but they are far from knowing how it really works.
But the worst part: Software designers and exspecially programmers have a long history of ignoring scientific progress, not only in their own area of expertise, but even more concerning other sciences.
All of the discoveries in natural language programm have been deliberatly ignored by newer languages such as C#. And languages like PHP and Perl are even inconsistent from a pure computer scientists point of view.
There are things, I would like to see:
- easy design of self-learning user-interfaces
- real natural language programming
- globalized object stores
- ...
but most of these things are just not going to happen.Either because somebody is to traditionalist about something or simply because clean, simple, standardized interfaces are definitely not what you get with such a heavily divers and varying area like current in-the-field programming.
Every Anti-Linux argument I have ever had (and some of them I have also started
- Linux ist faster
- Linux is the only operating system that
will prevail
The first one is an argument, that many great people in computer science have found fundamentally broken for other types of software. The same guys, that state, that Microsoft has made a big design mistake by moving the graphics drivers from user space back into kernel space promote the exactly same design mistakes in the Linux kernel.
This is about the same argument, that people have made, that mono executes faster than
L4Hurd is predicted to about 10% slower than Linux for a typical workload. I do not think, that this is an unacceptable price to pay for subsystems that do not compromise the whole kernel if they contain a buffer overflow.
The other argument is more subtle. It says two things: Linux is the best (which is surely not true _right now_) and that everything else will at some time be obsolete, when Linux has finally caught up in features and gotten better that the competition.
That's a nice one. It means "We want freedom, but we want it our way", very similar the ubiquitous call of standards. When developing the new driver interface for the 2.6 kernel series, the linux kernel developers had the choice between using two very good, already existing oo-driver interfaces (freebsd kclasses and darwin IOKit). They rather choose to implement their own version, incompatible to everything else and to improve it over time on their own. And they believe, implementing an interface badly first and improving it over time is good.
In my opinion, this way of thinking is fundamentally broken. An interface is a defintion of the way modules interact with one another. If it needs to be changed, that is always a big problem. Everybody depending on that interface will need to change his code. The argument from the Linux developers than is "We do not care to change our code and we do not care about everybody else's". In other words: "If you took trust in our interfaces you are a fool, give us your source and we will (probably, if we are in the right mood) fix it for you".
This is not distributed development and it surely it not freedom for anybody else than the "core" developers.