Slashdot Mirror
Open Source Projects Manage Themselves? Dream On.
cconnell writes: "One of the most tantalizing statements about open source development is that these projects manage themselves. Gone are layers of do-nothing managers with bloated bureaucracies and interminable development schedules. In their place is a new paradigm of self-organizing software developers with no overhead and high efficiency. There is just one problem with this assertion -- it is not true. This article shows that open source projects are about as far as you can get from self-organizing. In fact, these projects use strong central control, which is crucial to their success. As evidence, I examine Raymond's fetchmail project and Linus Torvalds's work with Linux."
I agree with his central argument, that structure is key, but I'd drive it even further. I'd say that the amount of necessary structure is directly proportional to the amount you can get away with it. Linux and other similar projects, due to their high degree of modularization or lack of size and scope, can afford a relatively loose structure. I do not believe that significantly more involved projects (i.e., development of a high end RDBMS) can afford any of the commonly praised open source management styles.
Though I believe Open Source may some day make great inroads against proprietary software, I think that'll come almost inevitably where structured is lent by those with resources. i.e., corporations. For instance, I briefly read an article in some IT magazine awhile ago that IBM, Redhat, VALinux, and few other corporations were going to build a PARC-like facility where they'd cooperate in the development and improvement of Linux with the intent to create a universal replacement of windows. Assuming the corporations can agree on committing the necessary resources, hiring enough programmers, giving the right amount of force, I could see it succeeding beatifully. But I don't believe we'll ever see Linux, as it is popularly envisioned, usurping the windows hegemony.
The difference between an open-source management model and a traditional management model is that the management in this case comes from the most expert software developers in the group. Raymond, being the primary force behind Fetchmail, and Torvalds, being the primary force behind the Linux kernel, are not traditional management types at all because they get their hands dirty, and they have technical expertise.
I don't want to insult anyone, but generally, people who have Computer Science/Management based degrees DO NOT know as much as people with Computer Science/Engineering degrees. They are taught to manage based on traditional techniques of business management, techniques which generally detract from the overall usefulness and efficiency of a project. They obtain a small amount of technical knowledge, enough to become merely adequate at their field, enough that they are not ignorant of the goals and ideas of a product (not entirely ignorant, anyway), yet they would be completely confused if actually asked to develop a worthwhile piece of the puzzle.
The open source projects mentioned in the Cathedral and the Bazaar, i believe, are similar to traditional projects in that they require orginization and central administration to become useful, but they differ in that people who don't understand the development process are not welcome to participate.
I'll just leave you with this: in our society, actual work is done by a large amount of people; products are produced by a large amount of people, etc. Yet the small few at the top who do not actually produce any products; who do not know how or want to actually work for a living; who attempt to administrate despite their complete lack of knowledge about what is actually produced; those people retain the largest cut of the money and hand out the rest to everyone else who did something. Is there something wrong with this societal model??
Always. Honestly though most of the geek coders I know supposedly suffer from A.D.D. (or so they were told in school). Breaking my concentration into several tasks at once with only one or two being important helps me concentrate.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
No, but I will now! Very interesting.
Your point is entirely valid; however, in general (i.e. not at every company) shareholders and management benefit each other, and help the employee very little. IPOs usually happen when a company is only becoming large, not when it's already a corporate monster, so an IPO for instance probably would be very nice for all of the employees. But if you go work for a company that's already really well established, I doubt you'll see a cent of any of that wonderful management money.
At this time, wouldn't Gnutella be considered the best example of a bazaar project? Given up *forceablely, I might add* by it's creators, and now steered by a collection of people attempting to add new platforms and options. It would be interesting to see how Gnutella would have developed with the original people guiding the project, as opposed to how it is currently developing.
Thanks for the clarification. I wasn't making the point you thought, but you told me stuff I didn't know, so I came out a winner. The point I was trying to make was that if you consider the content of any typical linux distro disc, the software came from many places micro-managed many different ways. Taken as a whole, the wad is a completely decentralized unmanaged tangle that works as perfectly together as any software does.
My take on it is: ESR had a valid point. However he doesn't understand how cathedrals were made, or what a bazaar actually is, and so he chose the wrong metaphor. I'm tempted to assign this to religious prejudice, but it's probably just ignorance.
yes! That paper was making its point so poorly I couldn't keep reading it. Religious prejudice? How about "ordinance toting, gun stroking, anarcho-somethism" :)
Also in my defense for always being on Slashdot..
:)
The boss is usually on too..
I keep up-to-date by watching Slashdot and other key sites which is important for sysadmins and coders I think. Sometimes I need to know something happened (like security alerts) within minutes of the rest of the world knowing it.
At what price learning? At what cost wisdom? The price is a man's peace of mind, and the cost is his life.
businessmen. They found out pretty quickly that it takes more than coding skill to run a business, your competitors will mop the floor with you
yeah, but we're talking about individual projects here, not businesses. Nobody said that coders should run the business, just the project.
Politics, Culture, Food?
Yeah, I have a tendency to over-parens (or whatever) when I am writing (typing, e-mailing (messaging of any kind, really)). Just a habit I got into (can't remember when) - maybe when I was programming in LISP? Of course then all of the words in the sentence would be in a different order...
--
"It's tough to be bilingual when you get hit in the head."
You know, there will be plenty of flames (esp. on Slashdot) about this article, but there's one thing to keep in mind:
:^) Linux is held together with excellent management skills. I've even seen statements from ESR to that effect.
He's absolutely right.
ESR's description of how Linux was developed isn't really accurate. Many folks already knew it (or, like myself, have read it in other sources.
One could state that this is merely semantics, but it's not entirely correct. It's true that people were not part of some corporation like IBM or Compaq; they were mostly just people who wanted/needed a feature/fix for the kernel, and did it. But that kernel growth/improvement was *managed*, not just some big cloud of code that condensed on a server somewhere.
Stating on Slashdot that I like cheese since 1997.
Hello,
:)
The link in your signature is broken, it should lead to http://www.nader2000.net/issues/c orporations.html
As a side note, Nader's coming to my campus in about a week. I think this is the first time I've ever actually felt a bit of excitement for a candidate
-Denor
Oh, but you do say what I claim you say... Here is a quote from CatB:
... software project management has five functions: 1) To define goals and keep everybody pointed in the same direction. 2) To monitor and make sure crucial details don't get skipped. 3) To motivate people to do boring but necessary drudgework. 4) To organize the deployment of people for best productivity. 5) To marshal resources needed to sustain the project. Apparently worthy goals, all of these; but under the open-source model, and in its surrounding social context, they can begin to seem strangely irrelevant. We'll take them in reverse order..."
"
You then spend considerable time arguing that each of these functions is not needed in the open source model; that they happen on their own.
When TCP/IP came along, though, this approach was abandoned to great extent. And when Unix GUI programming came along, it was totally abandoned -- there's no such thing as an "X" component.
And finally, Microsoft did the world a grande disservice by deciding that no, threads of execution did not need to be firewalled off from each other so that stray pointers could not crash the entire system, let's run them all in the same memory space! Multi-threaded programs are now the norm, and are a maintenance nightmare. Components in a multi-threaded program can no longer be tested in abstraction, they must be tested amidst the object environment that they were designed for. Successfully writing a component in a multi-threaded program is not just a case of writing some routines to process a stdin and stdout file handle, you now must know about the stuff in the global shared object depository, and how to arbitrate access to them.
A project I'm working on started out as a Windows-style multi-threaded program. It swiftly became obvious to me, however, when my office-mate (an excellent programmer) could not understand my architecture, that the chances of less-than-stellar programmers ever coming into the project and contributing were next to nil. Thus the project was re-written as a Unix-style program rather than a Windows-style program. While newcomers to the project need to have various services running before their programs will work, the startup learning time is greatly reduced, and their components can be written in isolation without me worrying that their component is going to break *MY* component. And on modern systems, this isn't even slow. Granted, sometimes we have 10 processes running (a plumbing nightmare!), but user response is still quite snappy.
Threads: Just Say No.
-E
Send mail here if you want to reach me.
I don't see much wrong with the model. If you're lucky enough to work for someone who pays you well, you may not feel so underpaid. On the other hand, the leader in a company takes risks, and can loose it all. The reward for that risk can and should be compensated to encourage more risk taking and foster economic growth. Being a business leader means you're the last one to get payed, and that's worth a lot. There are some other options.
1) Move to a socialist nation.
or
2) Start your own business.
Ah, but an important point (missed by the author of the original article) is that these are the first benign monarchies to have revolution built directly into the system.
Most people have pointed out the significant difference between having centralized control that is assigned (as in the cathedral model) and centralized control that emerges (as in the working of most successful Open Source projects, whether or not this is truly the 'bazaar' that Raymond describes.) But the Open Source model has another safeguard, should the existing 'monarchy' cease functioning effectively, and that is the freedom of anyone to emerge as a new center of control, by taking the source and starting in a new direction.
Whether such an attempt succeeds is a different story. Quite naturally, it faces all the same obstacles that the current "monarch" faced to rise to their position (Do they know their technical stuff? Do they have the organizational and people skills to coordinate volunteers? Do they have a vision which they can communicate to enroll others?) and a few more besides (Can they convince others that they represent a change for the better, rather than just 'a change'? Have the circumstances that caused the split also caused hard feelings, and if so, can they be overcome?) But the fact that it always exists in potentia heads off the classic dilemma of centralization, that the whole system could be paralyzed by neutralizing the center.
It's almost certainly true that Open Source projects do not 'manage themselves' but they redefine 'manage' itself from "tell everybody What Needs To Be Done and Who Will Do It, and Lean On Everyone To Get Their Part Done" to "toss out good ideas to people you encourage, and coordinate what comes back to you as a result." And ... not to belabor a point that's already been made, but unlike the business world, no one will ever, ever be made the manager of a crucially important project just because he's the one with the nicest hair. ^_^
If people are to respect the law, perhaps the law should begin by respecting the people.
Generally, there are many ways to self
organize. If you have a community developing
something, it makes sense for the more
able people to take charge. This
technocracy is largely what is meant by
"no managers", read "no clueless PHBs".
In OSS projects people manage/lead
by example, and motivation is less important
because people are free to join and leave,
so if they are there, you can assume they are
motivated.
I don't like ESR's shallow writings, but there
definitely are differences between how management
is done in closed and open environment.
Lastly, to suggest that OSS projects are
always managed by one person is wrong.
Very often a junta is in charge, as in BSDs, KDE
and others. This is similar
to internet's DNS scheme among others.
It does offer variety of approaches typically
associated with pure bazaar and also allows
some coordination.
You can also have Debian style democracy. It
works slower but it works. Debian is a bit
of a junta system, but it disguises itself as a
democracy (inevitable so long as participants
have varying skills and degrees of competency).
Linux now has a few people with enough trust.
The benevolent dictator model will most likely
not outlast Linus.
Are you talking about bugs? This isn't an accurate statement at all concerning bugs. When an engineer hears about a bug, his first reaction is not to deny it's existance, but to fix it. He (or She) want's his 'baby' to be perfect. That's my experience, at least.
Are you talking about major architectural changes? Certainly you shouldn't be asking for major changes if you aren't a major developer. You wouldn't know what you were talking about.
If the problem is that the developer is overworked and can't devote enough time to your particular fix, that's a different issue entirely. Don't make it sound like the developer doesn't want to fix bugs.
Got HTML? Want LaTeX? Try html2latex
"Anyone that has ever gotten an idea based on any of my work and done something better with it-good for you."--J.Carmack
When anyone starts a new project they more likely than not began due to a personal interest.
This person will obviously be contributing the most to the project at the start. Although anyone is free to fork the project off, most people will not see a need to re-invent the wheel unless that is what is necessary.
Now since the originator of the project is probably spending the most time on the code, they become a de-facto manager. We see, however , there is no requirement for people to follow the "manager's" lead, the GPL allows for this.
The point is although the current development model of most popular free software seems like there is a heavy hand controlling the process cycle.That hand usually has indirectly been elected to lead, and will most likely continue to lead since most people like to suggest/contribute ideas/changes than go and branch off softwareIf desired a new branch can be forked, perhaps this branch will become more popular than the original, and the forker now becomes the de-facto manager, and the originator, nothing more than a contributor.
Or something like that
There are, I think two examples of open source extremes:
First, look at Mozilla: everyone knows that a browser is really one of those tools which are very important in today's desktop environment. But instead of planning and developing a fast, stable, usable one, Mozilla just implements funny, but useless things like a terminal emulator. Don't get me wrong: everybody should do whatever he wants to, but sometimes I think it would be better if we had a browser at first and then think about which nice applets could enhance it.
Another related aspect is the fact that there are so many new immature browsers (look at "Express", "Gzilla", "Mnemonic"). The developers working on these project should work together, and together with the Konqueror/KDE developers and, of course with the Mozilla people in my opinion.
Second, look at the various BSD projects: they are organized tightly, with people doing lot of work, people who make decisions and people that implement and test new features. I don't think that any of the other large open source projects is organized as good as the BSD projects.
While this is a marvelous attitude (and an appealing one for me, coder-in-training), you must realize this is why Mozilla (and other open-source products) haven't shipped. My limited experience in development has led me to conclude the following (at least at the last place I worked) about the process.
:)
Program managers come up with ideas, everyone goes to meetings about them (even testers). Development leads for each feature team (btw, all were developers for 5+ years) then come up with schedules, working with the devs.
The devs (me) try really hard to get it done. There is an absolute ship date.
It slips a little bit. Everything takes longer than imagined.
We cut a whole bunch of stuff so we can ship close to it.
We finally ship a product.
If we were coding for coding's sake, the work would never get done. It would, however, be a programmer's dream.
There's no mystical energy field that controls my destiny. It's all a lot of simple tricks and nonsense.
...that those who are in leadership roles in the open source and free software movements are not only capable of managing people; they are also very technically adept. No PHBs here! In many cases the leaders are the original authors of the project, and in most, the leaders have assumed their role only through the development community's collective respect of the leader's technical merit.
So while our (open-source/free) development communities do, in fact, have leaders, our leaders are, in fact, better.
Washington, DC: It's like Hollywood for ugly people.
I was going to say the exact same thing, that it sounds like how the distributions work. Sure, the linux kernel currently is managed by Linus, but due to the bazarre nature of open source*, as some random CS guy from Finland, he was able to write a kernel and people started adopting it alongside the BSD and herd kernels, which were already in development.
Although he manages the kernel he does not manage any linux distributions.
* I use the term open source here since we are talking about ESR's open source theories. I usually call it free software.
I've sent an email to the author of the article explaining how the *BSD systems (particularly FreeBSD) are orginized. Waiting for an update. :)
- Alfred Perlstein - Programmer and Administrator, Wintelcom.
With good project management, the management is invisible. Management is an enabler, and nothning more (and nothing less!).
Having said that, I feel that open source projects encourage an enabling management, but it's management nevertheless. The people in charge know that they can't motivate people by creating artificial deadlines, or stupid requirements. The motivator is enthusiasm and success.
Open source attracts good management.
(An essential book for Managers, talking about this stuff, is Peopleware, by Tom DeMarco and Timothy Lister. Do yourself a favor and get it for your manager!)
Under capitalism man exploits man. Under communism it's the other way around.
Calling Open Source methods the Cathedral is simply ridiculous. The cathedral is monolithic and it's architecture can't be changed. Bazaars involve negotiation, haggling, keeping a sharp eye out for prevailing opinion, constantly changing power relationships. Which of those two describes the Open Source community we all know.
This is one of those "They're all wrong and I'm right" articles. Only he isn't.
Before we actually start to flame him, maybe we should all read the article first.
I'm not saying whatever he says is all true. But he got some interesting points there.
The 'real' bazaar model in software engineering would not really work. Do you actually think that it would really work, among all the chaos that would result from such a model? Someone, or some trusting entity, will have to put an order to the chaos at the end.
However, there's probably something that Connel failed to point out. The open source model (flat cathedral) eliminate a lot of efficiency out of the traditional "pyramid" model, where a lot of decisions are made by idiot management types who have no technical skills. What results from that is poor quality softwares, coz those management types (most of them) don't know what they are talking about but insist that certain features be in the system or certain architecture be designed in a certain way.
The open source model is centrally controled by capable and smart people who are themselves contributors to the project, not just people who wander around in the office with note book and claim to be project manager. And the final decision on what will go in the source database take into consideration a lot of opinions, suggestions, debates from all participants. As a matter of fact, open source projects that do not do this are loosing contributors, e.g. x86BSD, just like the traditional model that do not allow any say from contributors does not prosper well.
Take a look at a major failure of Linux--its complete inability to produce anything resembling a coherent, useful desktop environment. In contrast to the Linux core, which is well and capably managed by Linux and his lieutenants, a whole bunch of different groups are trying to make Linux "user-friendly". The result is a user experience that, IMHO, hasn't much improved over the last seven years (the period of time I've been using Linux). Everyone's out there chasing neat ideas, not enough people are paying attention to the boring but vital details that make a quality user interface. And there are so many different projects, with none a clear leader, that people who might put the effort in keep getting drawn in different directions, diluting their effect.
Over the years, I've found that the more tightly controlled a project is, the more it has a chance of being _really_ good, as opposed to just useable. FreeBSD is (IMHO) is significantly nicer system to use than Linux because of the level of control the core FreeBSD team has. Documentation is very good, a single (and incredibly good) package managment system is used, installation is easier than most Linux distros, etc. etc.
I think "closed source", "proprietary", and "commercial" software got a bad rap in large part because of Microsoft, which has insanely bad quality control and design practices. That isn't to say that I think the average controlled or closed source project is better than the average open source project, but it certainly isn't as much worse as people would think, and the _really good_ projects tend to be very tightly controlled in one way or another. I'm hoping that BeOS grows in the future, and that Mac OS X turns out well too. Just because they come from companies doesn't mean they're automatically bad.
Ciao
Absolutely! Which is why we have clinicians, nurses and physicians working for us. They use the equipment/software in actual clinical conditions. Then we have external clinicals. Then we have beta sites where the stuff is used in real life situations.
:-)
But what it all eventually comes down to is "the customer is always right, even when they are wrong". If there is a feature that they need/b but they do not use, then they do not use it, QED. And if they don't like it we cannot force them to use it. In one instance we improved our medical imaging quality so much that the customers hated it. Physicians are very set in their ways, and the new improved medical imaging was *different*, so much so that they could not recognize the "texture" of a tumor with the new imaging, but relied on the older blurry "texture". However, younger physicians who are not so set in their ways prefer the new style imaging. Go figure
Open Source non-commercial projects have the freedom to work on whatever they want without worry of financial loss. But commercial projects, whether proprietary or open source, cannot afford thousands of man hours for features that they customers will not purchase.
A Government Is a Body of People, Usually Notably Ungoverned
Actually manage the project. Decide who does what when, book resources, order dependencies, plan to try and meet the deadline, decide how to deal with crises. Open Source projects could use very small amounts of this, mostly motivation. More management is usually not needed or welcome since there are no deadlines and the developers are volunteers and want to organize their own tasks.
Control the architecture of the project. Define what is to be built. Choose what overall approach to take. Decide how to solve particular technical problems. Ensure quality of work. This is definitely needed in Open Source projects. Either a single architect or some elite group of well-respected and active developers does it.
Ensure that developers put in their full day's work, evaluate their performance, promote them, select new ones to be hired. This is obviously an artifact of the employer/employee relationship, which is inherently adversarial and so requires a large amount of formal monitoring.
In my opinion, Open Source projects are limited by architectural vision more than anything else, and this is a sort of failure of management. Either there is not much vision and the project yields an outstanding implementation of archaic ideas, or there is one overloaded visionary (the compromise that works OK in practice), or there is fine vision but the visionaries fail to communicate it to the masses and the project never manages to turn into actual code.
Pavlos
Unless a project consists of few people and your paying them, Chaos is inevitable. Good programmers rarely tend to work together. We all think we are magicians, why would we want more of one of us in a project. We all want to be a hero. And OSS is NOT a place to try to practice Egoless programming.
UNIX is the answer, but only if you phrase the question very carefully
"uncontrolled", sure. But who controls? Not development, surely. They have their own skeletons in the closet.
Honestly, this is the job of a competent executive team and board of directors. It's unfortunate that on most executive teams the sales people seem to be the biggest wielders of political power, but there are ways to structure a company around that. (see Apple, which seems to have a good balance between development and marketing)
-Stu
Sports analogies are out of fashion, though. Oh well.
I recently thought about attempting running an OS project off of SourceForge. I kind of attempted to consider a way for people to assist on the design and direction on the project. I've ended up realizing that leadership is a necessary thing, which, as I've kind of learned through other experiences recently, is a natural occurrence whenever a group of people get together to work. Someone, official or not, usually ends up leading the group. This article had some good reasons to support the necessity of leadership in OS projects, which could actually be a more general law. I don't know too many teams of people getting to a state of self-management without some kind of leadership being the catalyst for the group dynamic. Most of the time it seems like the group starts out in a kind of "bazaar" form, and then official (or unofficial) leaders kind of help guide the group to achieving some kind of organization. The level and type of organization varies, but, then so does the productivity of the group. It's a very organic process. I, for instance, originally tried to set up a protocol for setting design goals/requirements as a group then achieving these goals without having to be the leader making the decision. I don't think this can happen, mostly because it really depends upon the people who are contributing to the project. The group dynamic should try to strive for self-regulation, but it takes time to build that dynamic, and, well, it's dynamically allocated :) People are different, thus, groups probably will tend to organize themselves differently, pertaining to the task at hand. Some may have that "central authority" organization, al a fetchmail or the Communist state, others may organize around several leaders.
My point? Groups become productive when the dynamic of the group (the skills and communication capabilities of the group) is developed and refined to fit the situation at hand. This refinement usually happens from some kind of leadership, be it official or not.
Some groups might require that central pivot. Doing develop over the internet might require a leader just keeping records of who's doing what so work doesn't get duplicated. But if the group communicates amongst itself particularly well (via message boards or whatever), that kind of central authority might actually be kind of a hindrance.
To say that OS development will require a central leader is a little harsh. A leader will likely be required to give any OS project a kick in the pants, but good organization and a willingness to understand the components of the group will lead to more of a bazaar model over time.
Perhaps groups tend towards entropy.
Just my $.02, which rambled on a lot longer than I thought.
-Tristan
Traditionally the business model of almost anything as been about producing some widget as cheap as possible and selling a lot of them. The qualities of the widget were unimportant, as long as it wasn't blatantly defective (and even this wasn't a problem, so long as the others in its category were defective).
What made the business model sing was MARKETING -- someome discovered that if you had really catchy marketing it didn't matter what your widgets were, you could make people buy them and create a market for your widgets.
The computer software industry suffers from this attitude. They think that it doesn't matter what kind of quality their software has as long as its not demonstrably worse than their competitors. They'll use MARKETING to to sell their products. cf. Microsoft.
(Incidentally, this attitude towards marketing as king is also what makes software companies drool for cheap guest workers on H1B visas -- you never hear about the shortage of high-quality marketing people, you only hear about large salaries and benefits packages for them).
It puts the software industry in a weird position. You can make a pile of money selling crap software with good marketing, but at the same time the rank-and-file IT people have figured out that the free stuff off the internet (Linux, BSDs, etc) is as good or better than the crap pushed down their throats from the marketing people, and they're defecting to the quality stuff in droves.
I think at some point the software industry may figure out that they need to put the development people in charge of the business process and make marketing and sales respond to development instead of the other way around.
Using marketing to drive your business model is probably entirely appropriate for products with little practical difference (soap, jeans, et al), but I think it might be the ultimate undoing of the software biz.
Putting marketing on a level with development is a sure way to skew the organization and make it marketing-driven. Marketing is fundamentally different from development, and needs to be treated as such, not just botched in as "just another team".
--
It's a
-- Danny Vermin
Of course there is a single website for GNOME. Of course there is a single person managing the Linux kernel. Of course there is only _one_ Nautilus app.
:-)
But if I want to add something to GNOME, I write what _I_ want and offer it to the community. GNOME may or may not accept it into its core, but that's valid: GNOME is the "market visitor", picking out what it likes. If I have a patch for the Linux kernel, I offer it at the mailing list. Linux visits the "marketplace" and picks out the good stuff.
I've seen people "playing marketplace" with their GNOME stuff. A good instance is Galeon; it is not accepted at the GNOME core, but it is very popular amongst individual "visitors". I've seen people offering their stuff at linux-kernel; some of their stuff is accepted, some of their stuff is rejected by Linus but remains a seperate patch. I've seen someone offering to write Samba support for GNOME-VFS, at which the Nautilus folks responded: "that would be great". I've seen distro's choose between package formats. Etc., etc.
Now, if this isn't a marketplace...
Of course there are project managers, but all they really do is "shop". And everyone is free to offer their goods. Never, ever, does a project manager tell somebody what to do in the Open Source world; instead, they only say "I'd like to have this, if there's someone who implements it, I'd be grateful". If there's no-one implementing it, it's not going to be there.
Go look at some Open Source projects' FAQ's, you'll often find these two:
Q.: When will it be finished?
A.: Depends on your input
Q.: Will there be feature X?
A.: No-one is working on it, but you are welcome to do it.
'nuff said, right? A project is nothing more than a "market visitor", that says what it needs and where it should go. The "market people" then can make their offers.
It's... It's...
"We can confirm that Debian does *not* ship the version with the trojan horse. Our version predates it." [CA-2002-28]
Another compelling (and often-quoted) section of The Cathedral and the Bazaar is the discussion about debugging. Raymond says: "Given enough eyeballs, all bugs are shallow" and "Debugging is parallelizable." These assertions simply are not true and are distortions of how the development of fetchmail proceeded. It is true that many people, in parallel, looked for bugs and proposed fixes. But only one person (Raymond) actually made fixes, by incorporating the proposed changes into the official code base. Debugging (the process of fixing the program) was performed by one person, from suggestions made by many people. If Raymond had blindly applied all proposed code changes, without reading them and thinking about them, the result would have been chaos. A rare bug can be fixed completely in isolation, with no effect on the rest of the program.
The above quote from the article suggests that "debugging" consists essentially of making the code changes themselves. - WRONG!
The most excrutiatingly painful part of the "debugging" process is to actually find the bug, not necessarily and exclusively fixing it which I consider more like the fun part of the challenge. Well that can be inverted in some cases too ...
Anyway, the Open Source model, with "many eyeballs" looking at the code, helps you FIND a lot more bugs than one person would, and identify the weaker areas of the code and architecture. Of course the code doesn't fix itself because a bunch of geeks looked at it, and the OpenSource concept never ever claimed to allow that, but it does allow a more thorough debugging process which will eventually make for stronger code.
One of my favorite Open Source projects is the Xalan XSL processor, which I believe is another marvelous example of the success of the Open Source concept. I have tested its compliance against W3C Standards by using a good 99% of XSL-T and XPath's features in some complex XSL-based applications I'm working on, and I can tell you this thing is rock-solid, which I find even more laudable as language processors are some of the more complex applications one can develop.
Extraordinary Vacations. Exceptional Prices
This no manger thing could prove to be a problem. Imagine the scene, three years hence. A street corner, in a cold New England city. A lady walks by a home-impared person. "Got a dime," he asks "for an old mid-level technical manager?" This will be happening all over the country, as companies large and small shed their carbon-exchange units. There will have to be a government program to help these unfortunates of the "Programmer Revolution." It warms the heart.
Rafe Singer: Sorcerer's Apprentice, Doer of the Impossible, Sayer of Sooths, Believer in Murphy's Law
But that's NOT how they work. They DO have management and projects and plans. Do you really thing Bob Young's official title is "Guy in the Red Hat"? Do you really think Mandrake has no PHB's? Do you think SuSE has no offices, and every employee logs in from home whenever they feel like it?
The article is right on the money. There is no bazaar. It's more of an open-air cathedral.
A Government Is a Body of People, Usually Notably Ungoverned
... one of the safest and most productive forms of government.
They are not democracies. Democracy doesn't work for this sort of thing.
Most projects, have 3 letters and one person:
CVS and the guy who approves changes and/or does most of the work.
By the time Open Source (the good stuff) is released its all-ready pretty pollished and/or has an established framework and scope of application.
Hence yes, there is central organization. But the advantage is that said organization comes usually from those with detailed technical knowledge.
-Daniel
But you are missing the one important role of upper management and marketing drones: they are not geeks. Seriously. When you take a look at open source projects without PHBs you'll see geeks writing software for other geeks. (I am a geek, no insult intended). Looking at those OSS projects that aren't geeky, you'll find that they do indeed have upper management.
In the company I work for, we make medical imaging devices. If the hardware and software engineers had their way, we would have a vastly different system, and possibly more robust and powerful. But it would be useless to the physician end-user. We are lucky in that our CEO and many of the top dogs are in fact physicians. Those that aren't are in daily contact with physicians and nurses and hospital admins. If the majority of the users want a feature, we will do that feature, regardless of what the engineers think. This is a Good Thing!
A Government Is a Body of People, Usually Notably Ungoverned
You are working on a code project that anyone can look at. You have a 'duty' to make sure that it is the best code possible to make. Also, look at it this way: You have a bunch of geek coders all working on the same project. If you don't have some sort of management and organization system, then the project is pretty much doomed to failure.
There are various Software methodologies to use, each with their own advantages and limitations. Frankly, I don't reccommend Team Software Process (TSP) for Open Source, as it relies too much on time management figures (which may not always be available) and requires too many forms, which may not be the best things for Open Source coders.
Just my $0.02
Kierthos
Mr. Hu is not a ninja.
All in all, it's not really terrible, but it could have been a lot better if he would have stuck to his main point. If I were marking this as a paper, I'd give it a C+.
"Anyone that has ever gotten an idea based on any of my work and done something better with it-good for you."--J.Carmack
I'm just curious - how does the KDE project is managed? They have no one head figure, no "dictator", yet it seem very organized with planning ahead of dates, feature-freezes, code freeze etc
Is it a true bazaar? Or maybe the management is taken by "volunteers" for each task? Someone from KDE could answer?
I wonder, on the assumption that a large number of people believe the alleged fallacy, what reasons they would give for accepting said concept? I mildly suspect there would be a lot of "geeks are solitary by nature, this lets them work the way they're used to" comments.
-TBHiX-
Consider the Illustra/Informix DataBlade stuff. It makes perfect sense to consider a DataBlade the equivalent of a Linux driver (same with Postgres plugins). As long as the interface is well defined and adhered to, the people developing the datablade kernel don't have to have strict control over the datablades.
The problem is that RDBMSes were designed in the 70s and 80s when modular code wasn't really in vogue. And even if they were, there are minor performance tweaks you can get if you ignore modularization in favor of an additional 1% bump in your TPC-C numbers.
If there were an RDBMS which was designed correctly, it would be a perfect candidate for open source development, as long as there was somebody sitting around saying "This version is stable enough to run your business on", which is the equivalent of the Linus/Alan system of test releases and final kernel versions.
Just because Oracle is designed wrong doesn't mean that the basic premise is any less sound.
The nice thing is that the reason the project is being done is to "scratch an itch" instead of $$$ (although they can overlap).
In ESR's bazaar, usually the one that becomes the codekeeper is the one with the biggest itch (or the first one with that itch). They have the personal drive to make sure things are done appropriately to adhere to that itch. Suddenly that is one less thing that is needed out of an official manager.
When the user requirements part is being dealt with, the programmer(s) and the user(s) are one in the same. One less need for management.
Code submission/Debugging? (I apologize in adance for all the imagery) The whole "blindly throw all the fixes in" approach doesn't work. OSS is more like seti@home in that the codekeeper takes in all submissions and filters through them and picks the best or most appropriate. Management can rarely do this and this is a very valuable asset because the same functionality can be written many different ways. The ability to pick and choose the best has to be there.
Code review - In the oss community the codekeeper does the initial code review by keeping what (s)he feels is important to keep and ditching what isn't. Then the code goes through yet another code review by the rest of the community. So, that is a 2fer on code reviewing.
In the oss environment management, the engineers and the users all get rolled into one. (Well, mostly anyway.)
:D
-Vel
Interesting stuff.
Besides the good comments about 'self-' and 'central-' organization lying along orthogonal axes, I would add the following:
There is a strong tendency these days to favor distributed, decentralized control. This is natural and good, as new technology allows new ways of communicating and cooperating. Many projects and communities lend themselves very well to this model, and getting rid of old school central authority is clearly progress. In fact, nature itself seems to favor this approach.
But there are further lessons to be drawn from the natural model: after millenia of trying, one of evolution's highest accomplishments has to be seen as the good ol' Central Nervous System.
So while there are certainly many exciting new possibilites, it seems foolhardy to judge Central Organization and Control entirely obsolete. After all, it has worked fairly well thus far throughout history, and evolution is about one thing only: What Works.
I am not sure I fully agree with the conclusion of this article.
;-) but that the ONLY way to manage complexity of Open source systems is through a modular approach that largely tends to a peer-to-peer model. It is an interesting thing that the code produced through modularization tends to be of great quality as well.
I started and still run a medium sized open source project (50 developers, 10 core, many 10,000 users) jboss.org (www.jboss.org) and we develop an EJB (Enterprise Java Beans) container.
I suppose every project leader has his style and the domain in which they operate dictates the structure they finally adopt but for the J2EE infrastructure we found that the "modular" approach works best for us. I will argue (for those that are looking for a short message) that for "Operating Systems" like efforts (Linux, J2EE) you need a mix of strong central management and a SELF MAINTAINING CODE ARCHITECTURE (modular, ala Linux 2.0) something I believe the article misses.
Here is the deal, we are today in our third iteration. The first iteration was done by self, really a "let's get started" code base that passed the first stage of putting some people together around the project (2 years ago). The second iteration was a clean rewrite with some advanced concepts given by the group and some of its most outstanding contributors. jboss1.0 was released with that code base. It was a fully featured container but we QUICKLY ran into a wall: it was too complex for a casual contributor or even for 3rd party contributors to come in the code base and integrate.
The lesson there for us was kinda trivial, OS like system grow extremelly complex extremelly quickly and there is little chance (or time) for people to wrap their heads around it. There was no significant growth of the code base, not because we were not dedicated (we were full time on it) but because we could not manage/teach the flow of contributions.
We took a good look at what Linux had done over the years and realized that Linux2.0 based systems were relying on a "modular" approach to Systems design. why not try the same for web-systems design? In clear the idea is to isolate the parts of the container (our kernel) so that folks that want to include a Transaction Monitor, or a Database persistence Engine, can focus on a few places and not worry about the whole architecture of the system. It is the equivalent of "divide and conquer". We modularized our container with interceptors and plugins and released an early version of the clean architecture.
The payoff was immediate, we started seeing contributions pouring in, both from corporations and star developers since they could spend one night to understand the scope of their work and start banging on it right away, knowing that the integration was PURELY CODE BASED. That was one of the advantages of Java's Object Orientation and interfaces for us, it was trivial to do.
The point that I am trying to make, and that the article misses imho is that these contributions are sort of automated. Sure we look at all new modules and we OK-notOK projects but we have given RW passwords to our CVS to litterally EVERY contributor on our list (I think we have like 40 guys on CVS). Sure now and then we will see bad fixes, but these are limited and the interfaces and modular approach ensures us that the integration is almost SELF MANAGED.
I don't claim that this applies to every succesful open source project (yes we are succesful
Code can be automated, in the integration sense, and if you don't have that, you won't keep the exponential growth of code base... there is NO management (read no man/woman) that can replace it.
marc
The real mnf999 always posts as anonymous coward
Look at the FreeBSD. All of them have a strong central group that oversees everything with other developers working on various things with specific levels of access. They are a good example of a success story.
Personally I think many porjects int he commercial world are either under-managed or over-managed to the point of ridiculousness. The worst is whe nyou have the "admin by committe" or the "develop by committee" approach where you ahve to have 10 meetings to get one thing changed.
this space for rent
I completely agree; I posted on this as well.
:)
;)
Fortunately, lots of people didn't seem to notice our posts; that's good, because Angband is also highly addictive.
Have you tried out Mangband? I messed with Angband, and The Angband Borg, and played Zangband a little, but then I lost interest for a while. (I need to change the behavior of Vampires sleeping in an Inn: waking up in the morning can be fatal!
---
pb Reply or e-mail; don't vaguely moderate.
pb Reply or e-mail; don't vaguely moderate.
When an engineer hears about a bug, his first reaction is not to deny it's existance, but to fix it. He (or She) want's his 'baby' to be perfect. That's my experience, at least.
:-)
Good mature ones, yes. Be thankful for your positive experience. I had a gig at a very large financial company for awhile and got to know the QA people. This experience of yours would be new and wonderful to them.
C'mon, you never heard "It works on my machine!" ???
Personally I learned a long time to never say "that's impossible", but only after I had done it too many times. That's why I'm so humble now.
Or, as Richard Pryor once said "Woman, who you gonna believe, me or your lyin' eyes?"
The revolution will NOT be televised.
One of the key reasons people adopted open source OS's lately like Linux is not because of their freedom but their stability. Yes, the earliest hackers were all about "freedom" in its many definitions. But as of the past few months I've seen college students buy boxed versions of Linux simply because "they want something more stable".
Then Windows 2000 comes along. It's stable (the Win2K side of my Linux/Win2K box hasn't crashed once). It also runs a huge majority of my favorite apps well, like Unreal Tournament. It doesn't have emacs, but it has Visual C++ which can do both editing and compiling (in my mind) far easier. As for Gimp, I can wait for it to come out on the Windows side.
The bottom line is, Open Source for us college students has really lost its flavor. We're hackers in a sense, but we don't (excuse the pun on my name) have the same fervent attitude about writing Linux. We hate Microsoft, but we like some of their products. It's like the line in "Clerks", "I hate parties but I like large gatherings. Strange, isn't it?"
- I don't care if they globalize against free speech. All my best free thoughts are done in my head.
I work for a large and nameless consulting organization. And we organize, pretty much all of our projects with a Project Manager, who is responsible for budgets, client relations, timeframes and administrivia, an an Architect (or Technical Lead, or whatever) who is reponsible for the technology and the solution. Now the two have to work reasonably close together, but the PM is not there to micromanage the developers, and the Architect doesnt get bogged down in budgets and politics. This works well. And dont get me wrong, we are are a large and structured organization, but an effective architect has as much opportunity to earn the big bucks and reap the other rewards as a PM or an administrative manager.
One of the things that made the Apache project successfull is the easy way it can be extended by modules.
Anyone can decide to "scratch is own itch" and write a module, helping by that way to extend what Apache can do. So, this could be considered a "Bazar" kind of developpement...
When I think of a Bazaar, I think of something different.
To me a bazaar is an entity, which is amazingly well organized, among many participants, with equal rights, for the sole purpose to offer each merchant the possibility to sell goods to the public.
A bazaar has no unifying, common goal or project other than giving each merchant equal opportunities at a central meeting point.
So, the fact, that everbody can look at other people's open source code and can use and play with any way they want, to me is analogous to giving each merchant the opportunity to learn his trade, look others over the shoulder and borrow some tools. The bazaar is the natural environment where you can do this efficiently.
The only unifying goal they all might have, is to never allow to close down the open bazaar as a meeting point and trading place, because it's so essential for each merchant's survival.
To me sourceforge is a bazaar and slashdot is the bazaar's coffeehouse.
But it is not sourceforge or slashdot, which produces the code for a software package, nor is it the organizational entity of the bazaar, which makes the production of code happen, it's just the place, where you find the code and talk about it.
It might be fun to study how cathedrals were really built, how long it took, how often they
stood half finished, because they couldn't get "the financing" right, how often they were picked up again by the next dynasty and finished half a century later.
And when it comes to the actual design, construction and engineering work, I don't believe one minute that you can achieve anything without a certain hierarchy within the architects masons, artists etc.
Usually there is a maestro, a "Meister", a "genius" in his craft, who just is recognized as such, because of the outstanding work he produced ON HIS OWN (which he even might have displayed at the bazaar).
Bystanders are then so much in admiration of his craftsmanship, that they deliberately follow, learn from him and work with him on "his" project.
I believe in order for this to happen, that project also must serve the whole population in a charitable way. Anything else will not cause enough enthusiasm and devotion to cause a deliberate cooperation among programmer under the roof of a benign master.
They accept the benign dictatorship as long as the master is really a master in his field, as long as he is willing to teach, encourage and motivate and as long as the project is for the good of all. It is the purpose and usage of the project's code which makes it earn the financial support from third parties.
And then have you ever seen a bazaar, leaving anything else behind, as a project, than a messy, empty place in need of clean-up for the next day of survival ?
That's not to say that the bazaar is not absolutely essential for human beings to have to satisfy their daily needs. First, you can find talent and treasures there, actually you find there the master artists and craftsmen of tomorrow, and then you can just have a hell of a chat and deal making, and can find a lot of tools to help you out.
Within a cathedral you may find the artist working with the sacred silence of a child, playing completely absorbed with their toys and forgetting the whole world around them. No money in the world can make that happen, no manager can enforce it, but a hell of noise of a bazaar-like athmosphere can certainly disturb the process of creation and distract from the actual work.
And then didn't Michelangelo spent most of his life in St. Peter's painting years and years the cupola ? He had his little crew of master students and quite a strict management team to obey to, I believe...8-)
It took me a while to figure out, but "management" has a purpose. It wasn't until *I* had to be involved in, and manage, projects with *real* deadlines and goals, and a motley assortment of people with different skills and attitudes, that I realized this.
Managers should:
* Define goals, and save us from death by committee.
* Define a path to those goals, so people don't wander off and get lost. "Path" may be formal/informal methodology, standards, technology, milestones for features, etc.
* Keep people on task. Your presence along makes them feel guilty when they are not acheiving goals. I know this, because I waste my time reading Slashdot, and I feel guilty to my manager.
* Facilitate communication amongst people. Complexity of communication scales at n^2. Makes sure people know what others are doing and aren't blocked by dependencies.
This is not exactly coding, but it is definately work that needs to be done. The price a manager pays for not having to be ordered around, is that they are now free to:
* Get blamed when projects fail.
Remember, managers <!=> suits
It's 10 PM. Do you know if you're un-American?
The author does not seem to realize that the difficulty is in finding the bug in the first place. He seems to think the finding or the insight to find is trivial, and the coding of the fix, once found, is hard. This is the exact opposite of my own experience.
You're definitely right on about projects with no management - i'm sure it's a nightmare. What most projects need is organization that comes from below, not from above. Look at it this way:
Our current business model: Business is a separate layer existing on top of the development and production processes. Administration is for the benefit of business, not development/production, even though business's only logical purpose in existence is to organize and improve development. Administration has the most authority, so naturally they only work to benefit themselves. Developers/producers; i.e the real workers have basically no say in this process, which most likely interferes with the efficiency of their operation.
A better, more sane business model: Since the development of the product is the most crucial aspect of a business's purpose, developers should rule the company, with administrative, marketing, sales, etc. types as advisors on the deveoper's payroll. Money should be distributed by the people who do the most work within the company: the developers and producers! The administrators make suggestions, but have no real authority.
And yes, it could theoretically work, and could help to eliminate the fact that right now, business is just a separate society leeching off of the real society and bringing no benefit to it whatsoever (it's called a PARASITE.)
I've said my piece.
Although, you left some parts off of your corporate example. It should be more like this:
Your problem -> Your manager -> VP -> Big Boss -> another VP -> another manager -> grunt who actually fixes problem.
In the open source world there are few people (usually none) who are pure management. When you send your problem to people working on the linux kernel who forward it to Linus, they actually write the code and can solve the problem. In the open source world your bug reports and comments actually go straight to the ears of the people who not only solve the problems and implement new functionality but also decide what new functionality gets added in. It is the excising of the huge manager -> higher up manager -> yet higher manager -> ..... -> high enough management to make a decision -> lower manager -> yet more lower management -> .... -> manager intermediate step which usually has very little usefulness (often it's usefulness is in fact negative), that makes "The Bazaar" so special.
Case in point, OpenSSL. Great package, and I'm glad the developers went through the trouble to create it, however the documentation is all but nonexistant. I subscribed to the various mailing lists for a while and found that documentation basically was in the oral tradition form, with some third party, out of data stuff available (for SSLeay, the predecessor), and some examples. This is where a formal process in a corporate setting, with a profit motive is useful. Sure, the programmers don't want to document what they've done, management would rather do something else too, but the customer will insist on documentation, so you'd better produce it if you want to make sales.
Take this as a plea to make good documentation a part of your project, not a peripheral activity or one left for someone else to do after you're finished. My thanks go out to the projects which have been fabulously easy to use due to well written docs. I'm not going to name names as I'm sure to miss some.
Any ideas on where I can find it? I tried all the major search engines, and found references to it, but couldn't find the troll itself. To find the reference, search on google for "subtle art of trolling", find the article, and scroll down to the section called "The Successful Troll".
As far as I'm concerned, this is the Troll Holy Grail. Help me find it, please! BTW, can I be on the mailing list? I've never trolled, but I certainly enjoy them now and then.
No, Thursday's out. How about never - is never good for you?
As I recall from other IBM articles, the chief programmer team organization led to startling gains in productivity. But it never became wide-spread at IBM because they had an order of magnitude more projects than they had people who were qualified to be chief programmers. This is probably true at most places...
I am so fucking sick of hearing about fetchmail!
It's tiny and nobody cares! It's not an example of anything!
Any time anyone mentions fetchmail, I know to stop listening to them, because they're just taking ESR at uncritical face value again.
..." In fact, these projects use strong central control, which is crucial to their success."
This sentiment seems to be frequently on Timothy's mind. A while back, he gave us this topic: Peter Wayner on the Spread of Information , along with this comment: "Complexity seems to demand individual autonomy, doesn't it?"
My reply is the same:
Complex systems require control at a more fundamental level.
Clipped the long quote below from an essay by John Zerzan, http://www.furious.com/perfect/zerzan.html , while seeking out a discussion of T. Adorno indirectly on the subject of complexity and autonomy. Adorno is a dead Western Marxist (Frankfurt School, ie lots of cultural criticism...). Discovering a Zerzan essay on the topic was a happy accident.
Some talk about the composer Schoenberg's experiments with atonality (ie musical notes freed from the dominating center - the harmonic scale) in order to better express the 'Modern' individual's alienation followed by...
"By the early 1920s he had given up the systemless radicalism of atonality: not a single "free" note survived. In the absence of a tonal center he inserted the totally rule-governed 32-tone set, which, as Adorno judged, "virtually extinguishes the subject." Dodecaphony, or serialism as it is also called, constituted a new compliance in the place of tonality, corresponding to a new phase of increasingly systematized industrialism introduced with World War I. Schoenberg forged new laws to control what was liberated by the old tonal rules of resolution, new laws that guarantee a more complete circulation among all twelve pitches and may be said to speak to capital's growing need for improved recirculation. Serial technique is a kind of total integration in which movement is strictly controlled, as in a bureaucratically enforced mode. Its conceptual drawback for the dominant order is that while greater circulation is achieved via its new standardized demands (none of the tones is to be repeated before the other eleven have been heard), the concentrated control actually allows for very little production. This is seen most clearly in the extreme understatement and brevity in much of the work of Webern, Schoenberg's most successful disciple; at times there are as many pauses as notes, while the second of Webern's early Three Pieces for Cello and Piano, for example, lasts only thirteen seconds.
The old harmonic system and its major/minor key points of reference provided easily understood places of departure and destination. Serialism accords equal use to each more, making any chord feasible: this conveys . a somewhat homeless, fragmentary sense, suitable to an age of more diffuse, traditionless domination."
Zerzan continues on, of course, but this is a good place to stop.
What looks like autonomy (equality, homelessness, fragmentation, traditionlessness) on the part of individual notes is actually the result of a heightened system of control.
EC
Marketing is the ultimate undoing of the software biz?
Do you realize engineers have been saying this about *everything* for the last 50 years?
"Development" doesn't drive the organization, as the organization is usually aimed towards a goal, presumably defined by people as a "collective itch that needs scratch". It's not development for development's sake, it's for meeting an unfilled market need (even if that market is a small niche of developers).
What drives a business organization is the continual battle between innovation and marketing. Innovation creates the new and destroys the old through research and searching for opportunities. Marketing is about making the world know that you have a product available, and then finding out if there's any enhancements that can be made.
It's just the nature of the economy.
-Stu
Perhaps for the upcoming second edition I shall have to write in more detail about the Benevolent Dictator role, if only to prevent the kind of misundestanding this article is founded upon.
>>esr>>
The comparison fails because it tries to map the open source development model to the traditional development model. Of course there are going to be a lot of similarities as much of the open source development model output set easily fits into the definition of the traditional model. However, try mapping the traditional development model to the open source development model. One finds that the traditional model is restrictive and the open source development model is much more flexible.
The traditional model has more of a solid state, whereas the open source model is like a liquid. The management structure only resembles traditional development because those are the roles that people evolve theirselves. Liquids settle if undisturbed. However, no role is required and no pathway through the system is the only pathway. Should a manager fall out of the chain, or change goals, the open source model flows around it and meets at the next link. The article points out that Linux kernel development slowed when Linus had to take some time off. That's not true. The "official" releases dropped off, but development continued and releases were still made by the other managers. The development simply flowed around Linus and he caught up with it when he came back. Of course in the open source model all of the managers are developers. When anyone, not just Linus, disconnects from any network of developers, there is a loss of total production potential. However, if he disappeared from the face of the earth the same thing would happen, except a respected individual would be declared the "official" release source, and any specific skills he had been providing would need to be taken up by new developers. Also on this subject, releases occur at all points, not at the top level. I've made a change to the kernel for the benefit of myself and a very few other people. Although the audience was very small it was still a release, I became the manager. I set the goal, I followed the goal to completion, I checked the quality, and I made the decision to put the patch out. Note that this was also a highly parallel development, as the entire workload was handled by myself. No supervision was required. Being part of the network, I did consult the existing developers though. Eventually my patch was obsoleted by a parallel effort from another part of the development network.
As other posters have pointed out, the control of the management is with the developers. The development model graph really does look like the Bazaar network for many of the projects, it's just that the links between developers are redundant. The connections are there but aren't used frequently unless another link breaks. Keep in mind that large projects like the Linux kernel aren't just the tar ball put out by Linus. There are numerous projects running in parallel that provide modules for the kernel that aren't contained in the kernel, and none of those people have to report to or accept goals from anybody in the core kernel deveopment. It just happens that they do so in order to maintain compatibility. Some of these modules make it into the "official" kernel, even though they were never in the goals of the official kernel.
Be careful here. There is a distinct difference between what a customer needs, and what the customer thinks they need. When a customer comes to you requesting feature X , you, as a developer, should ask for the larger context that X fits into, and the larger need that's being filled. Perhaps X is difficult to implement and fits the need in a narrow fashion, but a different solution Y would fill the need better and may be easier to implement.
Really, the specification process needs to be a two-way street between the developers and the customers, so that the developers produce something truly useful for the customers, and the customers ensure their needs are met.
--Joe--
Program Intellivision!
If you want an example of true Bazaar development, you need look no further than Unix itself. Unix, not Linux. "Return with us now, to those fabulous days of yesteryear!" ...
The August 23rd Slashdot has a pointer to a wonderful Unix family tree in graphical form. As we all remember, Ken Thompson created Unix, and over time it leaked out from Bell Labs to assorted universities. It also flourished inside AT&T, producing quite a number of variants, each with it's own goals. Once UCBerkeley's Computer Science Research Group got going, we had the *ix equivalent of the Cambrian Explosion. Quite apart from the 2BSD vs. 4BSD vs. AT&T variants, every hardware manufacturer on the face of the planet started carving on 4.2BSD, all with the intent of "differentiating themselves from the rest of the market".
The end result? Just watch a GNU configure script run, or (shudder) install an older sendmail (circa v5). (Hey - remember when there were lots of sendmail variants running around too?) "Unix" doesn't mean a darn thing in terms of functionality, interfaces, etc.. There is, of course, a "Unix flavor", and that's why we all recognize Minix, Linux, et al. as Unix-oid systems. But it took repeated attempts like AT&T's SVID, IEEE's POSIX, and others to simply pare down the list to an almost-managable size, and even then the real factor was that the market couldn't support all those hardware variations, so their software variants vanished with them.
The Bazaar was bizzare for Unix, and it nearly killed it. The community sorted itself out after a time, and grudging annointed Scott McNealy and Solaris the winners, but still leaving a few others in play. In the mean time, a lot of effort was wasted and a lot of money was spent that could have fed the homeless or done something equally worthwhile.
Charles Connell is right, even if he does mistake the meanings of the Cathedral and Bazaar models for the shape of their org-charts. Fred Brooks was right too - if you haven't read TM3 lately, you owe it yourself and to your fellow human beings to let the Prof. lecture you once again.
*First, look at Mozilla: everyone knows that a browser is really one of those tools which are very important in today's desktop environment. But instead of planning and developing a fast, stable, usable one, Mozilla just implements funny, but useless things like a terminal emulator. Don't get me wrong: everybody should do whatever he wants to, but sometimes I think it would be better if we had a browser at first and then think about which nice applets could enhance it.*
XMLterm is NOT developed by the core mozilla developers. It is a project given to mozilla by OUTSIDE CONTRIBUTORS, which uses mozilla's rendering engine.
Believe me, the majority of mozilla developers are NOT free to work on whatever they please. Right now, they can only work on bugs that are marked nsbeta3+, they cannot add any new features, and although a lot of annoying bugs are fixed in various developer's trees, they aren't allowed to check those bug fixes into the main tree because of "risk". Every OUNCE of energy is being focused towards shipping.
Just because YOU don't want a full-featured browser such as mozilla, doesn't mean that EVERYONE doesn't want a full-featured browser. You would probably like gaelon much better, so go use it.
...the 'managers' in this case cannot be ignored, they are there. BUT they are not the least able to contribute, but often the most able. I see a lot of OS projects start out small, and grow and grow. The original, and often very very talented, individuals usually wield the most control. From what I've seen of the business world (a few internships), the managers are usually the least competent. So these projects are "self-managed" if you look at it the right way, it's managed by specific people, it's just not managed by some seperate entity.
Health is simply dying at the slowest rate possible.
Amen to those hearty and needed comments.
ESR has no degree. Having a degree doesn't make you a competent programmer, nor does the lack of one prevent you from becoming one.
No, Thursday's out. How about never - is never good for you?
Managers figure out how to use resources (i.e. people); architects design systems. While there's a little management in Open Source, most of my work with SAX (Simple API for XML) has been a combination of design and implementation. After all, I can hardly threaten to fire people or offer them bonuses or promotions, so I cannot make anyone do anything but what they wanted to do in the first place. That's what Eric meant, I think -- the architecture evolves according to what people are willing to do, not (usually) according to a strong central vision. I remember being part of the thread that annoyed Linus into allowing loadable kernel modules (hardly part of Linus's grand vision at the time).
I chaired a W3C working group for a year and spent most of my time worrying about absurd procedures designed by non-coders to make sure that the coders couldn't do any real work. That was management.
Do the Cathedral and the Bazaar means two different project management models or two different approaches to software development? Sure the open source projects may have a cathedral model, but it lives in a bazaar environment where competition thrives!
I think this is what ESR meant in The Cathedral and the Bazaar.
I fail to see how that saying that and saying bugs are difficult to find are mututally exclusive. Some of my dumbest users find bugs, that doesn't mean I'd want a million of them trying to patch the code. Nor does that mean that the leader developer alone could necessarily detect the bug. When you program, you develop a certain ingrained approach to what you're developing--you implicitely assume on some level that your testing efforts are going to be inclusive of all the users. Thus it is easy to develop certain blind spots, it's hard to look at something with a totally fresh set of eyes. Even if "fixing" the individual bugs is trivial, you have concurrency issues that CVS and the like can only go so far to solve. This is especially true when bugs, as they often do, require a higher level fix than just in the place where it makes itself known.
What has a tv show with a huge amount of tits got to do with Slashdot? This is hardly 'news for nerds'. I think rob has sold out...!
It's funny that this just came up on /., I was just talking with a co-worker about software project mismanagement.
Of course OS projects are managed, that much I agree with. The thing the author doesn't delve into is the people doing the management (or better said in my opinion he doesn't give enough time to this topic). We have never and will never see anyone who is used to managing a plastics factory suddenly put in charge of an OSS project. We will also never see a kid fresh out of business school, with no software experience what so ever, put in charge of a web application (these are both examples from my workplace and the topic of the discussion I was having with my co-worker). OSS managers are there because they love the software they are working on and because they have a high level of skill and experience. I'm not managing an OSS project, I suck at coding. A friend of mine, who is a pretty fair coder but absolutely hates doing it, is also not an OSS coder. Open Source allows us to have the RIGHT people working on a project for the RIGHT reasons. Nobody works on an OSS project because they need the paycheck. In fact, I think it's more typical that the paycheck suffers so that the OSS project can be better.
Politics, Culture, Food?
Furtermore, these people are hired because of their resumé, not their qualifications.
The irony implied in this statement is beautiful.
-Mars
Go read "Debugging the Development Process" by Maguire.
The book is basically for the audience of "Technical Manager."
They might be called App Architect, Tech Lead, Technical Project Lead, Senior Engineer, whatever.
The role exists, and has existed for years, as the author of this essay points out... Brooks talked about it in the 70's.
Now one thing in a corporate environment, you have all the HR crap.
I think many technical people do not mind falling into the tech lead role. But most technical people don't want to deal with HR crap.
Divide that off onto another person, and you start to achieve success. Your tech leads provide their feedback into the reviews, but this team manager actually handles all the hiring, firing, training, salary, budget, whatever crap.
Many successful companies divide these duties up. Let a technical person manage the software development, and some other non-tech person handle the other crap.
However, Angband also has Ben Harrison, who did an extensive rewrite of the game making it an extremely easy framework to hack around. I've done my own hacking on the Angband source, and it was *so* easy to find and change little parts of the code, and then *see* results. I guess the point that I'm trying to get across here is "low barrier to entry"- I'm a fairly experienced programmer, but I didn't *need* to be in order to play with the Angband code.
If you want to get rid of the bathwater, you've got to throw out a few babies.
1. I don't think that it's ever okay to be in a position where ACID properties might be violated. I know that it's inevitable in the case of bugs, but it really should be strenuously avoided.
2. Even if you do look at that, what about if Alan Cox checks in a minor bug fix to something that will keep a certain driver from crashing in some situation which will only happen on average once every 2 years? The point is that people DO care about fixing bugs, and that there IS public acclaim for people doing that.
3. What about somebody writing a new DataBlade which allows you to have semantically more interesting responses to things (like the CIDR type in PostGres)? I think that's a lot of help, and it's the sort of thing which is most akin to the device driver thing. Somebody issuing a new driver which supports a new bit of hardware is always issued praise for it (as long as the driver doesn't suck).
4. What about if somebody writes a new sorter which is 20% faster than the old one? Same scenario as the Alan Cox example... There are a lot of things in databases where you can get very quick performance enhancements which people will recognize and see every day, whether it's in query optimization, execution, drivers..... If all you care about is performance hacks, there are so many places to look at in a database that it's hard to know where to begin!
In my opinion, I think you're mistaking what are the sort of really CORE bits which some truly dedicated people are going to work on (like the networking system, the file system stuff, etc.) and what are useful modular systems which are akin to software drivers.
Besides, one of the things that you're ignoring is that there are a lot of people who are paid to work on bits of open source projects which aren't quite as glamorous as the others. Think of the HelixCode/Eazel guys. Do you think that every single one of their engineers is paid to just work on highly visible, glamorous stuff? Of course not. But if they can get people interested in the glitzy stuff, then they can more wisely invest their development efforts on the things that are likely NOT goign to get done by random open source guy (like an ACID fix).
You might believe it when you see it, but the winds are changing and I think it's just a matter of time before this sort of thing happens to an RDBMS.
The Debian management is a complete mess for example...
In the old days we had a benevolent dictator (Bruce), and it worked pretty well. Eventually he got tired of getting abused, and we had project leader elections.
Now, a few project leaders later, we got a Debian constitution which makes sure that:
Oh, and that's not it. The packaging manual is also subject to voting...
The sad part is that only a few vocal people within Debian drove the project where it stands now. These people should play nomic and let the developpers do their jobs.
...and the managers above him sacked his ass inside of two weeks. What we saw as extremely useful to drive the project and get it done properly and in a timely fashion, they saw as a threat to their control.
As far as we could see, though, the company was winding down to die anyway. Sad.
Ask your doctor if getting up off your ass is right for you! -- Bill Maher
It's almost like a 'worse is better' thing- the Bazaar stuff has a hard time being more sophisticated than Cathedral stuff, but if you have the connectivity, it becomes a hell of a resource just by virtue of statistics. Things that work out really well propagate- if I was giving people my mp3 of my 'Protocol' tape, some people might be looking for jazz, or Scofield, and notice the new tune, listen, go 'woh!' like Keanu Reeves and keep the track in their own collections with great delight- the new track would propagate based directly on how great it was, and if it was no good nobody would keep it. There are spooky parallels with genetic algorithms... only when you have a lot of random noise and junk does a GA-like effect start to happen, because it defines the situation as 'looking for good stuff amid the bad stuff' and forces quality judgements to be made.
Orthogonal is the right word. Anything less is weak and confusing.
;)
>>Having "strong central control" is orthogonal to "self-organization".
This means that having "strong central control" tells you nothing about whether or not it is "self-organized". And vice-versa.
Not sure that I agree with your premise, though. Seems like most groups are composed of followers who will take any kind of leadership they can get
according to the author's description of the bazaar model, would be various Gnutella clients.
The author says that after the initial "plausible promise" release has been made, the original authors merely step back to become a part of the development community, and not a leader of it (just like Nullsoft did with Gnutella). Also, he states that from that point on, development would fork several ways, with the user community ultimately deciding (by usage) which of the forked paths is the most popular, and focusing their attention on that one. Again, this is pretty much exactly what has happened to Gnutella.
Pretty much, by reading down through his checklist of a "true bazaar" project, I was able to say "Gnutella... Gnutella... Gnutella" for almost every point.
Perhaps a better example of a bazaar is UNIX itself. It forked from UNIX to BSD and then to Solaris (which came back to SysV) and now Linux has taken it by storm because it's considered the best option by a large community. No one talks about the "glory days of UNIX fragmentation" though, which gives a lot of weight to Connel's argument.
"I may not have morals, but I have standards."
"I may not have morals, but I have standards."
I'll just leave you with this: in our society, actual work is done by a large amount of people; products are produced by a large amount of people, etc. Yet the small few at the top who do not actually produce any products; who do not know how or want to actually work for a living; who attempt to administrate despite their complete lack of knowledge about what is actually produced; those people retain the largest cut of the money and hand out the rest to everyone else who did something. Is there something wrong with this societal model??
Slave ethic. If they paid the actual workers at the top of the pyramid, they'd work for a year and retire, and the leadership would no longer have a work force.
At least that's my theory.
I hate it though. Most companies, if you're really good at what you do, they reward you by paying you more money to not do it any more (promotion to management). Where's the logic in any of this?
The answer, of course, is that business doesn't work on logic.
~ radiographite: art by john shepard
however, although organization may arise from chaos, good management is more than mere organization. it is the application of organization towards specific goals. this article points out that the requirement of such application is orthogonal to the source code fungibility, which, once again, underlines the difficulty of cultivating good leadership.
when goodness is lost, there is kindness. when kindness is lost, there is love. when love is lost, there is justice. a true bazaar is just, and no more. this is because it is impossible to expect from everyone compatible expressions of even love. the real world is beautiful like this.
what a thing is called is not the thing.
1) Not all managers are inept paper pushers.
2) The recent Ask Slashdot I was speaking of concerned a gentleman who had coded up a fix and could not find the right 'manager' to submit it to. The point being that anyone can actually write the code and can solve the problem but it still requires approval of the 'OS manager' to get it in the code base.
3) The bazaar model hypes the idea that everyone gets to decide how to fix a problem and decide how to put it in, not just the blessed few. Few projects follow this methodology, the point of the article and mine.
4)I've dealt with a few OS projects, and from my POV it is easier to talk to upper-management than deal with OS manager/engineers. Engineers tend to have an emotional attachment to the code they've written, and they don't like criticism of their 'baby'. The path I've taken in the past is to fix my local copy of the code and then post my fix to a newsgroup, or even to Slashdot, rather than fight that what I've fixed is an actual bug. Uppermanagement tends to have a myopic view of the world. Convince them that a change saves/makes money and it's in there.
If a company follows the process you outline, it is headed nowhere. Most companies I've seen are much more flexible about the introduction of ideas. At IBM the process was:
submit idea to study group->study group ask customers->study group makes a decision.
At my current job the process is:
submit an idea->start implementing it (unless the idea is completely off-base, the chances rejection are very small)
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
I think the author set out with a poor understanding of the process, took a claim about the process much too far, and then set out to disprove a claim that wouldn't have needed to be disproven except that he took it too far in the first place.
Bruce
Bruce Perens.
...are irrelevant. Jack (thinks): Foo is blue. Jill (thinks): Foo is red. Jack: "The sky is foo!" Jill: "The sky is not foo!" Jack: "Yeah it is!" Jill: "No it's not!" ...
Of course the myth of self-regulating Open Source projects is just that - a myth.
The reality, and why this model does work over the 'established' corporate model, is manifold:
* Centralized control in OS works, because the people in charge are so by virtue of their qualifications, and because they are accepted as thus by the rest of the developers working on the project.
In corporate environments, traditionally the least qualified individuals are hired to oversee and manage the project - this is even more true for ANY internet related venture. Furtermore, these people are hired because of their resumé, not their qualifications.
* The people doing the work, or managing the project do so because they have a stake in the project (the proverbial itch), and because they approached the project because of interest, not a paycheck.
In corporate environments, hiring chains are so long and convoluted that people are, again, hired by resumé and committee, not based on true qualifications, or interest for a project.
You *can* still get qualified personel in both instanaces using the corporate methods, but either it's rare, or if you get someone who knows what they are doing, their input is squelched by those above them.
Coporate structures are total, centralized empires, with small fiefdoms battling continuously for their own mini-empires, and self-justification - while not getting much of anything done.
OS projects are benign monarchies, that achive an objective because everyone is involved in wanting the project to succeed for their own reasons.
That probably sums it up best.
Harry
I'll believe it when I see it. The modularization that I refer to is not just an issue of organization, the issue (as I see it) is that it requires less effort/drive than a simple highly modularized driver does. Modularization in Linux's scope allows for a certain lackadaisical approach, that I don't believe larger [ even modularized] parts can. There is only so much that one can modularize away before it becomes excessively costly to performance.
/told/ to do it, and /told/ to do it a certain way. Modularizing and laying out a frame work can only go so far; they don't fully describe a path in the way that a manager can. Granted, there may be some significantly large and complex parts to Linux, but a modern day RDBMS requires a certain level of singularly sustained effort.
.01% less likely to fail ACID compliance test, who is going to thank that developer for that, for spending the past 2 weeks of his time to attain a level of safety that only a large corporation is apt to notice [and only then, when things go bad].
The "structure" is necessary not just to restrict additions that don't conform, but to drive, motivate, and encourage. Not every project is going to be particularly thrilling to the developer, sometimes it requires something a little more than the self-motivated desire to "scratch an itch". It's not even necessarily about greed. A developer may well think their efforts are better directed towards something else of their choosing; sometimes they may be right, but I've also known a number of situations where someone with the requisite skills must be
I feel this will be especially true when the open source developers efforts fade more and more into the background amongst many others. When say, Alan Cox writes a driver for my particular piece of hardware, he's sure to recieve plenty of thanks, especially if it's fast. But when one modification of one condition of an INSERT statement that is just
I realize many people will disagree with me there, but I remain skeptical. So I reiterate, I'll believe it when I see it. There may be a way, but not the way popular "Open Source" is envisioned.
This raises an interesting point. At some point any large project becomes too complex for a single person to control changes competently based solely on his technical knowledge of the project.
Take the Linux kernel as an example:
The project manager (Linus) is no longer capable of effectively controlling the project based on the fact that he is a great programmer. The fact is that the kernel is now too large for any one person (including Linus) to understand all facets of the project well enough to prevent breaking new releases. Thus, the value of a version control system, such as CVS.
This leads to an interesting situation when the project manager (in the above example, Linus) refuses to use the tools (like CVS) which would allow the kind of quality control necessary to prevent new versions from being broken due to the complexity of the project. The effect, in this example, is that fairly often new releases are broken by the very person who provides the technical leadership for the project.
How should the community deal with a crucial open-source project led an ego which is so great that it refuses to ensure quality at a fundamental level through simple, standard, and reliable mechanisms like CVS?
The Linux kernel certainly has a centralized (and hierarchical) control structure, but there isn't really a "Linux Roadmap" there's Linus' plans for the kernel, and things he insists happen before a release, but he does not say "we need a posix threads interface". He does, to some extent, make it clear what he'd like, but while he suggests, and encourages direction for the kernel, does not dictate it.
Secondly, with any GPL product, large or small, you have the right to take the whole thing, make your own modifications and push it as the way to go. Try doing that with the real-life component produced by the company of a real life CEO.
Of course, attracting support for your version, your vision, is something else. But if you have the talent and the product, it can be done.
For a start, if you can provide the goods you can march straight into the "boardroom" and work right alongside the "CEO" on the kernel.
Bzzt. Wrong. Thanks for playing. The Ask Slashdot question I posted about concerned this VERY issue. The person had a problem that he had a fix for. He marched to the "boardroom", fix in hand, and was soundly ignored by the "CEO" and all his "VPs".
Secondly, with any GPL product, large or small, you have the right to take the whole thing, make your own modifications and push it as the way to go. Try doing that with the real-life component produced by the company of a real life CEO.
Of course, attracting support for your version, your vision, is something else. But if you have the talent and the product, it can be done.
Translation: If you don't like how the project is being managed, you can go start your own company and manage it yourself.
Note that this happens a LOT. Engineers left Palm and created the Handspring. Engineers left Intel and created Weitek (math co-processor for the 386). This doesn't change the fact in any way that large, successful OS projects have a rigid management heirarchy. Be they good a knowledgeable, or be they bad and uninformed, they are management and they are indeed rigid. You will comply with their decisions/edicts, or you will have to start your own company/project.
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
Why the fuck can't you take 2 minutes off to read the damn artcile before commenting? No matter how important what you have to say is, read the article first. Yes, mark me as a troll, I do not care, but I read the entire article, word for word, and it is in my bookmark. I also agree with 95% of the content.
------ Curiosity killed the cat. {satisfaction brought it back | it didn't die ignorant | lack of it is killing mankind
Let's not forget one of the most important points about coders on OS projects - they are more interested, dedicated and interested than coders that managers are usually called upon to "manage".
Coders on OS projects are usually working on it by choice and have personal motivations in the project - or at least in their aspect of the project rather than working on code assigned to them by "management". These workers are easier to "manage" because they'll do the work anyway - they don't need to checked on or motivated by posters of eagles snatching fish or fill out time cards.
The management falls into a larger view of actual "project" and "code" management rather than hands-on or micro management. The bottom line is that ANY project of a significant size needs management if you ant it to reach a predetermined destination - but OS projects should need less management as long as it has sufficient coder support and interest.
=tkk
Bill Gates - Creationist?!?
I also feel you're giving too much power to marketing. Marketing is not "on the same level" as development, it exists in a totally different dimension, and should be viewed as a necessary thing given that we're developers, not as an end in itself.
The fact that marketeers think that marketing is an end in itself is a big problem.
I might be misunderstanding you, though, because in the Open Source case you seem to consider marketing as "what the user community wants" rather than as the process of finding out what the user community wants, and selling it to them. (Just because money doesn't change hands doesn't mean the OSS community does no marketing!)
--
It's a
-- Danny Vermin
Therefore, while perhaps the kernel dev. is not managed as a bazaar, it is quite clear that if we stand back, and we call all distributions forks in the road of Linux, though it may be bizarre, we most certainly have one big bazaar! (sorry about that one)
Professor quote of the day:
"When I first introduced this, everyone thought I was crazy. The TAs were scared, other professors wouldn't talk to me, but the students said, 'this is too easy, give us more!'." -Professor XXXXXXXXX, CS31; discussing one of the major assignments for the course
...but you've already made at least one fallacy in the intro. Having "strong central control" is orthogonal to "self-organization". The former is about who is in control the latter is about how they got there.
What's the diff? Well, in a non-self-orgranizing project there are by definition external (i.e. non-contributing) organizing forces--this is management. Linux doesn't have that, despite having "strong central control".
--
Linux MAPI Server!
http://www.openone.com/software/MailOne/
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
Of course, you're going to see some of both, but I think the 'bazaar'-style projects would usually have to be simple enough for the average coder to at least understand and modify parts of it, with a positive effect on the codebase. Otherwise, they'll just do something else, and leave the hard stuff to a core development team (cathedral-style) or let the code rot.
However, I think Angband is a good example of some free code that has mutated as people have changed it. There are a few main developers, and they do accept patches, but there are also tons of forks, and some healthy, nifty add-ons. (Zangband, Mangband, the Angband Borg...)
I would love to see the Tk version finally back-ported to Unix, or for that matter, any graphical front-end on top of X would be nice... If I have some more time, I'll try to work on that. I have messed with the Borg code before, and it wasn't that hard to do; I got it to use (and not sell) my Rod of Restoration, and to value items more like a Mage and less like a Paladin...
---
pb Reply or e-mail; don't vaguely moderate.
pb Reply or e-mail; don't vaguely moderate.
The key aspect of open source remains, however - once it reaches critical mass, for whatever reason, the project quickly takes off and you would be wise not to stand in the way.
--
I am co-developer of fancylogin, a neat login program for Linux, and we have big troubles organizing ourselves. Especially in the beginning we often had dialogues like "did you already implement feature foo? - No, I'm currently working on feature bar - damn, so do I". Software engineering in general can't work in this way, not only at open source projects.
A monkey is doing the real work for me.
Here we have one extreame view claiming that the other extreame is wrong, will ppl never learn? --typo
Wow, man !!!
Some 200 years ago a guy named Sigmund Freud said a similar thing: ".. and as evidence I examined this 3 ladyes and their condition matched exactly my theory, that I, by the way, made based on my observations on those same 3 ladyes...
Now, I'm not saying that your results are wrong, only that the method to get them were wrong.
--
Why pay for drugs when you can get Linux for free ?
echo '[q]sa[ln0=aln80~Psnlbx]16isb572CCB9AE9DB03273snlbxq' |dc
The difference is that Open Source Projects involve programmers managing programmers. As such, there is a level of understanding that is not present in a corporate bureaucracy, where feature sets and codebases are controlled from "above." While there must a a controlling authority of an OS project, that determines which modifications get included in the codebase, there is no guiding principle for development other than putting out good code--no hidden motives and no idiot managers.
ByteMyCode.com: A Web 2.0 code sharing community.
He quotes ESR as saying "the development of the Linux operating system by a loose confederation of thousands of programmers -- without central project management or control" to hint/point out that the statement is misleading. But, ESR meant to say something more like GNU/Linux, or even GNU/BSD/Linux/X/and_then_some which was a collection of many disparate projects. Not that the whole thing managed itself anyway: Unix has been around and honed for a long time, so there was a central architecture that was being worked toward.
Some years ago I tried to participate in several open-source projects. What really surprised me is that nobody seemed to notice my and other's bug reports. I can not insist on inplementing fixes at once, but hey, guys, say something ! Later I wrote a letter to Raymond, after reading his 'Cathedral and Bazaar' pointing to the problem. He answered that there is no such problem. I am using a lot of free software, but the best free programs were made by individuals. The Raymond's famous article discusses a kind of new relationships between people, the whole new culture. As management is 80% psychology and psychology is 80% based on culture, it means that new culture needs new management. They are talking about 'gift culture'. But nations with this kind of culture (Polynesia, Russia) somehow managed to achieve some, maybe humble, results. It's a topic to study.
What we need is a source forge like system that, once a project hs been created, lets anyone upload a patch/update, and have that fork (or a "press release" for it) appear on the origanal front-page.
Its just an naked idea, but it could be developed into a powerful system. The big boys of OS should think about it.
Well, thats my idea. I'm letting go now. Develop it, fork it.
Thad
Thad
Maybe the reason that open source projects work more effciently is that geeks are the only people who are capable of writing a spec that programmers can understand.
Only two things are infinite, the universe and human stupidity, and I'm not sure about the former. (Einstein)
From the article:
"Given enough eyeballs, all bugs are shallow"
and "Debugging is parallelizable." These assertions simply are not true and are
distortions of how the development of fetchmail proceeded. It is true that many people,
in parallel, looked for bugs and proposed fixes. But only one person (Raymond) actually
made fixes, by incorporating the proposed changes into the official code base. Debugging
(the process of fixing the program) was performed by one person, from suggestions made
by many people.
This is utter nonsense. Finding bugs and coming up with patches is the MAJORITY of the work. Simply reading patches, do minor mods on them, and applying them is absolutely minimal, compared to the work put in to the patches themselves. Noone is claiming that open source keeps anyone from having to work at all, just that collaborative development allows the workload to be better distributed. No matter what, there always has to be central control of the tree, or chaos results, whether that's in open source or not. However, in open source, you don't have to worry about scheduling people for things - people fix what they think is broken, you just have to be there to put in the changes. And, if you're not, guess what, someone else will take over tree control. The difference is that the management overhead (not the coding overhead) is completely eliminated.
Another point is that open source is self-organizing. That doesn't mean that a magical structure appears (like the author perceives it), but that natural leaders emerge (i.e. - the ones that code). If you started a project, people look to you for leadership. If you contribute a lot of code, the leader will give you a lot of authority in the project. This differs from normal management in that (1) the leaders are obvious, and
(2) you won't ever have a situation of inept leaders leading talented programmers (they will simply find one of themselves to lead). None of this takes management overhead.
Then he mentions how all of Linux kernel development halts when Linus goes on vacation. This is utterly not true. Development continues at the same pace as before, but the kernel tree does not get updated. As the Linux users and developers, _we_ have decided that we'll hold off on committing to the tree while Linus is away. Otherwise, we'd make our own tree. In fact, Alan Cox did this for a while, with very good results (many people for a while used Alan's tree instead of the main kernel tree, because it was better kept). Thus, if there is a flaw in the system, it is easily removed.
He then gives an example of how he thinks bazaar-style really works if it is really in a bazaar style (he argues that Linux and fetchmail aren't). I'll post it here:
Someone creates a minimal working version of the software. (This follows Raymond's
advice to start with a "plausible promise.")
The originator releases the working program, a description of how to use it, and all
the source code to an appropriate forum such as a newsgroup or public web site. From
this point forward, the originator becomes just another member of the user community,
with no special status.
Anyone may download the program and source, try it out, look for bugs, and suggest
fixes and enhancements. These ideas are communicated to the entire user community
through the forum.
Anyone at any time, or multiple people at the same time, may decide to create a new
version of the program. They do so by using ideas and code from the user community,
along with their own contributions. They post the new version to the user forum.
Most likely, the code "forks" as several people create new releases at the same time.
This is part of the bazaar process.
The user community attempts to settle on the best fork to follow, by trying all
available versions and focusing their attention on the best version. No single person or
small committee manages this process. Perhaps the best fork is widely recognized and
quickly selected, perhaps not.
Several forks may live in parallel for quite a while. If so, it is the decision of the
user community no one fork is the clear winner. When the community tires of parallel
forks, they will select one to follow.
Development continues in this dynamic, organic method. Leaders emerge briefly, as they
create a new release or argue for one fork over another, but they then become equal
community members again. All decisions about features, design, bug fixes, etc. are made
in this way.
The strange thing is, this is what happens in both the cathedral open-source style and the bazaar open-source style. Take for example, GCC - it existed in egcs and gcc form for many years. However, egcs was technically better, and this was recognized, so they merged. Take the Linux kernel. He is wrongly stating that there is only one tree. Let me see how many trees I can count:
And then remember that for a long time, PCMCIA was on someone else's tree, and it wasn't until recently that they merged. GNOME works in the same way. People just think in terms of Linus's tree, because that's an easy reference point. However, Linus's tree is probably the least-used of any tree in a real environment.
This guy obviously knows nothing of what he is talking about.
Engineering and the Ultimate
Take this quote:
Anyone here ever use Emacs? What about XEmacs? Isn't that exactly what Connell describes?And here's another tidbit which he says describes "true" bazaar work:
Someone her please tell me that I'm not the only one who knows what the Perl Pumpkin is? Someone?While he may have some intersting points, someone ought to whack him with a clue stick a couple times...
-B
Ash and Hickory, straight-grained and true, make excellent bludgeons, dandy for the cudgeling of vegetarians.
Now what was so bad about that comment that I get tagged as a troll?!
~~~~~~~~~~~~~~~~ Blazemail rocks!
You are confusing what the metaphor is. The Cathedral vs. Bazaar metaphor (at least as the author of the paper in question is using it) is between the method of their being constructed, not the final constructions.
I confess this entire metaphor has irritated me for some time. It betrays a lack of understanding how actual cathedrals were actually built. In the days when they were popping up like toadstools (i.e. ~12th century), the "architects" were an exalted flavor of stone masons, and as such shared a technical vocabulary and culture with the skilled craftsmen (implementors) who worked for them.
It irritates me that these medieval engineers (such as Villard de Honnecourt who was the Geeks' Geek of his age) are being lumped in with PHBs and the skilled workers who dressed the stone and raised the arches are being characterized as peasant grunts -- both are thereby slandered.
----------------------------------------------
-*- Any technology indistinguishable from magic is insufficiently advanced -*-
Did you even read the article?
/nobody/ would dispute that he's the central manager, the person who vets all ideas before they can be implemented in the language or interpreter. Hardly the "bazaar" that Raymond posits -- in fact, he's very much a traditional software project manager in this regard.
He doesn't speak to the viability of OSS; he is disputing the often restated premise that open source projects "manage themselves."
You mention XEmacs and Perl; wouldn't you say that both projects have distinct and very stringent management structures? Sure, XEmacs forked from FSF Emacs -- but it's just as strictly controlled.
And while Larry Wall does indeed delegate authority to the various Pumpkings,
The author is making a point about the organization of large software projects, that is orthogonal to the licensing issue. What his conclusions seemingly do imply, though, is that open source is not by itself a revolutionary development in the world of software engineering.
(jfb)
To spur "enterprise Linux," Big Bang, the distributed two-phase commit.
Key to designing any successful software project is having strong input subject area experts. It's no suprise the developers can write good compilers and programming editors, because these things are an integral part of developers' first-hand experiences.
The problems start when developers write software for problems that are outside their personal skills or experiences. Without the structure of a conventional organization to ensure that software get designed and written in a user-centric manner, projects like this are doomed. And even with such an organization, things are probably going to get very painful for everyone involved.
Of course there are the developers who have a deep understanding of disciplines outside of the realm of programming, and these people often go on to create great software outside of the realm of developer tools. Photoshop is a great example of an application originally developed by a programmer-artist. (And maybe Gimp too, but I don't use it, because I'm willing to pay for my tools, and while my primary development and mail-reading paltform is FreeBSD, I do my design work on a Mac.)
Anyway, how many people are there out there who: 1.) are technically clueful enough to program their way out of a paper bag, 2.) have the administrative skills to get a project out of the one-person phase, 3.) are equally talented in some non-technological discipline, and 4.) have been brainwashed by Richard Stallman to such an extent that they don't reaize how successful they could be writing commercial software?
I don't think such people are very common.
I don't entirely buy this. While an absolutely tremendous amount can be done via email, don't knock high-bandwidth communication, i.e. f2f, or the dreaded meeting.
Meetings don't necessarially have to be what you think of them as -- they can be simple hallway chats or dropping by someone's office. But frequently they are *essential* to breaking through conceptual logjams and coming to a consensus.
I'm currently involved with a commercial development project in the US and Europe, and we use email, but we also have weekly videoconferences, and exchange staff every quarter. You just can't beat it.
I think we have here a which came first, the chicken or the egg question?
The fact is that open-source projects can not be controlled by anyone, but everyone has a need that requires other peoples cooperation, so there is a strong pressure to unify to get maximum benefit from a limited number of developers. In a propriatory software world, the pressures are to differentiate yourself from competitors - however old style competition does not make sense in software development.
In a free market society, having seperate factories for say shoes creates pressures to keep costs down for each unit and innovate - however in software that model breaks down - having seperate software development shopes developing the same code is silly because the natural cost of copying is minimal compaired to providing to people who need service or improvements on existing code bases.
Let us not forget that intellectual property is not a basic property right, just because the government calls something a property does not make it so. It was never even designed to be a property right, and it's ignorance of fundameltal economics like natural limits in supply and demand (which is artifically put on information instead of services) dooms it to failure, as well as closed centralized software projects.
"MOEQ? Managers of excellent quality? I don't think they exist."
With apologies to the ROUS's of The Princess Bride.
--
As a matter of fact, I am a lawyer. But I play an actor on TV.
One problem is that the role of 'technical manager' doesn't really exist in the standard business world. I see very few companies that have technical managers, or, if they do, have ones that are technically competent.
OTOH, I know people who would be technically competent who shy away from any kind of management role because they no longer get to do interesting things to code. They no longer get to make technical contributions.
The Open Source model solves both these problems. I think the problem of good technical managers being incredibly rare is more a failure of the standard business process to find and select such people than it is a lack of capable people.
I just read the book 'The Cathedral and the Bazaar', and I noticed the same problems with the 'Bazaar' model as this author has noted. There was still a central person who made most of the important architectural descisions.
The Open Source model mainly takes advantage of the fact the the net makes wide hierarchies much more manageable. It also reduces a lot of the corporate politics associated with management hierarchies because there are many fewer steps to the top.
Need a Python, C++, Unix, Linux develop
I haven't read it either, but I was going to say the same thing. Whether there is centralized control or not, and whether the project "organizes itself" are not related.
More to the point, the voluntary participation of all involved means that the better (from the participants' point of view) manager is available to them. If they aren't producing what he wants, he doesn't incorporate their work. If he isn't realizing their goals, they won't contribute. If the whole project is less successful from the user's point of view, they can "fix it themselves" or more likely use the product that is better for them. Don't get me wrong, I know there are bloody wars in any project, large or small. The difference is that when you're in a bad position at work, you have to consider changing your whole life to get around it. With voluntary projects you aren't forced to choose between your livelyhood and a bad manager.
Freedom of choice means none of the traditional metrics of software development matter. It doesn't matter who's code is cleaner, has less bugs, runs faster, looks better, who manages their project the best, who works the hardest, or which tastes better. You have the choice to pick the one that suits your needs, no matter what your role in the system.
This is the same mis-understanding people have about political freedoms. People seem to think that freedom is advertised as "automatically better", but that's not it at all. Freedom means you have to make your own life better, and there's noone else to blame but yourself.
Of course, this is true no matter what your environment is, but with free software and all other freedoms, there's less mis-representation. Noone is claiming to take care of you.
Here are some my random thoughts on management and software production. First two disclaimers...
(1) I'm spoiled. I work in a shop with some excellent managers.
(2) I also work in a somewhat atypical shop with thousands of programmers on the same project. While not exactly unheard of, projects of the size of mine are not exactly the most common.
If it weren't for our managers of excellent quality, little of what we need to get done would get done in an efficient manner. Our managers help prioritize hours to be spent working on defects, new features, enhancement requests, etc. They work with developers to come up with reasonable and workable deadlines. They do a whole lot of logistics so we lazy programmers can just kick back and code.
Now, in the past I have worked with some bad managers. Some of the managers of projects I've been on have been so bad that they would have been better with no management at all.
Good management is an essential part of the software development process. It doesn't necessarily take someone with the title of manager. But it does take someone that can manage the project well. Some few programmers (especially the high profile open source ones like ESR, Linus, and Miguel) also happen to be fairly competent as managers of their respective projects.
However, given that out of all people that do programming tasks relatively few are what I would consider 'competent' and given that out of all the people that do management tasks relatively few are what I would consider competent, the union between competent programmers and competent managers is certain to be pretty small.
have a day....
-l
Angband is actually developed in a bazaar style. There are a lot of variants, created when people think things should be different. Patches move between them as the variant maintainers see and like what other people are doing. People sometimes create generic patches, which are based on one variant but could easily be put into different ones.
It actually seems to work quite well, for what they're doing. Part of it is that there's a large artistic component-- determining what stuff should be in the game, and people have a reason to fork the code to suit their tastes. Part of it is that there are still maintainers, so there is a single person in charge of determining whether a patch is applied to a file; but the patch may go to many different people, or circulate as an unofficial patch.
Linux (for example) does seem to follow this model if you include the large collection of unofficial patches which circulate. It's just that what you tend to see is the cathedral version of the program on the official site.
I believe Charles got a point, even if it is well known to all the people working with Open Source. The weakness with open source is that the project probably wont optimize its use of resources, with people doing the same stuff. The strength is on the other hand the fact that many people working on the same feature enables different solutions on the same problem as well as superior ways of fixing bugs and such. I believe that the person who acts as a moderator or patron over the bazaar needs to view the different solutions as well because they may be better solutions in a new environment and with new criterias.
Sigh...
Rob
I agree with the analysis, but not completely with the conclusions. Certainly, even in Bazaar style projects there is central management. And even in Cathedral projects, good ideas originate outside of the central authority guiding the project. The dichotomy between the two is a metaphor to describe the degree of control that the central management of the project exerts, the degree of ownership that is claimed. It helps explain not the actual flow of information (ideas and code), but the psychology of the relationships between the actors.
There are elements of the Bazaar in Cathedral projects. Many of the elements of Emacs began as ideas that were never suggested by Richard Stallman. For all his coding talent and vision for the future of his creation, he is still one man. He eats and sleeps and lives the same 24 hour days as the rest of us. Others come up with answers to their own problems, scratching their own itches. His control has been as a gatekeeper, determining what is in and what is out.
The article itself gives examples of the Cathedral in Bazaar projects. But it overlooks the spirit of the participation. There is a feeling of shared ownership. To the extent that a volunteer participates in the project, he feels a joint ownership of it.
Perhaps I am merely stating my own impressions. I have contributed in minor ways to both styles of projects. Both are open enough for my taste. But I can feel the difference in the degree to which the control is perceived to be exercised. It is really a difference in management style rather than the presence or absence of management. But on such things a project can succeed or fail.
The net will not be what we demand, but what we make it. Build it well.
... am I the only one wondering how you'd set a flamethrower to "stun" ?
In your "better, more sane business model" what you are effectively saying is that its the developers who need to run the business side of things - so who then, will do the developing?
Antidisestablishmentarianism would lose its point if it were hyphenated
RE: Hmm. Not sure about this. Management is a priori "heirarchically superior" because the management commands the developers.
Well, I'm arguing that its proper function is horizontal co-ordination, not a Mosaic "message from above".
RE: I also feel you're giving too much power to marketing.
Depends on the organisation. Many are marketing driven. They'll slash tomorrow's throat to sell 20 units this month.
RE: Marketing is not "on the same level" as development, it exists in a totally different dimension, and should be viewed as a necessary thing given that we're developers, not as an end in itself.
I'm saying that if marketing, sales, development, etc. are properly to be viewed as teams, then we can only succeed by having them all on the same level.
RE: I might be misunderstanding you, though, because in the Open Source case you seem to consider marketing as "what the user community wants"
As DISCOVERING the user need and keeping on at it, yes.
--- Jump!! Fire!! Bullet time!! - Lego version of the Matrix
Mmm, let's see :
A 'conventional organization' can hire marketeers, which should find out what users really need. But, often enough, marketeers are instead payed to find out how the organization (read:corporation) can collect the most money from the users. Which is not the same than giving the users what they most need ( though there is a partial overlapping ).
An 'open-source project' starts often ( not always ) from programmers which want to scratch their own need. So, it looks like it will not be able to cover generic user needs. But, please consider that programmer does not necessarly need 'working as software engineer' or 'computer nerd': it only means 'person which has enough brain/attitude to write a program'. I use a driver which was written by a student of nuclear physics (iirc) ). So, open-source programmers are representative of more branches of uman activities than one would expect. And OSS programmers love having a user base (is part of the 'score keeping' ), so they may go after non-programmer needs. Of course, ego and personal interests get into the way (nobody pays them, after all, so they have full right to to whatever they fancy ).
So, I'll call it a draw. Both 'conventional organizations' and OSS projects can do a good job for users. Both sometime don't. That's why it is good to have both.
Ciao
----
FB
"Take a look at a major failure of Linux--its complete inability to produce anything resembling a coherent, useful desktop environment."
KDE would have to count as something that at least 'resembles' a useful desktop environment.
Stephen Molitor steve_molitor@yahoo.com
We are at first reserved to accept his cathedral analogy because it implies that the vision and direction of the project is from one source, and that is set in stone. Open source is organic: It's leaders are not chosen by seniority, resumes, or knowledge, they are chosen according to competence. By his logic, if Torvolds died tomorrow, linux would stop developing; The architect is gone. But that wouldn't happen; a new central leader would emerge, and that position would be thrust open someone based on their competence to do a good job. Open source (or all software, for that matter) does not have a beginning or an end, it consistently evolves and changes.
The other implication he makes is that projects in open source are necessarily 'steered' in a direction by a central manager. And while I wouldn't argue that a central figure would suggest direction for a project, he or she wouldn't limit or try to focus the development by only his/her own direction. Linux is a prime example of that; we find it now in everything from robust mission critical servers to home desktop PCs to appliances and wrist watches. Linux did not become such an adaptable operating system because it was carefully guided by one central figure, it became this way because the main central figure allowed it to grow.
He discounts the Bazaar analogy because it is, in his mind, too chaotic to produce anything of value. He seems to confuse chaos with freedom. In the Bazaar you can do anything you want, but if you want to succeed, you best do something of value. This doesn't mesh with the corporate vision (Either traditional or contemporary views of project management) because it allows an incredible amount of freedom with the developers at the bottom rung. A programmer may, one day, fix a security hole, and the next, decide to write a driver for his brand new joystick. A corporate environment is very unsettled by this: If nobody has an assigned task, how can anything get done? Things get done because the people doing them see a need for them to get done. They, as users, recognize a missing piece of the puzzle, and feel compelled (by whatever personal reason) to complete it.
And it is for precisely this reason that the open source model will never work in a corporate environment. Once a concrete direction is assigned to a project, it stops being organic. People loose interest. Moreover, people soon learn that the software they create for a company, any company, is ultimately created for the profit of the company, not for the good of their fellow users, and their true motivation will be lost.
The Internet is generally stupid
The second thing that caught my attention is the twisting of the Cathedral portion of the analogy. Did anybody here really think the paradigm was that of builders of Cathedrals? Isn't the idea that the Cathedral represents the top-down approach to managing software production, with an anointed priesthood in control? The ten commandments vs the thousand suggestions? Sure, most of the suggestions won't be worth the bandwidth used to send them, but I expected those would be culled out, not added without discretion. I don't think that expectation falls outside of the idea of the Bazaar either. If I'm running a booth in a bazaar, I am free to accept or reject any ideas about how to run things. If I buy a McDonalds franchise, I'll serve Big Macs and Quarter Pounders, not Whoppers.
--
As a matter of fact, I am a lawyer. But I play an actor on TV.
Management used to be relatively simple. You had some guy spinning a wrench on an assembly line, and you had some guy in a walrus moustache and a suit fuming and humming and making sure the work got done.
With software projects, management really should be seen as a separate enterprise, not a hierarchically superior one, to development, marketing, etc. You have two basically competing units - marketing (or in the sense of open source, what the user community is looking for) which expects everything under the sun, sure it's possible!, and development, which is asking for time and energy to create a good product to static project requirements.
In a commercial enterprise, management should be about insulating both marketing and development from each other, and making sure that both is on the same phase. Marketing should figure out what the customer really wants, and the development people should put it out on time and bug free. Management should make sure that marketing doesn't decide to abrogate its responsibilities and just nod to every moronic request any customer makes, that development has enough computers, manpower and software to do the job, and completes it to spec, etc.
Now, in terms of open source, the "there is no management" could come from the fact that there's no hierarchy, but the idea of managing a project is no less important. Management in this instance should come from the user base - instead of just creating, say, yet another dock applet mixer, the development team with the idea of doing XYZ should go out and say "do we need X and if so what do we need XYZ to do?" That way, when the English people say they need the steering wheel on the right hand side, and the Americans on the left, we don't build a car with two steering wheels, but one with a wheel that can be put on either side of the car as needed.
The biggest challenge is getting people to stay on track. In a corporate environment, if you don't agree with the spec, do it or quit. In Open Source, if you don't agree with the spec, split the development effort by fracturing the team, or whine.
--- Jump!! Fire!! Bullet time!! - Lego version of the Matrix
If fetchmail and Linux were not run as bazaar projects, what would a true bazaar project look like? A real bazaar software development method would proceed as follows.
Hmm; doesn't this sound suspiciosly similar to the way distrobutions work?
(The article did have some interesting notes though.)
Part of the success of an Open Source project vs. traditional methods is, I think lack of overhead and taking advantage of better communications.
People do not have to commute in to work in the same place, they do not have meetings in the same room, they do not have to fill out request forms, submit reports and deal with pointy haired bosses.
That is the no overhead part.
The better communications part is using email, CVS, IRC, Web and FTP. Businesses use these also but they throw in the physical meetings, telephone calls and reports. The latter dilute the former. If people are limited to email et al. things actually become IMHO, more efficient, not less because of the limitation.
(I waste a lot of time in pointless meetings where I work, and I believe others do also).
People involved in Open Source Projects seem (I am not in one) to only do work, not deal so much in administrivia.
There was an Ask Slashdot question a few days ago about how you make contributions to an OS project. Several of the answers went along the line of read the mailing list and send your bugfix to some of the lesser daemons. Let them take it to the great gods. This hints that an OS project (especially one as important as the kernal) can quickly have a structure identical to Figure 1 of the article.
If you have a problem at work, you talk to your manager, who will speak with the VP, who consults the CEO. How is this any different from emailing the 'less important people on the mailing list', who then contact the component owners, who might be able to get a response from Linus?
Aah, change is good. -- Rafiki
Yeah, but it ain't easy. -- Simba
This should be patently obvious to everyone. Whenever you have a project involving multiple coordinated contributors, *some* kind of management is necessary. It's just not the traditional business type management. The managers of free/open-source projects tend to do some of the work themselves, taking an active role in the writing and maintenance of code, as opposed to sitting in a plush office and telling other folks what to do.
N4st0r, trixx0r h0bb1tz0rz! Th3y st0l3 0ur pr3c10uzz!