Slashdot Mirror
Which Open Source Projects Are -Really- Collaborative?
An Anonymous Coward asks: "I'm a highly competent and occasionally respected software engineer, who has worked on several Open Source/Free Software projects; some of my code is in the Linux kernel. Within the OSS
community, we maintain that the main point of publicly announcing OSS projects is to invite programmers to join the project and collaborate to make better software. But in about 90% of cases, I've found that publicly announced projects in development are not actually open to new members - the project leaders will ignore unsolicited code, won't respond to emailed queries or suggestions,
and in many cases the projects in question remain in an early stage of development forever." What projects do you know of that don't make an issue out of incorporating user submitted patches and design changes, and what projects put forth huge restrictions on such submissions, even to the point of not accepting them at all?
"This happens even when the project has explicitly asked for collaboration, and it happens when the project leaders are big names in the OSS community as well as when they're relative unknowns. So my question is, who actually collaborates? Which projects make unsolicited development effort worthwhile by making it part of something bigger?"
I've found that, while a great many projects are considered 'Open Source', making the source available does NOT make an open source project. To be a truely open project, you also have to have an effort to do things such as merge and include changes that have been provided BACK into the project.
Many the project has a list of 'contributed' changes, with many things that NEVER actually make it back into the source tree. Why? I'm unsure, beyond the fact that many individuals who do these things, do so for themselves, and are nice enough to make the source available. They are NOT, however, interested in fostering a community around the further development of their code. The blokes who wish to step up and take on these reins are few and far between, and it is really a different 'role' then many of the developers out there WANT to have.
In order for a project to do this, it requires 'evangualists' that wish to take on these roles. Some people serve both roles, indipendent developer AND evangualist. These people serve as the best 'Project' leaders. They posess skills that many people dont have, only ONE of them being good developers or engineers.
I dont think this is a BAD thing, really, it just determines which projects will be wildly successfull, and which ones will simply be pet projects, used little, possibly important, but very often, eventually abandonded.
Perhaps a way to say this would be an 'Open Source' project, versus an 'Open Development' project.
-- I'm the root of all that's evil, but you can call me cookie..
Yes, and its not a bad thing. It just happens to be a project that suites its purpose, and thats it. It will never, however, be wildly popular. This isn't a BAD thing, so long as it suites the needs or its target audience.
Many times, the needs happen to be simply trying to do it. In that case, the developers themselves are the audience, and they gain the experience.
And many, OH so many of these small things, that end up sitting in sourceforge limbo, serve, at the very least, as GREAT sample code for the next project to do more then likely basically the same thing...
-- I'm the root of all that's evil, but you can call me cookie..
The author of the paper brings up a good point that ESR overlooked in his original paper Cathedral and Bazaar paper, which is that Bazaar style development does not necessarily mean Open Source and Cathedral style development does not necessarily mean closed source.
It is possible, and actually occurs quite often, that a project may release its source code licensed under an Open Source license but has a development process that is elitist and closed (one has to look no further than the *BSD camp). Similarly it is possible for commercial projects to be developed in a Bazaar style manner especially with the rise of software development techniques like Extreme Programming where no one specifically owns a particular part of the project and people are encouraged to participate in all parts of the code and as well as test and review all parts of the code.
I thought this would be some interesting food for thought.
You will find that if you submit a diff to the list and author of a project which is active, they will be quick to either accept it, or explain why they didn't. After a few good patches on the list, you can ask for CVS access and work on projects as you wish!
I remember pre-KDE-2.2 working hand-in-hand with the release manager (cheers to David Faure!) to update libraries while I updated code. Bouncing patches back and forth to get it 'just right' for my app needs and the overall needs of the project is when you feel open source is working best for you!
...PLEASE do not think that only open-source collaborative projects are good! The quality of a software is the software itself and the associated license (choosing a good open-source license is essential). I think that big open-source projects *have to* be open to external contributors because one man can't do everything. But having a good coder for one small open-source project is sometimes better than having several students in software engineering too much self-confident. Also the fact that it's open-source software guarantees the possibility to fork when people are unhappy with the maintainer.
Similarly it is possible for commercial projects to be developed in a Bazaar style manner especially with the rise of software development techniques like Extreme Programming [extremeprogramming.org] where no one specifically owns a particular part of the project and people are encouraged to participate in all parts of the code and as well as test and review all parts of the code.
Ownership doesn't necessarily mean non-participation of others in test, review, or contribution of code. All ownership does is divide a large project into areas of responsibility. How the owner administers his piece, and whether multiple owners within a project have a common style, are separate issues from whether there's one owner for the whole project or a set of owners for the components.
Typically an owner or his team negotiates interface definitions with the owners of adjacent components. Beyond that the owner makes final decisions about feature set and what code is included in the component. He may keep control of it all himself or break it up and hand pieces to subordinate owners. He may keep his deliberatioins private, make them public without accepting feedback, or invite participation. He may accept contributed code and integrate it, accept suggestions but write the code himself, reject assistance, or even actively try to do something different from what is suggested. He may do his debugging himself, accept bug reports, or stage walk-throughs. And so on. Finally, he may change his policies with time.
All ownership does is create a clear set of responsibility boundaries in a multi-person project.
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
Alot of the submission-friendliness depends on the ego of the maintainer. If its a person who wants to head up the project just for the fame of it, you'll probably find it hard to submit good high quality code.
Such maintainers are probably more likely to take the ideas from your code and implement it themselves, for better or worse...
I am a maintainer of a project myself, and I know that any submissions that come to be will be scrutinized quite harshly for quality. If they're not up to my own standards then I wont accept them. My project is rather small tho, and very focused.
I've also submitted code to other projects. In one instance the code was completely ignored. In the other instance the ideas of the code was implemented and I was credited for it, but it still wasnt my code that was accepted.
For those who wants to contribute code, the most valuable code you can give is probably that which is nonexistant. Code that offers functionality that is wanted but for which code doesnt exist yet.
Code that simply tweaks currently existing functionality will have less chance of getting accepted I think.
If all else fails, you can always look into forking the project. If you are unhappy with the current maintainer, maybe there are other people who are aswell? Can you do a better job? If so, dont be afraid of forking, thats what free software is for I think. Letting the best man do the job.
/proton
I've watched the wine-devel list closely for the past 3 or so years, and I've observed the following:
With FreeBSD and PHP it took a little while to get my changes in, because the people working on it are volunteers, and believe it or not, they do have lives of their own. In the FreeBSD case especially though, my code was thoroughly audited, and I learned quite a few new tricks in the process.
For both Subversion and Mono, from what I've seen the developers are very very good about incorporating changes quickly. And again, specifically in Subversion, they're very good about auditing your work to make sure it's the best it can be.
So yes, while some projects can take some time to get your stuff in, and some can be quite exclusive, there are a lot of good ones out there, you just have to look for them.
Here's a couple guidelines off the top of my head:
.tar.gz. If you're going to be doing it anyway let others benefit.
1) Follow the goals of the project
Usually a project leader will have in mind where he wants the project headed. Follow it. Ask him about it if you can't find any information about this on the web page or mailing list. (Sometimes a project is organic however).
2) Follow the existing design unless it's broken
Don't change the design unless you can articulate good reasons for it. This forces people who already know the existing design to take time to learn a new one.
3) Coding style
Follow the style of the rest of the code. Some people will reformat it for you if it's good enough, but don't bet on it.
4) Keep it manageable
It's difficult to read and verify large patches. Send separate functional pieces if possible. It takes me much longer to merge big patches than smaller ones.
5) use cvs diff
Unless keeping it manageable prevents it, use 'cvs diff -u'. This generally makes things easier for you and whomever is applying your changes. Especially if you're never made a diff before.
6) Tell the project leader what you're doing
Even if you're not going to be done anytime soon, let someone know what you're doing. I had two people come up with independent debian packages for a project because one of them didn't mention it to anyone.
7) Put it on a web page somewhere
If your patch doesn't get merged put it on a web page. Send the url to the mailing lists and keep it up to date. Maybe provide a prepatched
That's all I can think of at the moment. I try to reply to all patch emails even if I reject them but some people don't have the time. Don't feel bad if nobody replies, just manage the patches yourself if you find them that useful.
Rick Haines (rick&kuroyi!net)
http://dxr3.sf.net
From that start of Kinkatta (formally Kaim, kinkatta.sourceforge.net) two years ago one of my top priotities have been to get others involved. When I get e-mails I try to answer them right away and if it is a problem I save the e-mail and send another repley once it is fixed in cvs. I constantly ask people for problems that they have so I can fix it, I have been accepting patches and all of the other normal things a good lead developer should do. I figured that doing all of the above would make my app the best it could be. All of these would get people involved and they would know that if they submited their problems that they would not be ignored. When other developers have found interest in working in Kinkatta I have tried to help them out in getting involved. I wrote a hacking file that helps expleain what code is where. I would give them little bugs to try to fix and so forth until they were comfortable with the system. I do not think that I would have been anywhere as successfull if I hadn't done any of these and am surprised that people arn't doing them in the first place.
Do you changes clothes while making the "chee-chee-cha-cha-choh" transformation sound?
To some programmers, having working code is not as important as having the experience and knowledge of the codebase. These are the guys who will understand the codebase in months and years to come. To me, accepting a large number of patches early on in the dev process could lead to a sordid understanding of how and why everything works. (Will your static char buf[] fit on the PPC stack in that reimplemented sprintf() call? What? You didn't consider that when accepting the patch last year?)
I've had code turned away by the svgalib developers, because my implementation wasn't full enough. Sure, I coded in dvorak support before they released a version that could do it a couple years back, but my patch was too hacky for their tastes. Theirs allowed a true remapping of the keyboard in raw mode. Mine was an on/off translation switch.
You can always investigate the CVS logs and the CREDITS file (warning: shameless plug for my own open source/free software project).
Regards,
Zooko
Hacker, Evil Geniuses For A Better Tomorrow
PostNuke is a good example of a true collaboration that not only allows, but encourages individuals to participate at all levels. Meaning, it's even okay of you're not a programmer.
... in fact, the PostNuke project was started because of the closed open/source nature of PHPNuke.
For those who don't know. PostNuke is a weblog / Content Management System (CMS). It offers full CSS support, HTML 4.01 transitional compliance throughout, an advanced blocks system, and is fully multi-lingual enabled. PostNuke is a fork of Francisco Burzi's PHP-Nuke
healyourchurchwebsite.com - WWJB?
- Does what the contributor is offering line up
reasonably well with my vision of the project? Sometimes people are looking to do something very different with my code than I intended
... perhaps radically expanding the scope of a project for instance. I don't mind people doing this, but I tend to be unwilling to accept this all back into the core, thereby becoming somewhat responsible for it, preferring instead to just point off to
their work.
- When supplying patches, does the contributor
explain clearly what the patch is supposed to accomplish?
- Does the contributor seem competant?
Some situations where I didn't end up utilizing patches include:- Patches to get things building in environments that I don't use, and will have
trouble maintaining, like Borland C++. I can't
test the changes, and I can't maintain them.
- Changes to fix a small/uncommon problem that
I fear might break lots of other stuff. This is especially true of "configure" changes that tend
to be very fragile, and that I often have trouble
being confident of the ramifications.
- Library bindings for environments I don't use
(like python, tcl or perl bindings). I will often
dump these into a contrib source tree, but I don't
want to be responsible for them.
- Contributed ideas that will be hard to
utilize. Some people think they have great ideas
but are not in a position to implement them and
think I will be thrilled to have their ideas. Well, they can offer them but don't expect me to
change the direction of my project to try them out.
I would like to note I have had very good experiences submitting patches to the Mesa project. Patches I submitted with detailed explanations have been applied within a few days.I have also had excellent submissions to many of my projects, most notably to libtiff which is of wide use and interest. In the case of libtiff though, I have found some patches to be difficult to apply because it is hard to be sure the submitter has thought through all the possible ramifications.
Finally, people have many things drawing at their time, and it is often difficult to find the time to review a patch and apply it to a project you aren't actively work on at the time. So to the extent possible, make it easy for the maintainer.
Geospatial Programmer for Rent
I may be feeding a troll, but, just in case it was an honest question, we already have a very simple tool that helps people fork. I have used this mysterious and little-known tool to maintain my own forks of several projects when I was making changes that couldn't or shouldn't be folded back into the main tree. (To truly be effective, you should use it in concert with another hard-to-find tool.)
--
pétard
.sig: file not found
I have several tiny open source projects for which
I receive patches from time to time. I do not accept
all of them.
Some of patches are not satisfy my quality requirements to the code. Being professional developer, sometimes when I see code written by some 1st year CS student I would rather rewrite it.
Some of patches are platform specific, and integrating it into real project
requres gread deal portability tweaking (#ifdefs, configure.in changes, etc.). I am not always have time to finish patch for contributor and they are not interested in making in works besides their platform.
Some patches are solving small local problem without taking bigger picture into the consideration. I would
later implement more generic solution.
My words to contributors: when you submit the patch, this is not end of your involevement. Good contributor
supposed to work with project maintainer to refine the patch. Also project migth need your help fixing bugs
if they are found later in your code.
Go to sourceforge and look int he "help Wanted" needed. There are always tons of projects there that need developers, Doc writers, Web Developers, QA perople, everything!
One open source business model that has been ignored is that of a coordinator between users with needs and programmers willing to write open source code.
I've submitted code to some projects (like util-linux, sendmail, and GNU libtermcap) and had my ideas accepted but someone else made the change (in ways that better fit the "big picture"). Nothing on my patches page for Cistron RADIUS has been integrated into that code, but that is because Cistron RADIUS is considered "stable" and doesn't get much in the way of feature changes (new features go into FreeRADIUS). I've had patches accepted for OpenSSH, wu-ftpd, and Cyrus SASL that went right into the core code base. When I suggested a change for Cricket (which is maintained on SourceForge), I was given direct access to the CVS tree and basically told to "make it so!"
Don't give up because your changes are not accepted. Talk to the maintainers, and if it is clear that they aren't interested, put your patches up on a web page and let others know about them. If they are useful and people use them, the original maintainers may decide to pick them up. If not, that's okay too - thats one of the great things about Open Source software!
Great! I completely agree with your points. Thus, I am sure that you make sure that
But I wouldn't call this a problem. Say 0.1% of all opened projects on Sourceforge (just as an example) will really become mature, widely used, big open source community projects. Aren't those 0.1% worth all the effort? Aren't those 0.1% worth putting up with thousands of dead projects because the developers didn't have time or weren't stubborn enough to carry it out? Would you rather have to certify yourself and prove your product's quality before you can post it on the 'net?
I myself started four OSS projects (one of them a community web site actually) with several people, and three of them are unmaintained ATM because I simply don't have the time to continue them. That doesn't mean they rot, though.
So, to summarize, I would LOVE to have 1000 closed, elitist, non-maintained projects floating round the Web - if that's what it takes to produce the one single genius project that changes our world. See Samba, Perl, GCC, Apache, or whatever you like ...
Home Page
Collaboration is great. Everyone uses it, even closed source development shops. But it can be overdone.
Pulic collaboration is a tricky beast. You CANNOT let just anyone commit into your code base. Someone you trust must review that code first. And if no one has time to review it, the code won't get in. This only makes sense. An Open Source software project is not a place where everyone wandering by is free to leave their mark on the fire hydrant.
The question to ask is "who gets commit privileges?" The answer is those you trust, and only some of them. Too many contributors and the project loses focus. Limit commit privileges to those who have proven their coding and domain expertise. But this doesn't mean that only a few people can participate.
I rather like the FreeBSD model. There is a small group with full commit privileges, but a much larger group called "contributors". These contributors may have commit privileges in limited areas of the code tree. Anyone else wishing to contribute may do so, at any time, but not directly to the tree. Instead the submit their code and patches to the appropriate contributor. Eventually a submitter will gain trust and be admitted to the ranks of privileged contributors, and maybe eventually to the core committers.
A Government Is a Body of People, Usually Notably Ungoverned
I wish I could mod this up. :) all I can say is 'yeah!' despite it being a bit too personal. Not everything is about producing big powerful software projects to take over the world. The simple ability to see the code and learn from it and (typically) incorporate ideas from it into your own stuff, or adapt it to a specialized purpose, has huge value.
The 'mtx' package in all its glory consists of 5,000 lines of code. This also limits the amount of 3rd party collaboration needed -- it just doesn't take a bazaar to write a 5,000 line program. Of those 5,000 lines, perhaps 500 lines were inherited from Leonard Zhubkoff (I re-wrote the remainder of the original 'mtx' pretty much from scratch when I split it into two parts, added a .h file, and changed over to use GNU Autoconf for configuring). The next biggest contributor is William Smith, who contributed the HP/UX port, which is approximately 150 lines of code. Those are pretty much the biggest chunks of the program written by other people. (Note: The 1100 lines of code in the 'contrib' directory are not included in any of this).
Yes, I receive patches. No, most of those patches are never applied. This is not malice. Usually the patches fix a particular problem with a particular loader by hacking around the problem. If a problem has a general solution, though, the patch goes into the garbage and I instead program the general solution. The last time this happened, a guy sent me a 5-line patch to make 'mtx' work with a particular optical jukebox. I looked at the problem, figured out a general solution for the problem for *all* jukeboxes that had that same general problem, and re-wrote the READ_ELEMENT_STATUS code for the third time, splitting it into three different subroutines and issuing separate SCSI requests for each element type rather than one big SCSI request. I ended up changing about 200 lines of code to solve the problem that this guy's 5-line patch "solved". The problem was that this guy's 5-line patch would have resulted in random core dumps on various architectures, though it worked on his particular machine with his particular loader, and his patch, while it solved the problem for that particular loader, did not solve the general problem (which was the problem of multi-drive loaders reporting "ghost drives"). My changes solved the problem -- but required an intimate knowledge of how the READ_ELEMENT_STATUS parser code worked, and how other loaders handled that same basic problem. I will not be modest here -- there was not another person on this planet who could have solved this problem within hours, as I did, and nobody did. I wrote the code, I knew how it worked, I could swiftly split up one big routine into three different routines and add a driver routine because I knew exactly what needed doing, no study required. Collaboration, in this case, was necessary in that I needed the guy who had the problem as a test bed for the changed code to make sure it actually solved the problem, but the patch was tossed because -- like most patches I recieve -- it simply patched around the problem, it didn't fix it.
I would count 'mtx' as a successful Open Source project. When I took over maintenance it supported DAT autoloaders with one drive and up to 8 slots. Today, it supports virtually every SCSI SMC-compliant device out there, including enormous optical jukeboxes with thousands of slots and dozens of drives. The participation of others was crucial for this -- but their participation was most valuable their suggestions, and for helping test whether my code actually worked with their loader, not for their programming contributions, which, with the exception of Mr. Smith, were negligible. I don't know how it could have been any other way, because I was the one who had the knowledge of the code and the knowledge of SCSI and the knowledge of all those other busted loaders out there, and someone who was just coming in from the outside with a particular problem with a particular loader simply does not have the knowledge, in the general case, to contribute a meaningful patch
-E
Send mail here if you want to reach me.