Which Open Source Projects Are -Really- Collaborative?

An Anonymous Coward asks: "I'm a highly competent and occasionally respected software engineer, who has worked on several Open Source/Free Software projects; some of my code is in the Linux kernel. Within the OSS community, we maintain that the main point of publicly announcing OSS projects is to invite programmers to join the project and collaborate to make better software. But in about 90% of cases, I've found that publicly announced projects in development are not actually open to new members - the project leaders will ignore unsolicited code, won't respond to emailed queries or suggestions, and in many cases the projects in question remain in an early stage of development forever." What projects do you know of that don't make an issue out of incorporating user submitted patches and design changes, and what projects put forth huge restrictions on such submissions, even to the point of not accepting them at all? "This happens even when the project has explicitly asked for collaboration, and it happens when the project leaders are big names in the OSS community as well as when they're relative unknowns. So my question is, who actually collaborates? Which projects make unsolicited development effort worthwhile by making it part of something bigger?"

Comments

It's a time problem

[Anonymous Coward]

I've worked on an open source project, but as soon as the code was in a state suitable for our purposes, we stopped development in the project and moved on to new frontiers. I think it's quite normal that a lot of these projects stop in the early stages.

Re:It's a time problem

[Thomas Charron]

Yes, and its not a bad thing. It just happens to be a project that suites its purpose, and thats it. It will never, however, be wildly popular. This isn't a BAD thing, so long as it suites the needs or its target audience.

Many times, the needs happen to be simply trying to do it. In that case, the developers themselves are the audience, and they gain the experience.

And many, OH so many of these small things, that end up sitting in sourceforge limbo, serve, at the very least, as GREAT sample code for the next project to do more then likely basically the same thing...

The ego of the maintainer

[proton]

Alot of the submission-friendliness depends on the ego of the maintainer. If its a person who wants to head up the project just for the fame of it, you'll probably find it hard to submit good high quality code.

Such maintainers are probably more likely to take the ideas from your code and implement it themselves, for better or worse...

I am a maintainer of a project myself, and I know that any submissions that come to be will be scrutinized quite harshly for quality. If they're not up to my own standards then I wont accept them. My project is rather small tho, and very focused.

I've also submitted code to other projects. In one instance the code was completely ignored. In the other instance the ideas of the code was implemented and I was credited for it, but it still wasnt my code that was accepted.

For those who wants to contribute code, the most valuable code you can give is probably that which is nonexistant. Code that offers functionality that is wanted but for which code doesnt exist yet.

Code that simply tweaks currently existing functionality will have less chance of getting accepted I think.

If all else fails, you can always look into forking the project. If you are unhappy with the current maintainer, maybe there are other people who are aswell? Can you do a better job? If so, dont be afraid of forking, thats what free software is for I think. Letting the best man do the job.

/proton

My experience with Wine

[knitfoo]

I think this is a very interesting question, and one that is rarely honestly discussed.
I've watched the wine-devel list closely for the past 3 or so years, and I've observed the following:

1. Most OSS Developers are extremely helpful. I can't begin to tell you how impressed I have been by the responsiveness of developers on both wine-devel and on wine-users.
2. Some OSS Developers can be very rude to newbies who annoy them (yes, Andreas, I mean you, but you're getting better *g*).
3. Some queries to the list are simply ignored. And not just ones where the author failed to RTFM; there are often cases where someone asks a question, and it doesn't push anyones hot button, and no one replies. Lists can actually be problematic this way. Have you noticed that if you send two people an email, your response rate is lower if you send the same email individually to each person?
4. Most new posters want to swing for the fences, not pick up the litter. This is a real problem, IMO, with all OSS projects. For example, Wine is hard. However, there's lots of good work a newbie could do (testing, doco, simple test cases, small projects that Francois does an excellent job of collating). Most new posters want to make a real impact with their work, not start in the mailroom, as it were. Hence the enormous number of OSS projects (why spend your personal time cleaning the litter of Wine when you can be the lead developer on BobsCoolWidget?)
5. Most OSS projects see a lot of newbies come...and go. They respond best to new people who stick around for the long haul. I guess it's like Minnesotans. They're not very friendly for the first 10 years you're their neighbor, but after that, you're like family...*g*

Want closed Open Source projects?

[Anonymous Coward]

Look at XFree86.

This is something that's been nagging me for a while now: Open Source is bad term, not only because it avoids the issue (Free Software) in favour of looking good in the eyes of the marketing people, but it also gives the wrong idea, namely, that it is open. It's open only in that you can see the source (which is arguably the only thing that "Open Source" is intented to mean), but getting in is not as easy as it sound at first glance.

In the particular case of XFree86 the matter is complicated by the fact that graphics hardware companies have this fixation about not openly releasing specifications about hardware that's old. By the time a graphics card hits the street it's old technology. Or do you somehow think that reverse engineering that card (say, 6 months from the point when you start to the point when you get your clone to the street) will give you some sort of edge? Please. By the time you got your product on the street, your competitor has his next generation product there, too.

How would you get any work done?

[IceFox]

From that start of Kinkatta (formally Kaim, kinkatta.sourceforge.net) two years ago one of my top priotities have been to get others involved. When I get e-mails I try to answer them right away and if it is a problem I save the e-mail and send another repley once it is fixed in cvs. I constantly ask people for problems that they have so I can fix it, I have been accepting patches and all of the other normal things a good lead developer should do. I figured that doing all of the above would make my app the best it could be. All of these would get people involved and they would know that if they submited their problems that they would not be ignored. When other developers have found interest in working in Kinkatta I have tried to help them out in getting involved. I wrote a hacking file that helps expleain what code is where. I would give them little bugs to try to fix and so forth until they were comfortable with the system. I do not think that I would have been anywhere as successfull if I hadn't done any of these and am surprised that people arn't doing them in the first place.

I would have to think pretty long about it

[Sludge]

I'm all for working software, but I think I would be more than mildly hesitant to accept code patches for my software project that apply themselves to tightly repeated loops or areas of code that *I* wanted to implement.

To some programmers, having working code is not as important as having the experience and knowledge of the codebase. These are the guys who will understand the codebase in months and years to come. To me, accepting a large number of patches early on in the dev process could lead to a sordid understanding of how and why everything works. (Will your static char buf[] fit on the PPC stack in that reimplemented sprintf() call? What? You didn't consider that when accepting the patch last year?)

I've had code turned away by the svgalib developers, because my implementation wasn't full enough. Sure, I coded in dvorak support before they released a version that could do it a couple years back, but my patch was too hacky for their tastes. Theirs allowed a true remapping of the keyboard in raw mode. Mine was an on/off translation switch.

Most welcome good help

[Burdell]

Virtually every project I've been involved with have welcomed good code. Sometimes, changes are rejected because they don't fit with the maintainers' "big picture" view of where they want a project to view, but usually they'll explain themselves if that is the case (and you can try to adapt your change or their viewpoint :-) ).

I've submitted code to some projects (like util-linux, sendmail, and GNU libtermcap) and had my ideas accepted but someone else made the change (in ways that better fit the "big picture"). Nothing on my patches page for Cistron RADIUS has been integrated into that code, but that is because Cistron RADIUS is considered "stable" and doesn't get much in the way of feature changes (new features go into FreeRADIUS). I've had patches accepted for OpenSSH, wu-ftpd, and Cyrus SASL that went right into the core code base. When I suggested a change for Cricket (which is maintained on SourceForge), I was given direct access to the CVS tree and basically told to "make it so!"

Don't give up because your changes are not accepted. Talk to the maintainers, and if it is clear that they aren't interested, put your patches up on a web page and let others know about them. If they are useful and people use them, the original maintainers may decide to pick them up. If not, that's okay too - thats one of the great things about Open Source software!

Re:Linus rejects many non-x86 patches to linux

[t]

There's a very simple reason for that. I would probably do something very similar. The reason is that he doesn't have the resources to handle all architectures. He recognizes that there are other people who are competent and have the machines needed to actually test the patches. When you have decided to let someone else maintain some aspect of a project, you must do so unconditionally no matter how simple. o.w. the person who is in charge of that aspect will begin to feel unneeded.

t.

Collaborative design vs. collaborative enhancement

[Spinality]

This topic has generated many good comments and insights. Here's one point that might not be clear enough to folks who aren't full-time developers: There's an enormous difference between development in a young system verus in a mature system. No question, a large group of casual contributors can be ideal for maintaining and improving a stable code base. However, a system that is designed by committee (in my experience) always sucks. Good software architectures are (again, IMO) always the result of a relatively small, focused group who can devote their full energies to the project. Often, beautifully-designed systems reflect the singleness of purpose of a sole author.

Therefore, requests for assistance by an early-stage project are really requests for full-time, experienced volunteers who can pick up a large piece of the project. Well-meaning but low-level contributions suck down the available cycles of the lead developers, who should be focusing on the big problems. It takes a lot of time to manage a collaborative process. So getting annoyed because the four lead designers aren't spending half their time reviewing suggestions seems unreasonable to me.

Again, this is not true of all projects, but is often the case in an early-stage effort.

The point made in the main post about many projects remaining in this early stage of development forever is totally valid. Often, the system never makes it to the level of stability where a larger community can really work together effectively. This lack of progress reflects a combination of factors: human failings, poor design, poor documentation, lack of interest, lack of financial support, etc. Nobody feels good when a project falls short.

There's no rant here. Some projects have the right combination of key people, community participants, problem area, technical success, and support resources to coalesce into a strong open-source effort. Other times, it just doesn't come together. I don't think it's fair to blame volunteer developers if they're not able to make the whole thing work, unless of course they put out a clear request for specific help and then ignore it. Some comments below call for open source projects to issue clearer articulations of their plans and needs. This is very reasonable. Most of the ambitious projects already do a fair job at this.

But blaming volunteers for not volunteering even more of their spare time never seems right to me. (Not that this was the author's intent, of course.)

JMHO -- Spiny

contributed source has to follow guidelines

[Matt Ownby]

I work on a project which has some developers with full read/write CVS access and others who have just submitted patches from time to time. A few of the patches I've received have been nothing more than quick hacks to solve an immediate problem, but which (in my mind) do not solve _the_ problem. I have chosen not to apply these patches. The people who submit these patches may be upset with me, but I think it's important to realize that not all code is good code and we as developers shouldn't be obligated to accept anything just in the spirit of cooperation. If someone is going to modify the source code, I think their changes need to have a few attributes.

1 - The change has to be in harmony with the overall goal of the project. Just because a piece of code has bugs or is incomplete in functionality, does NOT mean that the main developers don't know what they want the code to do. It often just means that the main developers are too busy to implement the feature at that specific time. If someone comes along and submits changes that go against the main developers' wishes for the direction of that particular piece of code, then that submitter ought to first a) know WHAT the developers' original goal was so that he can demonstrate that he b) is informed enough to know why his new method will work better. The developers aren't going to trust someone who just submits a new idea without first getting acquainted with the overall vision.

2 - The submitted code should be well written. If it's just a hack using borrowed code from other source with a few modifications, and lacking comments, the main developers aren't going to be too crazy about using that code. "Oh great, he just went and copied and pasted this other code, made a few hack changes to it, and now he wants to use it." Come on, let's have some pride in our work here.

Re:contributed source has to follow guidelines

[Karmageddon]

then that submitter ought to first a) know WHAT the developers' original goal was so that he can demonstrate that he b) is informed enough to know why his new method will work better. The developers aren't going to trust

Great! I completely agree with your points. Thus, I am sure that you make sure that

1. projects that you work on publish clear and concise technical documentation which outlines both where the project is going, and details how the architecture is approaches the goals. Thus potential contributors will better know where to make their changes. As, I am equally sure,
2. projects you work on, when rejecting submissions, send back constructive comments that point out where in the architecture a set of changes should have been made so the submitter can make an acceptable set of changes.

The Real Problem (tm) ...

[Jens]

... seems not to be that there are so many unmaintained projects, but that there are so many in the first place.

But I wouldn't call this a problem. Say 0.1% of all opened projects on Sourceforge (just as an example) will really become mature, widely used, big open source community projects. Aren't those 0.1% worth all the effort? Aren't those 0.1% worth putting up with thousands of dead projects because the developers didn't have time or weren't stubborn enough to carry it out? Would you rather have to certify yourself and prove your product's quality before you can post it on the 'net?

I myself started four OSS projects (one of them a community web site actually) with several people, and three of them are unmaintained ATM because I simply don't have the time to continue them. That doesn't mean they rot, though.

So, to summarize, I would LOVE to have 1000 closed, elitist, non-maintained projects floating round the Web - if that's what it takes to produce the one single genius project that changes our world. See Samba, Perl, GCC, Apache, or whatever you like ...

Collaboration

[Arandir]

Collaboration is great. Everyone uses it, even closed source development shops. But it can be overdone.

Pulic collaboration is a tricky beast. You CANNOT let just anyone commit into your code base. Someone you trust must review that code first. And if no one has time to review it, the code won't get in. This only makes sense. An Open Source software project is not a place where everyone wandering by is free to leave their mark on the fire hydrant.

The question to ask is "who gets commit privileges?" The answer is those you trust, and only some of them. Too many contributors and the project loses focus. Limit commit privileges to those who have proven their coding and domain expertise. But this doesn't mean that only a few people can participate.

I rather like the FreeBSD model. There is a small group with full commit privileges, but a much larger group called "contributors". These contributors may have commit privileges in limited areas of the code tree. Anyone else wishing to contribute may do so, at any time, but not directly to the tree. Instead the submit their code and patches to the appropriate contributor. Eventually a submitter will gain trust and be admitted to the ranks of privileged contributors, and maybe eventually to the core committers.

Re:Not the point of free software.

[Chris Johnson]

I wish I could mod this up. :) all I can say is 'yeah!' despite it being a bit too personal. Not everything is about producing big powerful software projects to take over the world. The simple ability to see the code and learn from it and (typically) incorporate ideas from it into your own stuff, or adapt it to a specialized purpose, has huge value.