Open Source Project Management Lessons

cpfeifer writes "Paul Baranowski takes a moment to reflect on Open Source Project Management in his blog. His reflections are based on the first two years of the Peek-a-booty project." Interesting comments on media coverage, choice of programming language, when to release a project, and more.

Comments

Great, I would love to read all about it

[mao che minh]

I would love to read the article, but my employer uses a proxy filter program that filters out sites with the term "proxy avoidance" in it's META tags (or otherwise prominently displayed within the header). If you are going to run a site dedicated to the development of software that allows one to avoid detection systems and firewalls, then how about making the distribution source (the website) not so obvious and vocal about it's intentions.

Just a suggestion.

I bet...

[macshune]

The peek-a-booty project is a lot less interesting than I would imagine...

Project management Lessons

[stanmann]

Is Open source project management really that much different from any other project management?

don't release before it does something useful

This is a rule in "traditional" project management too.

and the other lessons read just like Project Management 101 too. I would have loved to have seen something insightful or interesting about how open source changes the development environment or the management environment from single location to distributed, but no such luck.

Re:Project management Lessons

[mmcshane]

. . . and the other lessons read just like Project Management 101 too.

Good point. Not surprising, I suppose.

If you're interested in Open Source project management you might find some of these Mozilla lessons learned interesting.

fascinating

[Fux the Pengiun]

This is a great entry, with some really good insights.

I first heard about this project from a BBC Article where they describe it as a "browser free from censorship or outmoded intellectual property laws," which is something I think we can all get behind! However, they made the point that the project could have been better named.

It seems the author has picked up on that now, too. I think most telling is this passage from the author's blog (weblog):

Be careful when naming your project. It's difficult for IT managers to convice PHB's that this project is useful for their enterprise class systems with a name like "peakabooty." This sort of nomenclature is detrimental to the future of GPL/Linux

Hopefully, he'll take these insights to heart!

I think not

[The Bungi]

While saying that C/C++ are not "viable" languages any more (and making the classic mistake of bunching them into a single blob), he muses that:

It requires compilation - as your code grows larger, the wait time to see if your code works increases. This delay directly affects how fast your code is developed.
It's really, really, really hard for people to learn it, and this directly impacts the number of developers you will have on an open-source project.
It uses static binding (Isn't that supposed to be a good thing?)
There are no standard libraries for C++, so there's a lot of reinventing the wheel. (Yeah, there's the STL and others, but each one has a huge learning curve associated with it).

So, basically, it has to be compiled (duh). It's hard to learn (no, it's hard to use correctly) and it has no libraries... eh?

I'm sorry, but this guy is not a software developer. The usual comments about "X is the One True Language" notwithstanding, I can't follow that because he thinks it's "too hard" and he thinks it's "not viable" and decides that it simply isn't a good fit for his project, then LanguageX must be dead. Perhaps he'd like to share with us which language his OS is written in. Maybe it's Forth or Scheme. Use the right language/runtime/lib/technology for the job and refrain from saying "X sucks because I don't like it".

Other than the dubious "this is how you do open source" slant I can't see how this article is even worthy of news.

Re:I think not

[GlassHeart]

A few other points in defense of older languages:

• An older language has had more lines of code written for it, so its weaknesses are better known, and more likely published.
• An older language tends to have more developers, which means any random volunteer is more likely to know it already.
• An older language has already fought "battles" for survival, and has been squeezed out of applications for which it is ill-suited, and continues to exist for a good Darwinian reason.
• An older language is more likely to be standardized, and more widely ported.

Let's take C, for example. An experienced C programmer will point you towards two or three books like van der Linden's "Expert C Programming: Deep C Secrets", or Koenig's "C Traps and Pitfalls". C is rarely used for GUI application development, but still widely used in embedded systems where space and speed are important. C is an ISO Standard, which is important for portability.

One mistake that many people make is to dismiss older languages when a new one appears with all the old features and then some. The old language does not become any less viable the day a new one comes out than the day before. That is, if a project will take you 6 months in C, it'll still take 6 months even after Java (which might cut it down to 5) comes out. The question is whether the unique costs of using Java instead justifies the 1 month saved, not whether C is still "viable".

Interface is Everything

[Snot Locker]

Often time this principle applies to people on the project, not just the software being developed. I've learned from experience that really sharp people with a broken user interface can destroy a project!! You have to try to minimize interaction points with folks like that and find where they can excel and use their talents without creating problems for the others on the team. A difficult nut to crack at times and a far more critical factor to project success that the programming language, source management tools, etc...

Compilation time bounds productivity?

[BreadMan]

It requires compilation - as your code grows larger, the wait time to see if your code works increases.

Everyone knows that once your code compiles, it will work!

Repeating the same old misinformation

[h_jurvanen]

There are no standard libraries for C++, so there?s a lot of reinventing the wheel. (Yeah, there?s the STL and others, but each one has a huge learning curve associated with it).

This is a huge error that casts doubt on the author's credibility. What is commonly known as the STL is the C++ standard library, and it has been since C++ became an ISO standard in 1998. Doubters may consult books like the clearly-named "The C++ Standard Library" (Josuttis, 1999) to get themselves up-to-date.

Maybe that's just another drawback of C++... a lot of people don't know what the hell they are talking about and thus repeat misinformation?

Re:Repeating the same old misinformation

[Rinikusu]

Actually, he just says that C++ is no longer a viable programming language. A quick look at the sourceforge project page reveals that peek-a-booty is indeed written in C++.

An excerpt

[Call Me Black Cloud]

June 14, 2003

Dear diary,

I have decided to record my thoughts on managing the project "peek-a-booty". The most important lesson I've learned is not to use booty in a project name!

Sure, it was funny two years ago after a few beers, but I swear the next person that makes a "booty" joke will die. I'm serious. "Dude, is it for peeking up skirts?" "Hey, if you integrate telephony you can call it 'peek-a-booty-call'"

In other news, I'm starting a new project to manipulate network traffic, this time using Java. I'm thinking of calling it 'jAck Off'. I like the sound of that. It will be good to get that whole 'booty' thing behind me...

Re:"C/C++ is no longer a viable development langua

[Arandir]

C and C++ are most certainly viable development languages. Let's see now: Linux, BSD, GNOME, KDE, Apache, Mozilla. Even Perl, Python and Ruby are written in C or C++. But maybe the author is saying those projects aren't viable...

Use the right language for the job. If all you're doing is interfacing to a database, then a scripting language may be the most appropriate. But if you're writing system software, then by all means stick with C and C++ with some shell glue.

Compiled languages are damned convenient to the user. "Here's an executable, just run it", versus, "here's a script, go download compile and install the interpreter first, making sure it's the correct version, set up your environment variables correctly, then run the script."

Re:Article text

[quasi_steller]

There are no standard libraries for C++, so there's a lot of reinventing the wheel. (Yeah, there's the STL and others, but each one has a huge learning curve associated with it).

Well, there is the STL and the Standard C++ library. These are not really hard to learn; it is just that they are both fairly new. The Standard C++ library and STL didn't become standard until around 1998 when the ISO C++ standard was formed. Because that is only 5 years ago, and C++ is much older than 5 years, there are still a lot of books and courses floating around that don't teach Standard C++. When you learn "Visual C++," or whatever, from an older book (or bad book) then it seems that the STL is hard because you are not use to templates and all that stuff (mostly because many older compilers are broken, ie VC++ 6.0, and don't support the standard).

If you learned STL and the Standard library when you first learned C++ then maybe they wouldn't seem so difficult. Also, many people who first learn programming in a language such as Pascal, like me! :), have difficulties learning generic programming because they are so used to thinking about what types your variables are. Templates and generic programming are very powerful, if you are willing to spend the time learning how to use them.

Duh

[apankrat]

C/C++ is no longer a viable development language

Sure, in the scope of this particular project and in the context of their skillset and development practices.

Don't Use Binary Protocols for Application Development

Bah, I'm speachless. Yeah, right. Better yet convert data to PNG images and pass those along - it will allow you to debug networking layer with a web browser ... Ever heard of protocol layering or data marshalling ?

With all due respect, it looks like Mr.Baranowski either learnt wrong lessons or likes to summarize things beyond reasonable limits.

What's this guy doing?

[JohnwheeleR]

1. He shouldn't speak so generally by saying things like "C/C++ is no longer a viable development language."
2. He shouldn't speak so authoritatively having only one OS project under his belt.

It sounds like he busted his ass on a project no one appreciates because it is generally useless.

Re:"C/C++ is no longer a viable development langua

[Cereal Box]

There is, indeed, a standard library for C++, one which is widely supported - STL.

I don't think that's what he meant by a "standard library". He's thinking along the lines of Java's standard library -- a standard library that gives you graphics classes, networking classes, XML parser classes, GUI classes, etc. You know, the kind of stuff that would be convenient to have bundled with a language. STL is a standard library full of basic data types like linked lists and hash tables. Big whoop.

Re: Documentation!

[alexjc]

So, after two years he still hasn't realized the importance of documentation?

"Document it and they will come."

A good project is nothing without it...

convienient

[pyrrho]

no, I don't think bundling that is particularly convienient... indeed, I think that it's more convienient to have a choice and not have things like that tied tightly to the language.

Right now there are well established libraries in C++ for anything you get from standard libraries that are tightly integrated... just with multiple competing established libraries. A wealth of choices.

There is no standard GUI library that ships with all compilers. That will come, but not before it's time. Java gives such a library... too bad it sucks.

Re:"C/C++ is no longer a viable development langua

[hackrobat]

I work at Oracle. C++ is banned here. Apparently, Oracle software runs on more platforms than there are C++ compilers for. Therefore, C++ is a strict no-no. The internal C coding standards doc reads somewhat like this:

1. Don't write C++.
2. If you've already written C++, rewrite in C.

:-)

So it's either C, or Java (lately). Anything else is considered as scripting (Perl, Shell, SQL).

Repeat after me: The Interface is Everything

[2TecTom]

As an interface designer and technical writer, this has always been my personal mantra. It's finally nice to see that at least one engineer finally actually gets it!

You probably won't believe how many MMI designers and technical writers are feeling totally vindicated at this point.

Really, it's not often one sees history in the making. ;~)

95-5 Rule

[jmacgill]

Usually it's the 80-20 rule, but in open source projects it's more like the 95-5 rule. Open source projects are usually run by one or two people doing most of the work. If you decide to lead an open source project, you must be willing and ready to accept this.

Actualy, if you are about to set out on a new project, its probably best to tell yourself that you are NOT willing and ready to accept this.

6 years ago I started a project called GeoTools and it was, for the main part, excactly that - two people doing most of the work. This was fine for a few years but over time the user/developer ratio got out of hand.

Eventualy it became all but impossible for the two lead developers to support 300+ users and although other developers wanted to contribute it became dificult to 'train' new developers as the knowledge of how things worked existed mainly in the heads of only two individuals who had done 95% of the work.

Two years ago we took the descision to re-design the toolkit from the ground up with as much input from as many people as possible. Since that time we have strived to make sure that as many people as possible have an input into the design process and we keep that process as open as possible by pubishing the IRC sessions in which discussions take place.

The project now has 9 very active developers who are members of a Project Management Committe and a number of other active contributers as well. The end result is that quiries to mailing lists get responded to far more quickly.

Getting other people to work on your project is often - TO START WITH - more effort than just doing the work yourself, but the pay off is HUGE, as you then have someone else who can explain things to others.

If you ever have a contributor who gets stuck or confused and you find yourself thinking 'oh, it will be quicker/easier for me to do this part myself' STOP. Spend the time, help them work out how to do the modification even if it takes a few hours when you could have done it yourself in minutes becuase after you have invested the time in them, they will be able to add things in minutes too, and they can teach others as well.

If you work on a tight, well defined, non-evolving project then most of my ramblings are probably irelelevent if not they they may be of use. The only danger is in investing time in helping developers who then wander off - it happens, but I tend to find that the more you invest in them, the less likely they are to loose intrest.

Re:95-5 Rule

[JamieF]

Or, you could write some documentation. It's not that hard but for some reason developers avoid it like the plague.

I keep reading these open source project managers bitching about how hard it is to answer all the emails from users and potential developers. Then you look at their project and see this:
FAQ: (under construction)
Documentation: (under construction)
and if you're really lucky they bothered to archive the developer mailing list, which is just about the least efficient way to document the project. (Yes, let's make every developer read all of our conversations ever, including all of the arguments we had and all of the things we decided to do differently later.)

It's not that hard; I've done it. You take a day or two and STOP CODING (oh no!) and write some actual words, document APIs, and lo and behold, you realize that you're also white-box testing your code because you were forced to explain what some code does ("hmm, that actually doesn't make any sense" / "oh crap that won't work in this case").

All of a sudden you can just point people at the docs and add stuff when new questions arise. Sometimes the answers are best expressed as a FAQ, sometimes as part of API docs ("oops, I didn't explain what these constants actually meant"), sometimes as end-user documetnation. But IT'S WORTH IT.

As for the common self-delusion, "we'll write the documentation after we're done with the code", all I can say is, read any process book there is. You're doing it backwards if you start with implementation and then back into writing down what the system does and how it's designed. Those things come first, THEN you write the code that does it. Otherwise you redesign the product dozens of times as you go, each time slapping yourself on the head and saying "oh crap I forgot it has to do this too" or "argh, I need this value from here but the API doesn't allow that". If you write it all down (which first requires that you *think* about it in detail) then you won't forget your good ideas, you can jump all over the place, you can write test harnesses that make sure the code does what the docs say, and you can point newbies at the docs and say "OK cool you can get started on the email notification part, we haven't touched that yet."

Re:"C/C++ is no longer a viable development langua

[dmeranda]

Okay , you
dont't
have to
like using whitespace
so
others can actually
read your code, but
I like the
way
Python lets me
do the
right
thing.

Re:"C/C++ is no longer a viable development langua

[nuntius]

You over simplified. As he said, static bindings are both a blessing and a curse.

For example: Oops, sorry. Mozilla on Linux is now being compiled with gcc3.2 so you'll have to get the source and recompile to run on that older (gcc2) system... Also, your old plugins will need to be recompiled before you can use them with Mozilla on the new system - if you can find the source.

Compare and contrast:
Java - install the compatibility VM; use the same binary on all platforms
C - no VM; compile and distribute different binaries for each platform

As the number of platforms increases [3+ Windows code bases (9x, NT, newer), 3+ Mac bases (system ...X, with or without Altivec), N *nix bases (Sun, HP, IBM, Linux*M, BSD*3)*(2+ GCC versions) = well over 10 popular platforms], you either have to manage binary chaos or you have to start distributing your code as source.

But wait! Distributing code as source requires the end user to install the compiler, and this (setting up environment variables, binary compatibility, support libraries, ...) is usually harder than installing Java.

So, we're back to square 1.
Lose a turn; don't pass "Go".

Not Project Management

[drmofe]

None of the lessons learned and reported here are directly related to Project Management per se. They are all by and large implementation issues.

There is also nothing new here. This does not advance the state of the art. History does not advance by people relearning the same lessons again and again. Just because they have been reported here does not make this article special in any way. This article could have been written in any of the decades of 70s, 80s, 90s (substituting en vogue languages for C++/Java) and still make sense.

In order to truly advance the state of the art, we have to think in far more advanced ways about project management and software development. True Software Practice and Experience requires much more planning and critical thinking than evident here.

If Open Source is to provide a useful and stable platform on which to build, then we certainly need a better vision of how to build software. Otherwise, we will be doomed to repeat history by implementing old things in different ways and not really gaining any control over complexity.

In summary, we still have a software crisis; Open Source will not change that; and summaries of software development experience that just say "I made the same mistakes as other people did" are not very helpful.

Re: Documentation!

[JamesOfTheDesert]

So, after two years he still hasn't realized the importance of documentation?

"Document it and they will come."

A good project is nothing without it...

Good point; somebody should document this.

NOT just open source

[Kefaa]

His point that this is not unique to OSS Projects is a good one. While OSS development has unique constraints most are around people and personality. In an office we all have to get along or get fired, in OSS it can sometimes be worse.

For example:
The press loves infighting because it's a good story. However, the infighting story is bad for a project that is trying to get funding. This creates an air of instability. People only like to fund things they feel will have a high chance of success, and instability erodes that confidence.

It is too bad on so many mailing lists ego/attitude/personality or just plain rudeness show up. Things you would never say to a coworker, make it onto a mailing list for eternity, or at least what looks like one. I hope people take this point to heart before posting.


95-5 Rule
Usually it's the 80-20 rule, but in open source projects it's more like the 95-5 rule. Open source projects are usually run by one or two people doing most of the work. If you decide to lead an open source project, you must be willing and ready to accept this.

Looking at sourceforge I see this lesson again and again. The idea that if I create it they will come, and build. Forgotten, or unknown, is that nearly all had a real need to be built first. I needed application ZAFDE so I built it. I then released it and people thought they could build on it, and so on.

I wanted to learn C++ or JAVA or XYZ is the reason we have 2,134,931 notepad applications, not OpenOffice.

C/C++ is no longer a viable development language
I knew we would see a flamewar as soon as I read it. My thoughts:
- Both are still viable. Much like his hammer analogy, they are not good for everything.
- What makes them "bad" for development, makes them "good" after they are developed.
Does it matter to the user that it took 81 minutes to compile? Nope, they have the binaries or compile it once and run it for years.
Every language has a shortage of people who know it. Or specifically a shortage of the people who know it and are willing to work on OSS project PDQ.
Static binding is good/bad/sometimes both. Yes it is.

All the negatives he spoke of are positives after it is developed. Which we hope is long compared to the time spent developing it.

If there is one thing projects should take away, it is probably this:
Interface is Everything ...The program should be fun to work with. There should be buttons and things that blink. The interface should be the first thing you do. The interface serves as inspiration and motivation and helps you to learn how the final product should look. Yes, it's going to change a lot. Yes, it's going to have to be rewritten multiple times. Yes, it will never be good enough...But when someone downloads your program they will have something to do. No one likes to look at command lines.

I like command lines. I use them, but I understand they are power tools. Most people do not like/use them and consider them an indicator of a poor product. Even while it may not be technically true, perception is reality in this.

Windows and WINE only...

[Ashtead]

I didn't have any problems with reading the site at the time of this posting, but I can see where his headaches with C++ might have come from. The MSVC versions of STL hasn't been around in a standard form for very long, although the MFC library has been. This is most likely the grounds for the complaint. And the MFC does not make for easy portability to native Linux/Unix environments.

However, designing things in C++ and doing it properly is damn tough; many designs may seem easy to begin with, but then run into trouble with things like multiple inheritance from related parents, or simply that encapsulation is difficult because of the need for exposing the inner workings of classes... STL fixes some of this at the expense of code bloat -- it is easy to produce executables tens of megabytes in size.

Another problem with C++ which has been bothering me, and I would presume, the developers of Peekabooty, is the tendency towards static compilation and inclusion of everything. I looked at the source files here, and the sheer number of include-files compared to source files indicate that this probably does not compile quickly.

There is a way around this, if the application can be divided into several major and fairly independent components which then are compiled and linked as a number of dynamical libraries (.DLLs on Windows, .so on Linux and Unix). Now, with proper design, recompiling the whole lot is not necessary for smaller changes within one of the parts where no changes in this part has taken place. The trick here is encapsulation: do not let code in any one part know about any of the internal structure of code in any other part.

Re:Article text

[Jellybob]

He said it's no longer viable for application level programming... Linux and XFree are certainly not application level, and argubly the Mozilla core isn't either (the front end is, but you could wrap the Gecko API for some other language to create that).

Re:"C/C++ is no longer a viable development langua

[Arandir]

Java - install the compatibility VM; use the same binary on all platforms

I'm laughing so hard Dr. Pepper is spewing out my nose! Let me wipe off my screen...

Trying to get Java applications to work on my Solaris workstation is a nightmare. I can't understand it because the company that makes Java makes Solaris. I try to figure out what the problem is, and hidden way deep in the README is this thing that says I need to install a different version than what Sun provided with Solaris. I fix that and it still doesn't work. Looking deeper into the problem, I see that I need a couple of other components that didn't come with the application. After about two hours of searching for the "myleetclasses.jar" and "ubercoolstuff.jar" files, I put them in a directory, fiddle with the CLASSPATH variable, and finally get the program up and running. And then it crashes five seconds later.

you either have to manage binary chaos or you have to start distributing your code as source.

Binary chaos is a pain. A royal pain. So I distribute my code as source instead. Easy. Painless. Users that don't want to compile can grab a prebuilt package from their distro or another repository.

Of course, I'm writing Open Source, and not cheesy shareware. Perhaps that's the true niche for Java and .NET...

Re:"C/C++ is no longer a viable development langua

[hackrobat]

I'll clarify:

• Existing C++ code in Oracle products has already raised serious portability issues.
• Oracle RDBMS is one of the first products to be ported to a new platform, often before the official release of the new platform. At the time of porting, a C compiler will be available, but a C++ compiler may not be.
• C compilers for 64-bit platforms are far ahead of their C++ counterparts.
• C++ compilers on some platforms are immature. It's far easier to write incompatible C++ code (than C).

<disclaimer> My views; not those of Oracle. </disclaimer>

The Mozilla C++ Portability Guide also restricts use of some key C++ features (rtti, exceptions, templates (which rules out the STL by the way!)).

Communication!!!!!!

[hughk]

Congratulations on a successful project. However, you mentioned what worked without commenting on it:

Two years ago we took the descision to re-design the toolkit from the ground up with as much input from as many people as possible. Since that time we have strived to make sure that as many people as possible have an input into the design process and we keep that process as open as possible by pubishing the IRC sessions in which discussions take place.

The moment you involve other people in a project, you need to document and explain design decisions and any discussions arising. By doing this you have made it very easy for other developers to get involved. I also like the way that you publish IRC discussions.

You are almost a text book example on how to do things right.