Gathering Requirements In Open Source Projects

webword writes: "There is an article in the July 2000 issue of Crosstalk (The Journal of Defense Software Engineering) about gathering requirements in Open Source projects. This is especially interesting because most commercial projects are driven by the requirements gathering processes (or the 'SDLC') whereas most Open Source projects are written to 'scratch an itch'. Let's face it, Open Source requirements are almost always in the mind(s) of the programmer(s), not on paper. However, if the requirements are captured, and shared, the quality and speed of development will improve." How many articles like this have we seen? How many open source applications have ideas like these spawned? I remain skeptical.

Comments

Software Engineers vs. Reality

[jamused]

1. Gather Requirements
2. Make a Design Spec
3. Model the project (i.e. UML)
4. Code
5. Test
6. Goto 3

That's the way software engineers wish it worked. What actually happens is this:

1. Gather Requirements
2. Make a Design Spec
3. Discover ambiguities in Spec, goto 1
4. Model the project (i.e. UML)
5. argue incessantly about the one true way to model the project, goto 4
6. Code
7. discover what the model requires is either unecessary, impossible, or both, goto 1
8. Test
9. discover requirements have changed, because either conditions have changed or what you were asking the users to do (come up with a complete definition of how the software is supposed to work without any software in front of them to poke at), goto 1
10. Attempt to Goto 3, but discover that the code has drifted from the model and nobody kept the model up-to-date

This is why I'm no longer very interested in CMM Level 3-style Software Engineering. Some of the disfunctional behavior in the second list is avoidable, but some of it is inevitable as death and taxes. Venting at the stupid users for changing their minds once they see the software working (or sort of working), or at the stupid managers for letting them, doesn't actually accomplish anything. Even if you "win" the political battle of getting the users to accept the software as written to the spec, it's a Pyrrhic victory.

I've come to see that it's far better to grant the users the right to change their minds as their needs (or just their understanding of those needs) change in return for the right to use processes like Extreme Programming that will help me as a developer to meet those changing needs without running myself ragged, than dumping a year or more of my life down a rathole producing software that only makes people unhappy (if it gets used at all) because it was coded exactly to a completely outdated snapshot of what the users thought they needed before they and the world moved on.

Software development ought to be an ongoing conversation between the developers and the customers, not a one-way order from the customers to the engineers to produce a widget machined to the following specifications for delivery on such-and-such a date. Software development isn't there yet as an engineering discipline, and it's not even clear to me that's the direction that we ought to be going, since it gives up one of the most valuable and interesting properties of software (that it's "soft" and flexible).

The "when" of the requirements

[Vokabular]

As a researcher in this field, I'm happy to see recognition, if not necessarily interest, in requirements in open source projects. As so many have pointed out, these projects come about as a result of someone's "itch", and as such there is no real customer to drive requirements early on.

We're left with a situation where we want to retrofit requirements to the project, which is completely backwards from what you will normally find in commercial, software-for-money type projects. I think this is a wonderful idea; different design methods call for different requirements methods.

The question then becomes: Why should I, an open source developer, bother stating requirements, especially after-the-fact? I think what we're starting to see, at least in my narrow view of the world, are strong correlations between good requirements definitions and the ability to safely modify, repair, explain, and reuse complex software. That is, by properly stating what your software "does", you can help ensure it's longevity and usefulness to people using it without access to the initial developers.

Perhaps a re-thinking of the meaning of "requirement" is needed for open-source projects. Whereas a requirement has traditionally been a pre-development statement of what the software will do, open-source requirements are perhaps more likely to be in- and post-development statements of the same thing...and perhaps they will be more accurate for being such ;)

Open Source vs. Commercial

[platypus]

The whole premise is faulty....
OSS and commercial software are orthogonal, so it's not possible to construct contradistinction between the two (...whereas open source ...).
For a good example go to http://dev.zope.org.
Zope _is_ open source _and_ commercial, and they put a lot of thought into how they want the development process to progress. Heck, read about the "fishbowl" in the above link, there are also thoughts about the progression of the planning of development.

Not to say this all works perfect (which is also always the case for closed-source projects), but open source doesn't mean that the software grows like cancer in any direction possible.

Re: Commercial projects have requirements?

[InitZero]

I have YET to be involved in a project that actually starts with the requirements gathering.

I just finished a four-year project to replace my newspaper's character-based publishing system with a WYSIWYG full-page layout solution. We spent eight months working on a Project Specification Document (PSD) before we laid down a single line of code.

The PSD was two volumes and over 700 pages. From the moment it was written, it was misleading, incomplete and, sometimes, downright inaccurate. Even with its problems, it was one of the best tools we had and I can't imagine doing a project of any size without a blueprint. That document kept us on track and often remided us of stuff we otherwise would have forgotten.

Yes, I did say of any size. For small projects, a line or two at the top of a script is enough. As the scope increases, so should the blueprint.

I wouldn't want a contractor to build my house without a plan but some folks do exactly that when it comes to their software. Specifications? We don't need to specifications.

Before I started my current job, my planning was half-hearted if done at all. I mocked people who would have meetings before starting a project. I felt that the time saved by not planning and by not holding meetings would more than cover any extra time needed to fix small screw-ups along the way.

The first few months I spent here were painful. meetings every day. Documentation all over the place. Very little 'work' actually getting done. Three years later, I have a lot more respect for doing things the right way.

The Open Source movement could greatly benefit from more Open Planning. Right now, for all our talk of openess, the organization is closed if not the source. Could you imagine how many more volunteers could get their hands dirty if there was a list of things that needed to be done and a roadmap as to how to get there?

InitZero

You Only Need An SRS If You Aren't The Customer

[Carnage4Life]

The problem with requirements elicitation is that it usually implies that the project isn't a spontaneously scratched itch, but instead a commercial or near-commercial product. Eliciting requirements usually means you have a customer and that the development team or developer-customer liason has exhaustively questioned all the potential users.

The purpose of a Software Requirements Specification is to enable the developers understand the customer's problem domain and learn what the needs of the customer are. Since most Open Source projects are written by the customer(s), it is usually redundant for them to create an SRS that describes the needs of the user.

Although an SRS is not truly necessary when creating a product where the developers are the customer it can still be beneficial for a variety of reasons including

1. Being a reference point for the entire team when implementing features and wondering what behavior should be exhibited on certain events.

1. It is a good way to handle feature creep since it is a formalized list of features. Only things in the SRS are implemented and if a new feature comes along that everyone wants, it should be retroactively added to the SRS.

1. It gives developers an idea of the size and scope of their project.

1. Priority levels for different features can be set by using the
2. must-will-should rule. Any feature description preceded with a must is absolutely necessary, any feature preceded with a will is important but not absolutely necessary while features preceded with should are typically wishlist features.

Second Law of Blissful Ignorance