Slashdot Mirror
Improving Open Source Using Software Process Concepts?
icanoop asks: "I'm working on a project to help improve open source development using mature software process concepts. What process issues do open source developers think are most important and/or can be improved? If you are interested in seeing what is being considered read the problem statement at the project site. It's not final so feel free to suggest changes."
It chases off professionals interested in real projects. 'Oh I don't want to get involved with that, there are 30 projects like it on Sourceforge.....'
Maybe my gripe it with how the opensource projects are handled.
Vaporware that sits for 2 years is not a project.
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
I can give you a list of things to avoid:
1) Allowing the developers to dictate the initial design rules. Allow a focus group determine what it is that is required, then let the developers determine how feasible it will be to implement.
2) Fear of COTS product integration. That is, use the right tool for the job. Of course, if everyone's a whiz with CVS and Emacs, then the more power to them. But don't let anyone make a project a "perfect fit" for their tool of choice which no one else is willing to use. That will cause problems later.
3) Not using outside code / help. Often times, portions of what you want to do have already been beaten to death. Look hard.
Of course, you know all of this. It seems your problem statement and proposed solutions on the linked site are quite thorough; I don't see anything that looks like a sticking point.
Maybe you want to restate the question.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
Save man-years by not saying things like "mature software process concepts" when you mean things like "good plan."
Your mouth is like Columbus Day.
Focus Group? If I'm writing code for FUN in my OWN TIME then I think that I should be able to determine what I write, not some focus group. I don't tell others how to spend their free time, why should they be able to tell me. If the focus group want feature X then they can code it themselves....
- Lack of a plan.
- Lack of peer reviewing.
- Lack of predictability (both feature and time wise).
There are many points here, but one of the most important is the lack of a plan. It would greatly benefit most OSS projects if there was a plan of features to be implemented. This would not only tell users and project members where the project is heading, but also prevents eyecandy and other code bloating problems to enter the project too early. It would be good if a feature had to be on the TO-DO list to be included into the project source tree. This way each feature has to be discussed, specified and granted before being implemented. This helps building more consistent software.The second problem, peer reviewing, could be solved by including it in the code versioning system (hense the subject of this reply...). All code must be tested and reviewed by an independen peer before included in the source tree. By introducing automatic testing, such as a small test bench application showing that the submission works, modularity is encouraged. By introducing good modularity, new patches are more easily tested and included in the source tree.
The last point is mainly a project management issue. Someone has to say that these features will be available at this date in this release. This problem is simply the addition of time to the first problem (a plan). This is the thoughest challenge when working with spare-time programmers. Not many will be happy about commiting to a project, then being forced to keep a time plan. Anyway, this can be enforced in the big, with partially paid work-time, projects.
I've made this suggestion several times before; what we really need in OpenSource development is a reliable and powerful code auditing controller.
While direction and design are problems experienced by a lot of OpenSource software, it is quality and security issues which are of more concern to mature projects, which have the highest visibility and widest use.
A code audit system would allow a project to be viewed as a graph of procedures/methods, and force every procedure to be marked as audited by a number (variable threshold) of auditers of a predetermined "skill level". i.e. the designers and core audit team together decide that certain procedures and modules are sensitive, and require an audit by three senior/trusted auditors; other less sensitive code requires only to junior auditors.
The system could track the auditing in conjunction with source code control, and use the software call graph to invalidate audits on procedures when dependancies are modified (and automatically marked as unaudited).
e.g. Procedure A calls procedures B and C. They are all marked as completely audited. Any change to Procedure C will necessarily invalidate the audit on C, and by relationship invalidate the audit on A. Once C is re-audited, the audit of A will still have to be performed again (unless, say, a special "interface not changed" flag is used on the re-audit of C).
In this way everyone can be happy that a project release is secure and reliable, based on the extent of the audited code.
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
Of course some of those "process" things can be valuable, and when a project meets a problem, they can use a solution. For some projects it even makes sense to talk of focus groups (KDE comes to mind), and regression testing is used in some (GnuGo).
I think it is fine to make some of these techniques available to those who see the need of them. Good introduction material about such would be welcome. But do accept that most OS projects are better off without a pointy-haired boss and his bureaucracy.
In Murphy We Turst
The fundamental question seems to be :
Do processes make better software
I've been involved with a lot of software projects (though never contributed much to Open Source...), and I have never seen a single project that was succesful because it followed a process. Nevertheless, whenever a project runs into trouble, the first call is usually for "more/better process !!". So let's look at this in more detail.
Succesful projects seem to grow their own process. The process seems to be simple, and often appears to be way less than you would expect, and rely heavily on interpersonal communication rather than documents and frameworks. There's usual a small core of "gatekeepers" who set the technical and philosophical tone for the project. The Linux kernel is a good example.
I am very worried about people using phrases like "mature process", "industry standard" etc. - in my experience, this often refers to the Rational Unified Process or the Software Engineering Institute's Capability Maturity Model. Both are laudable and when I go on holiday, I really want the airplane's control systems to be written using such processes. However, for many projects, the burden of bureaucracy is inappropriate (yes, I know you can tailor the RUP to suit your needs, but it contains over 140 different deliverables, none of which appears to be code). The training required to bring developers up to speed with these processes is significant, and usually expensive.
Instead, I'd look at the Agile methodologies at Agile Alliance website. The "Crystal" methodologies are especially interesting because they encourage you to actively choose the processes your project needs based on a variety of parameters - size, risk etc.
Having said that, I think a lot of the problems addressed are real - I think they get solved by people, not processes though.
It's all very well in practice, but it will never work in theory.
I believe the saying goes "with enough eyes, all bugs become shallow", not "with an arbitrary number of eyes ..."
If an auditing process like this is used, people might take a function for granted as "working" just because it's been checked by three different people, two of which are "experts". Even experts make mistakes.
A far more reliable solution is to have unit tests (like with JUnit or xUnit). An expert at writing good unit tests is far more useful to a team than someone that just scans code.
The unit tests also become important regression tests, so bugs introduced indirectly are found immediately. For more info, googlize yourself on "test driven design".
----- rL
To have a more coordinated setup. I have lots of misgivings about just selling alpha code in the webstore and claiming some victory.
It chases off professional software companies interested in real projects. 'Oh I don't want to get involved with that, there are 30 products like it at the computer store"
Maybe my gripe it with how the closed source projects are handled.
Vaporware that sits for 2 years is not a project.
> I believe the saying goes "with enough eyes, all ..."
> bugs become shallow", not "with an arbitrary number
> of eyes
But most projects have an "arbitrary" number of eyes, not "enough" eyes. Relying on thousands of individuals to review every projects code is very inefficient, and not realistic. It may work for the Linux kernel, but what about the rest of open source?
> A far more reliable solution is to have unit
> tests (like with JUnit [sourceforge.net] or
> xUnit).
The plan is to reuse projects such as JUnit and other testing facilities rather than write them from scratch.
Although no final design decisions have been made at this point.
> An expert at writing good unit tests is far more
> useful to a team than someone that just scans
> code.
This is very true. That's why the proposed solutions include testing as well as reviewing. But there are some defects that can be found in a reviewing process that can be missed by regression testing. Testing only finds the defects that are tested for. Reviewing can find others, as well as improve code.
One this that appears to be missing from the other posts is management systems,
OSS can potentially attact a huge number of devlopers, most of which can only spend a few hours a week on OSS, this is amanagement nightmare.
I'd like to tie up all the code/bugs/designdocs/developers into a nice heiricical linked structure kept in a configuration management system. That way I can look through the open bugs
find out what modules it's thought to relate to
find out who's responsible for the various parts and look an creating a patch.
thank God the internet isn't a human right.
I'm going to be a slight devil's advocate here, because I believe strongly in unit testing, but there is a significant amount of academic literature which demonstrates the falacies inherent in using tests to catch bugs.
Incidently, test driven design is concerned with demonstrating that the software can function as specified. It is not concerned with - and bad at picking up - software that performs those functions with side effects, or contains code which can be exploited in a security-sensitive context.
The simplest "proof" of the problem with testing is to consider a function with four arguments; let's say this is C, and two arguments are int while the others are char*. To rigerously exercise a single argument of this function, you need five to seven distinct tests: obviously good value, obviously bad value, inner- and outer-values for boundary conditions (may only apply to the int), and NULL. Unfortunately a black box function may have strange interactions between parameters which testing each individually cannot expose; a proper test set to ensure the function behaves correctly in all circumstances will require approximately 6 * 6 * 5 * 7 = 756 tests, being the permutation of all single-argument test cases.
No, I'm afraid this is not a suitable answer. Unit tests are best used for two purposes: testing that a function produces the correct and expected effect when used correctly; and testing that a function does not produce a bug that has already been discovered (i.e. regression testing).
For everything else, there's Mast^H^H^H^Hcode inspections. A code inspection (audit) will detect problems in the handling of parameters far more efficiently than building hundreds of tests. What is more, it will (when performed by an experienced developer) expose problems that tests cannot: tests are particularly bad at showing up race conditions, buffer overflows and other security risks.
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
I am working for Philips CE as a software engineer, and we have reached in our department CCM Level 3. As I have been here some time, I had some training in CMM and I had to participate in the Level 3 Audit.
I read this post already yesterday evening but I had to think a night about it.
What these software processes are about is formalising the right steps needed to obtain a certain product (plans and planning, development templates), the necessary parts of the process (like setting up a process database, configuration management, bug tracking, etc...) and probably the most important part, enhance and promote good ways of working in the development process and trace and remove bad ways of working.
I think that for an organisation like ours, this is very positive. We have scheduled releases of all kinds of software and it sure helps to deliver working software on the right time.
To implement this however, you need a large organisation and people who are really interested in writing processes.
Now for FLOSS side of things. I suppose that KISS helps here, because not many people working on a FLOSS project will be interested in process development. This means that the people which start the project will somehow need to define their own (simple) process(es) and formalise them.
What do you really need?
I had a look at eXtreme Programming, and I must say that if it is compared with CMM, then it probably fits most goals of CCM from a practical point of view, not from a formal point of view (CMM requires you to keep all your processes written down somehow, for inspection by the auditors).
However, XP (like probably most of these methodologies) is also geared toward fast and correct release of deliverables.
For FLOSS projects, it thus seems more appropriate to concentrate on things which need to be in place, than on procedures to produce deliverables on time. If you want to have this, then mention this at the start of the project and ask people if they are sure that they kan keep their promises.
We are brainstorming for features that Hoot will possess. See http://hoot.tigris.org/Brainstorm.html for details.