It seems to me that this is actually two questions with a serious caveat.
The first caveat is how we define Success. In a strictly software engineering sense, success is the fulfillment of all of the requirements in the initial requirements doc as well as the requirements in the detailed, technical requirements document. Out in the wild, software success is measured in different terms. Terms like installed user base, overall user satisfaction and often sales or other money-related, measurable metrics. For free and open source software, I see no reason to abandon the non-monetary metrics. Installed user base and market share are two very good metrics even when the customer got the software for free.
The two questions this one breaks down to are:
1. How do we measure the progress of the project during primary development
2. How do we know that the project is successfull once completed (this is basically answered in the caveat)
The answer to those questions relies almost entirely on the stated goals of the project (the stuff in that boring requirements document) and the definition of success
Question 1 is much more interesting and useful than question 2 for various reasons. What metrics are useful in guaging the success of a project while it is in development? This encompasses a whole lot of area but the primary thrusts should still be similar to traditional software project management. Namely, creating metrics to measure the quality of the product and to measure the progress of the project.
Quality metrics include things like comparisons to the requirements, code reviews, and statistical analysis of the code to estimate the number of bugs per XXXX lines of code and thus the number of bugs in the entire project. Many metrics like this are only useful if there are industry averages available for comparison.
Progress measurement is an entirely different beast. It can be based on a timeline(planned functionality implimentation over time) or based on the overall completion of the project (we just finished 3 of our 6 primary goals so the project is roughly 50% completed). Of course, the developer may weight the functionality points differently depending on their complexity. The real pain here is requirements migration over time. This is a big problem for OSS because developers love to continually add functionality as they go and there aren't any project managers to keep them in check.
Unfortunately, there is no quick and easy answer here. Finding the answers will take a bit of work and invariably depend on the individual project in question.
Slashdot Mirror
It seems to me that this is actually two questions with a serious caveat.
The first caveat is how we define Success. In a strictly software engineering sense, success is the fulfillment of all of the requirements in the initial requirements doc as well as the requirements in the detailed, technical requirements document. Out in the wild, software success is measured in different terms. Terms like installed user base, overall user satisfaction and often sales or other money-related, measurable metrics. For free and open source software, I see no reason to abandon the non-monetary metrics. Installed user base and market share are two very good metrics even when the customer got the software for free.
The two questions this one breaks down to are:
1. How do we measure the progress of the project during primary development
2. How do we know that the project is successfull once completed (this is basically answered in the caveat)
The answer to those questions relies almost entirely on the stated goals of the project (the stuff in that boring requirements document) and the definition of success
Question 1 is much more interesting and useful than question 2 for various reasons. What metrics are useful in guaging the success of a project while it is in development? This encompasses a whole lot of area but the primary thrusts should still be similar to traditional software project management. Namely, creating metrics to measure the quality of the product and to measure the progress of the project.
Quality metrics include things like comparisons to the requirements, code reviews, and statistical analysis of the code to estimate the number of bugs per XXXX lines of code and thus the number of bugs in the entire project. Many metrics like this are only useful if there are industry averages available for comparison.
Progress measurement is an entirely different beast. It can be based on a timeline(planned functionality implimentation over time) or based on the overall completion of the project (we just finished 3 of our 6 primary goals so the project is roughly 50% completed). Of course, the developer may weight the functionality points differently depending on their complexity. The real pain here is requirements migration over time. This is a big problem for OSS because developers love to continually add functionality as they go and there aren't any project managers to keep them in check.
Unfortunately, there is no quick and easy answer here. Finding the answers will take a bit of work and invariably depend on the individual project in question.