Complexity management has been a problem in software development from the beginning, and this has been known for a long time (see The Mythical Man-Month by Fred Brooks, a book nearly thirty years old).
What I find interesting is that there really have been no revolutionary improvements in complexity management in all this time. There have been incremental improvements, such as software tools like more advanced languages to automate some of the most menial work of programming or revision control systems that provide improved concurrency control and communication about software changes, and somewhat improved design methodologies like object-oriented design, but despite some grandiose claims nothing has really solved the problem that designing and building complex software is hard.
Open Source software is not immune to this problem. What source code availability and open development processes have done is introduce other incremental improvements, mainly increasing the available labor for implementation and debugging. Successful open-source projects tend to remain focused on fairly specific goals (usually development of specific applications) and have efficient centralized management. These usually translate into advantages over commercial software development projects working on projects of the same scale because open development tends to distribute work over more people and more time than commercial developers can afford.
Lewis doesn't seem to acknowledge that commercial software is equally susceptible to the complexity problem, or that the smaller number of participants in most commercial projects put them at a disadvantage for doing the grunt work of implementation and debugging. His assertion that open source software is somehow going to be unable to obtain wide market share is, I think, based more on cultural and psychological factors such as the pervasive belief that you're getting more if you pay for something than if you get it for free rather than on any inherent weaknesses of open source software relative to commercial software.
However, I do think it's important for Open Source software advocates to realize that it is just as susceptible to the complexity problem, and that distributed development is only an incremental rather than a revolutionary improvement in programming technology -- a brute-force solution that uses greater amounts of conventional effort to attack the problem, rather than a fundamental change in approach that would allow similarly complex projects to be done with less effort.
There are a lot of other flaws in Lewis's article, such as the assumption that Open Source software is about market share rather than people solving problems, that large commercial enterprises are "strong" and open development is "weak", or the various factual errors in his characterization of Linux and other open source software, but the biggest flaw to me is his apparent assumption that commercial development is capable of solving problems that open development is not even though both use essentially the same software engineering methods.
Slashdot Mirror
What I find interesting is that there really have been no revolutionary improvements in complexity management in all this time. There have been incremental improvements, such as software tools like more advanced languages to automate some of the most menial work of programming or revision control systems that provide improved concurrency control and communication about software changes, and somewhat improved design methodologies like object-oriented design, but despite some grandiose claims nothing has really solved the problem that designing and building complex software is hard.
Open Source software is not immune to this problem. What source code availability and open development processes have done is introduce other incremental improvements, mainly increasing the available labor for implementation and debugging. Successful open-source projects tend to remain focused on fairly specific goals (usually development of specific applications) and have efficient centralized management. These usually translate into advantages over commercial software development projects working on projects of the same scale because open development tends to distribute work over more people and more time than commercial developers can afford.
Lewis doesn't seem to acknowledge that commercial software is equally susceptible to the complexity problem, or that the smaller number of participants in most commercial projects put them at a disadvantage for doing the grunt work of implementation and debugging. His assertion that open source software is somehow going to be unable to obtain wide market share is, I think, based more on cultural and psychological factors such as the pervasive belief that you're getting more if you pay for something than if you get it for free rather than on any inherent weaknesses of open source software relative to commercial software.
However, I do think it's important for Open Source software advocates to realize that it is just as susceptible to the complexity problem, and that distributed development is only an incremental rather than a revolutionary improvement in programming technology -- a brute-force solution that uses greater amounts of conventional effort to attack the problem, rather than a fundamental change in approach that would allow similarly complex projects to be done with less effort.
There are a lot of other flaws in Lewis's article, such as the assumption that Open Source software is about market share rather than people solving problems, that large commercial enterprises are "strong" and open development is "weak", or the various factual errors in his characterization of Linux and other open source software, but the biggest flaw to me is his apparent assumption that commercial development is capable of solving problems that open development is not even though both use essentially the same software engineering methods.