Slashdot Mirror
Myths About Open Source Development
jpkunst writes "A thought-provoking article by chromatic on oreillynet, listing eight "myths" that Open Source developers tell themselves. For example: Myth: Publicly releasing open source code will attract flurries of patches and new contributors. Reality: You'll be lucky to hear from people merely using your code, much less those interested in modifying it."
This may be true for a minority of widely used projects, but for most applications, I've never bought this argument. Bug swatting, and especially code inspection, is and always will be a tedious process, not well-suited for a volunteer-only development community. The only advantage I see for open source in this area is that bugs can be fixed as they are encountered -- but this only works where the end user has the required skills to do the fixing in the first place.
Roving Web-Teleoperated Robot
My limited experience with open source is summed up with this article sentence:
~~~
Not all open-source projects are alike, however. A small number of open-source projects have become well known, but the vast majority never get off the ground, according to Scacchi.
~~~
Open source is obviously faster/better/cheaper when 1000's of people donate their time to a single project. The only open source project I've been involved in was a collaboration among several corporations, all of which wanted to leverage each other's resources, but none of which could really contribute their own.
There's nothing like money to motivate people to work on a project for which people aren't willing to donate their time.
Personally, I'm not convinced speed is related to developer quantity. There's too big a variation in productivity between experienced and amateur developers.
I'm also not convinced open-source is right for all types of software. How many open-source developers you know that conduct large-scale usability tests? How many open-source developers go around interviewing end users? When the developer and product consumer is the same, open-source makes much more sense to me.
The linux hacker
Myth: The GPL is the only open source license
Truth: Although it's the most popular, it's not the only license.
Sadly, I think this is what most people think of when they think of open source.
Fortress of Insanity
I mean, does anyone really think that how they package their product won't effect how many people start using it? Are there really a lot of people out there who assume that they'll have an instant dedicated following of skilled developers spring from nowhere the moment they publish their source?
I really doubt it, somehow. Charitably, I'd file the advice in this article under the "Obvious but sometimes in need of restating" catagory in that sometimes people will lose the forest for the trees. Still, no real revelations here.
Every year during my review, I just pray the words "slashdot.org" aren't mentioned.
Oh my God, this sounds exactly like my last job. 10,000 lines of Tcl, with not a shred of documentation in sight. Running a financial system that processed millions of dollars a day. And I know to this day, my old boss is still trying to figure out why she keeps losing employees left and right, and why it takes so long for new people to come up to speed.
I Am My Own Worst Enemy
It's not worth writing good design documents because everyone will read the code.
Read Epic the first RPG novel.
"I am sure that everyone will want to install Apache/mod_perl/mod_ssl and mysql and perl 5.8.3 and 17 non standard perl modules (8 of which are not available on CPAN), ImageMagick, python, zlib, libpng and glib2.1 and zend and php) to be able to use my practically useless and very buggy digital picture management system."
If you write something that is usefull and/or fun. People are going to use it. For example I use the Spreadsheet::WriteExcel module at work. Yes perl writing excel documents. I used because there was a need. I fixed a bug in one of the optional modules because that was a feature we use and need to work correctly. Would I ever picked up and use that module on my own. Maybe if I came across it and wanted to create an spreadsheet for some silly reason but I highly doubt it. But I had a need to create an excel spreadsheet on a unix server so I filled that need.
Get Movie Posters
What most developers don't think is "Hey, I didn't contribute anything. Nobody I know has contributed anything. Why will my project be any different?"
Myth 3: Reading code
I've tried to read large bodies of code before. It's damn hard, even if it is documented. And when it isn't documented, your beginning developers don't have a chance.
Myth 4: Packaging
Um...duh? Of course it needs to be properly packaged. And dependency lists? If someone can't get it to compile, they definitely won't use it.
Myth 5: Start from scratch
Don't start from scratch if the code isn't clean. Make new code clean, and go back to clean up existing code. Make sure you have those regression tests ready.
Myth 7: Perfection
Developers are humans. Humans are fallible. I'll make a perfect program - when Bullwinkle pulls a rabbit out of his hat.
Myth 8: Ignore warnings
If the warnings were ignorable, they wouldn't be there. My profs would take marks off if you got warnings in compilation, unless your documentation explained exactly why you let the warning stand (and it had better be a good reason).
Myth 9: Tracking CVS
Users don't track CVS. Developers track CVS. Users want quick-and-easy, working code.
Either I miscounted, or there's more than 8 entries on the site (they aren't numbered)
I can't say that I don't give a fuck. I've just run out of fuck to give.
I think one needs to differentiate between small and big projects. It's certainly easier to write a patch for a relatively short script, simply because it's easier to understand what it does. Try to write a useful patch for a big project like Mozilla and you'll spend quite some time trying to even understand which file you need to patch. It's obvious that smaller projects attract more patches while bigger projects attract more bug-reports.
This is true, however, most commercial developement groups already know that these myths are just that, if not the coders, then their managers at least.
The issue he is covering is the fact that many people on the FS/OSS movements beleive that these myths are true. This article is not a condemnation of the FS/OSS community, but a reality check for them.
Boobies never hurt anyone. - Sherry Glaser.
I also wrote a bunch of hacks that I just gave away, but I never expected patches, or for people to actually use it.
Open Source is like socialism, you just help out where you can, and share what you got. If people don't take it, then it's their loss :) At least it was useful for myself, and it might be useful for others.
To assume that one writes a few hundred lines of code, and then get instant fame is of course ridiculous :)
If open source developers are ever going to shake their image of being zealots, they need more of the kind of self aware pragmatism that this article provides.
Defensively crying "troll" in response to criticism isn't going to help matters any.
Granted, I don't think all of those are myths. But one really irks me as being false for any software developers:
Myth: New developers interested in the project will best learn the project by fixing bugs and reading the source code. Reality: Reading code is difficult. Fixing bugs is difficult and probably something you don't want to do anyway. While giving someone unglamorous work is a good way to test his dedication, it relies on unstructured learning by osmosis.
I work for a very niche market/profitable software company and thats exactly how the developers get their feet wet, by fixing minor bugs.
Seems like the only way to "learn a project" is to fix bugs and therefore read the code.
regardless of whether the project is an open source (or not).
We (popular IT community) are re-learning the lessons of IBM in the 60s which Fred Brooks distilled in his famous The Mythical Man-Month.
I think the bigger misunderstanding is that new developers/IT types/CS academics thinks that everything is new. Most computer security issues were first discussed based in the 1960s or 1970s. Much of Distributed Computing is now being "re-discovered" as Grid Computing.
I've found a few other misconceptions in open source development that have irked me over the years.
1. Using autoconf/automake will make my code portable.
TRUTH: You need to know what system calls are portable, which ones arent, and the nuances in using each on different platforms. The auto* tools will only make detecting and utilizing the correct versions easy. It's up to you to identify and code for them in the first place. (Ditto for compiler flags, shared libraries, linker options, etc)
2. Network programming is easy.
TRUTH: I've seen a lot of projects that implement their own network communication using TCP sockets and sprintf text messages. A number of others transmit little endian integers around. And others still use a blocking style request->response form of communication.
Good network programming is really hard, and unless you take the effort to design and implement something robust from the start, this kind of ad-hoc, inflexible networking will become embedded into the application and require significantly more rework later down the road.
And PLEASE reuse something that might fit before even attempting to write your own layer. The gnutella protocol is a great example of this problem.
3. Threading is as simple as using pthreads and mutexes.
TRUTH: Good threading code is difficult to develop and difficult to debug. It is always preferable to use an event based model where possible, and rely on threads only when you need scalability on SMP, work arounds for blocking system calls (gethostbyname_r), or background tasks that you dont want delaying interaction with a user or network app (there are many other reasons, but these give you the general idea of where threading is appropriate).
Synchronizing access to shared resources between threads is also very tricky. The level of granularity of locking, and the structure of your data structures themselves, will have a significant impact on performance. Too much granularity and you end up with extremely complex locking hierarchies that are difficult to debug, more prone to dead lock. Too little granularity and you get lots of contention for these shared resources.
Finding the sweet spot is tricky, and often requires lots of experience or tuning to get right. The lack of tools to provide visibility to lock contention and latency also make this difficult.
I'm sure there are others, but these are the big ones that come to mind.
It's a complete straw man though.
Opensource works because even though 90% of users of a project won't contribute, the 10% that do (not just code, but bug reports, comments, newbie help, documentation, etc.) make a huge difference.
Half of the stuff is assumptions I deal with *every day* from management on my paid work, so to say that OSS makes these assumptions exclusively is a pure troll.
Some of it is plain loony - saying that writing code once and sharing it is a commercial advantage is ludicrous - the *point* of OSS is that we write stuff once then share it. Commercial development does exactly the opposite, by protecting everything with patents and forcing everyone to re-invent the wheel when they write anything.
then please make it easier to contribute.
Show us your roadmap for development,
where you want us to contribute time,
and how we can get started helping you.
Make it easy to understand your software,
maybe by creating help files, diagrams,
real examples of how to use your software,
even comparisons to related software.
Source code comments are good;
technical overviews are even better.
Above all, get FEEDBACK from developers
on your source code and your documentation.
Is it clear? easy? How could it be easier?
The more your improve your documentation,
and your process for contributing code,
the more we can help you. Thanks!
Cheers, Joel
No, what the first myth was alluding to is this: when you release your OSS project into the wild, don't expect an army of l33t coders to materialize and assist you in developing it.
I've found this myself; I wrote a code for performing spectral synthesis of stars undergoing quakes, and released it under the GPL. There are quite a few asteroseismology groups around the world using the code now; but not a single person has contacted me and offered to help develop or debug the code.
As chromatic pointed out in his article, the majority of OSS projects have very few developers, even in cases where the project has a large user base.
Tubal-Cain smokes the white owl.
But we know that people _do_ contribute patches and improvements to the code. Many developers have first-hand experience of this (from both sides, contributing and receiving). So it's hardly a myth.
Unless the author of the article has done some measurements to see what proportion of users send back improvements - and there's nothing in the article to say that he has measured anything or that he maintains any free software himself - then there's no reason to believe him rather than the 'myth'.
-- Ed Avis ed@membled.com
Sorry folks, a programmer with no degree but lots of Open Source experience will still have a tougher time getting a job than a C.S. student with no experience.
It's wrong, but it's still true.
Myth: Publicly releasing open source code will attract flurries of patches and new contributors.
Reality: You'll be lucky to hear from people merely using your code, much less those interested in modifying it.
So. Just because something is open or closed source, it does not mean that it is a good program nor does it imply that anybody wants to use it.
Myth: Stopping new development for weeks or months to fix bugs is the best way to produce stable, polished software.
Reality: Stopping new development for awhile to find and fix unknown bugs is fine. That's only a part of writing good software.
I don't see too much disparity here between the "myth" and "reality".
Myth: New developers interested in the project will best learn the project by fixing bugs and reading the source code.
Reality: Reading code is difficult. Fixing bugs is difficult and probably something you don't want to do anyway. While giving someone unglamorous work is a good way to test his dedication, it relies on unstructured learning by osmosis.
This "reality" again does not dispell the "myth". Try having new developers interested in a project and reading source code in a closed source project. Yeah, its difficult to read code, but infinitely more difficult to read it if you dont have access to it. BTW, the metaphor or whatever "osmosis" is trying to make a point is pretty silly. Osmosis is the transfer of water through a semipermeable membrane.
Myth: Installation and configuration aren't as important as making the source available.
Reality: If it takes too much work just to get the software working, many people will silently quit.
Yeah, there not that important thats why we did silly stuff like create autoconf to configure and install software. That is why we carry around the install.sh form X11 to install software in a predictable and sane way. That is why we have plain readable text files to configure our software. The reality holds true for closed and open source as well.
Myth: Bad or unappealing code or projects should be thrown away completely.
Reality: Solving the same simple problems again and again wastes time that could be applied to solving new, larger problems.
This is again true for open and closed source projects. Go look at one of the windows (closed source) freeware/shareware depositories and you will find at least 5-10 programs that all do the same thing more or less. If these were open source projects, I would imagine that there would be a good amount of code reuse going on here.
Myth: It's better to provide a framework for lots of people to solve lots of problems than to solve only one problem well.
Reality: It's really hard to write a good framework unless you're already using it to solve at least one real problem.
Does anyone thing this is either a valid myth or something too terribly interesting to talk about? I will say however, that UNIX (I'm generalizing that opensource is more of a UNIX like thing here) in general is a framework and our stuff plays well with one another. We have programs have STDOUT, and STDERR messages that are formatted for external processing and parsing, we have exit statuses in our programs so they can be &&ed and ||ed or test for their success or failure. We have signals, pipes, and sockets for IPC. Look at the number of opensource installs and the wide variety of things that they do and tell me that we are not solving a number of real problems well.
Myth: Even though your previous code was buggy, undocumented, hard to maintain, or slow, your next attempt will be perfect.
Reality: If you weren't disciplined then, why would you be disciplined now?
Axiom of life. If program sucks, noone will use it. This is true for opensource and closed source stuff.
Myth: Warnings are just warnings. They're not errors and no one really cares about them.
Reality: Warnings
...and the article doesn't claim *nobody* will, just that a huge team won't materialize out of nowhere. That's about right.
I've been maintaining cscvs, a tool for breaking a CVS repository's history into changesets and (among other things) importing its contents into the GNU Arch revision control system. It's adopted a fair number of users (more as the documentation and such get better), and a number of developers have contributed patches. If I weren't quite so busy with my job right now, I'd have been able to help *another* developer with a bugfix he's asked for a hand in putting together (to fix mismangling of the repository locations of CVS repositories which have a delta between the path used in the CVSROOT and the one used in rlog output other than the single such case I'm currently fixing).
The other project my maintainership of which could be considered active within the past year would be the "Ticket Applet 2", a GNOME applet for showing and updating the status of ones' Kerberos ticket. It's received a quite major patch from one outside developer (providing compatibility with his alternate Kerberos implementation), and feedback from a number of users at my workplace -- but there was certainly no flood of support washing in the moment I put it up on freshmeat, and had I been expecting such I would have been mistaken.
I think the actual claim in the article is a lot more defendable than the little slashdot blurb -- to the point that the blurb really does both the readers and the author something of a disservice. (Indeed, the only one I completely disagree with is the argument that it isn't sometimes best to throw out working code for a complete rewrite should it become unmaintainable).
I think it's unfair to count only the perl core; count the core and CPAN. Yes, even though most CPAN modules are not included with the perl core I think this is a fairer comparison. After all the point of programming in a higher-level language like Perl is that you don't have to make patches to the core in order to develop new features; you can write them as Perl libraries.
-- Ed Avis ed@membled.com
While I'm agreed that the best way to write good software is to not introduce bugs in the first place, I don't believe that this is an entirely avoidable problem.
There are certain types of necessary changes that inherently destabilize a codebase no matter how careful you've been. It's inevitable. Oftentimes, things like this are checked in to amortize the cost of producing, fixing, and improving said code. There are the unforseen interactions that your new subsystem has, that none of the regression or unit tests have picked up. I know - "write more/better tests" is a better solution. But omnipotence is an impossible goal.
To continue the author's "home" anology, relasing software is like preparing a meal. The pots and spoons simply must get dirty when you're cooking. Many try to "clean as you go," but at the end, you're still left with your dirty casserole dish. You can either choose to clean things up before your guests get there (feature freeze), or you can leave the dirty dishes lying on the counter for all to see.
I might be inclined to say that the shorter the feature freeze, the better. But I don't have any evidence to back this up - nor does Chromatic cite any evidence (except antecdoctal) to support or detract this claim. Maybe people by nature are better at fixing a slew of bugs at once. Maybe not.
Freezes, milestones (alpha, beta) and the like are inevitable parts of producing quality software fit for public consumption, short of "papal infallibility." We're only human.
Dom
As Linux and Other Open Source software get used more and more by less tech savvy users, eg non-programmer types, as a percentage of the community contributions will seem to decline.
,or not. appreciate the work that has gone into the free software that they use from day to day. When was the last time you took stock of just how incredible that linux box with its flashy gui you are using is when you consider that it has been bought to you by the hard work of the OS community.
I think most people, tech savvy
I think people need to find their niche, as to what they can and can't do in order to contribute. Many people think because they are not a hard-core coder they cant do anything to help. I've only contributed to a couple of things since I've been using Open Source stuff.(the past 4yrs) But when I do fix a bug or create something a project might find useful I usually send any files or useful info over to the project maintainers. It is the least I can do when I owe my redmond-free world to so many dedicated geeks!
I wonder just how many regular Open Source users feel that if they could, they would help, but maybe dont know how.
I would say project maintainers should encourage people to help out in other ways, There are loads of things people can do. Artwork, Documentation, Website maintennance heck , even give free support to people if they are nice enough.
I've been helping a few newbies through their first forays into linux, as indeed friends helped me when I got started. If you plant the right seeds in those newbie minds, they most certainly will grow a giving and generous attitude.
There is one more way people can support Open Source.. Lets introduce a "Send your favorite project A Beer Day" send em some beer money!
Nick !
Electronic Music Made Using Linux http://soundcloud.com/polyp
According to that link, Alan has a BSc in CS. Linus Torvalds has a Bachealors degree in CS, and an honorary Ph.d from the same school in Finland. I'm too lazy to dig up links for that. It's in several of the books about his life.
Kirby
Throwing away code blindly is a mistake, especially if it is working code. Then again, keeping crufty, bad code around is an equally large mistake. The larger point that Chromatic misses is that making uninformed code decisions is playing Russian Roulette. Throwing away code (or keeping code around) is only a mistake when one has no concrete rationale for doing so.
The important part is to have a good understanding of the problem scope, previous attempts (if any) at solving the problem, and what their advantages and drawbacks are.
You have to remember that code doesn't exist for code's sake alone. We write code to solve problems. Code is a window into how someone solved a problem. And not all solutions are created equal.
What is important is to understand the "whys" and "hows" of these previous attempts, and then chart the best course you see toward success. It may well be that the best solution is to scrap another's design. It may be the best solution to build off of another's success. However, it's probably a bad decision to build off of another's failures.
Dom
Programs Suck; Frameworks Rule!
Myth: It's better to provide a framework for lots of people to solve lots of problems than to solve only one problem well.
Reality: It's really hard to write a good framework unless you're already using it to solve at least one real problem.
Really-Real-World Reality: Frameworks that are developed in conjunction with one specific project are likely to produce lousy results when used in a different project.
I've seen a number of "generalized" frameworks that came out of one large project, only to wreak havoc when they were forced upon the developers of another project. When people are writing support code for a project, a lot of project-specific design decisions get mistaken for generic architecture because the developers are only looking at it from an insider's perspective.
Solve your real problem first. Generalize after you have working code. Repeat. This kind of reuse is opportunistic...
This is sheer idiocy. If anyone disputes this, I've got some code I'd like to show you...
(Trying not to flame) This guy doesn't know what he's talking about. The proverbial "reinvention of the wheel" is not really reinvention. The problem is that programmers do just what he suggests - rather than think through the problem, and how they can create reusable code, they proceed to cobble together some garbage which solves only the specific problem at hand. Which leads to other programmers having to "reinvent the wheel" because the first programmer didn't make his code reusable!
You can't have it both ways. Either you reinvent the wheel every time, or you write reusable code. It's a discipline, folks - sometimes you have to put forth the extra effort up front to make gains in the long run.
The first three years as a programmer, I must have written at least half a dozen linked list implementations. It wasn't until I had worked on some large projects that I learned that writing reusable code is well worth the extra effort. I was the guy who "just coded the solution". It took me a long time to learn that the more time I spent thinking about the problem, the less time I spent on coding and debugging.
The society for a thought-free internet welcomes you.
The surest way to gaurantee involvement in a project is to create a community around it. Forums, user/contributor publishing, blogs. Anything that will let your contributors express themselves regarding the project.
Let people get involved, encourage them, provide a forum.... hopefully provide the tools (sourceforge) but also provide a unique community experience. Create a brand (read a book on marketing) and you will reap the rewards for years... think about Aibo for instance...
A fool throws a stone into a well and a thousand sages can not remove it.