Why Your Software Project Is Failing
An anonymous reader writes: At OSCON this year, Red Hat's Tom Callaway gave a talk entitled "This is Why You Fail: The Avoidable Mistakes Open Source Projects STILL Make." In 2009, Callaway was starting to work on the Chromium project—and to say it wasn't a pleasant experience was the biggest understatement Callaway made in his talk. Callaway said he likes challenges, but he felt buried by the project, and reached a point where he thought he should just quit his work. (Callaway said it's important to note that Chromium's code is not bad code; it's just a lot of code and a lot of code that Google didn't write.) This was making Callaway really frustrated, and people wanted to know what was upsetting him. Callaway wanted to be able to better explain his frustration, so he crafted this list which he called his "Points of Fail."
"he should jut quite his work"
not a lot, just a little
What's wrong with open sourcing previously closed source projects?
I guess linux fails since Linus wrote his own source control for it.
If we're going to talk about Callaway's Points of Fail, and create a link in the Slashdot summary that *looks* like it takes you to that list, then perhaps there should actually be a link to the list.
Callaway's original Points of Fail blog post.
You know, instead of the usual Slashdot way of pointing to an article wrapper that talks briefly about some of the points and then eventually links to the real list.
~Idarubicin
His list, instead of the link to a blog with an article about the list. That blog post is interesting - though the picture of the author scratching is just weird. Was that taken at a lice convention?
>> 1) If your codebase is too big, it's going to limit who's going to be able to download your code.
>> 2) There is no good reason in 2015 for a FOSS project to not have public source control. This helps people contribute and determine the health of your project based on the date of the last commit.
>> 3) If your source control has no web viewer and/or no documentation, these two are obvious things to have
>> 4) Code that doesn't build is worse than no code! You need documentation on how to build the project from the source.
>> 5) Use build tools
>> 6) Bundling is not going not be maintainable. Bundling leads to forking.
>> 7) Forcing people to install only in a specific directory
My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub, TFS online or one of the popular choices most FOSS projects already seem to use. (e.g. How would an experienced developer get these problems in the first place?) #4-6 are entry-level build issues. #7 refers to a best practice (let people pick their install directory) that's been commonplace in the industry for at least 15 years.
I see he's employed by Red Hat. Does this list as news suggest that Red Hat's internal development processes are immature too?
If it is common sense why do we have to pound it into people time and time again? Or do we just never learn?
putting the 'B' in LGBTQ+
"How would an experienced developer get these problems in the first place?"
A lot of projects do not follow widely-accepted best practices... even if they are experienced... and that is a problem!
A remarkable number of OSS projects fail to have a public source control system (#2). That includes many established projects that everyone depends on. Actually, a number of OSS projects - and projects that people THINK are OSS but are not (because they have no license) - fail many of these points. It's not that Red Hat's internal processes are immature; Tom was trying to bring in software from someone else (Google in this case) and was fed up by the poor practices from people who should know better.
Yes, #7 refers to a best practice (let people pick their install directory) that's been around for at least 20 years and probably much longer, but it's still widely NOT followed.
Anyway, that's Tom's point; there are a lot of widely-accepted best practices that are NOT followed, and that needs to change.
- David A. Wheeler (see my Secure Programming HOWTO)
As a software dev for closed source, our problems are creeping into open source at an alarming rate. Standups, Kanban, Scrum, swim lanes, and other political middle management bullshit is making it harder and harder to as theo de raadt once said, "shut up and hack."
The other issue is runaway devs. Gnome and KDE turned into piss pots almost overnight because they followed lockstep with whatever was trending. gnome grew hotspots that were clickable and draggable in an attempt to appeal to tablets, and KDE's widget framework turned into a swirling vortex of lights and colours that chewed through ram like none other. And the "fuck it lets move on" mentality has got to stop. Pottering epitomizes the swinging dick Linus so rightly kicked after his team was called out for set it and forget it code that ultimately broke more things and didnt play nice.
bottom line: dont lose focus in stability and function.
Good people go to bed earlier.
I saw it on the Reg.
putting the 'B' in LGBTQ+
I can't believe the billions of bits that died in the production and subsequent redistribution of this article. We're all dumber now for reading it.
I shall now drown my sorry in beer to see if I can't placate the inexorable struggle of those brain cells trying to make sense of this diarrhea and those others that are saying "WTF, Why did you read this?!? This is another let me share a bunch of drivel shit article. It's like Show and Tell in Kindergarten, just ignore it!"
Harrison's Postulate - "For every action there is an equal and opposite criticism"
I'm curious; why do you think that? Is it just the slam against "Microsoft Visual Anything"?
Note that he gives points of FAIL for configuring source "with a handwritten shell script" (+10), "editing flat text config files" (+20), and "by editing code header files manually" (+30).
If you wanted to compile the source code for Python 2.7 on Windows, you needed Microsoft VS2008 to do so. You couldn't use VS2010 or later. VS2008 is hard to come by and difficult to work with.
The summary starts with:
"At OSCON this year, Red Hat's Tom Callaway gave a talk [...]"
The summary has a link, which points to the article, which says:
"At OSCON this year, Red Hat's Tom Callaway gave a talk [...] My favorite part of this talk was Callaway's passion for the items on the list."
So where is the video. The list felt a bit bland so I also got the notion that the video would be more informative.
"editing flat text config files"
All while working for RedHat. RPM relies on shell scripts and doesn't have a reliable rollback/commit mechanism.
Is it just the slam against "Microsoft Visual Anything"?
But yeah, this obvious attempt at slamming business competition under the guise of technical know-how is oh, so 1995 (which was 20 years ago). But, in todays world, we have gotten to the point when it is not only easier, but more reliable to generate code than to write it by hand. And while they have some learning curve, visual code analysis tool are still better than text-only ones. Even the resurgence of C can be mostly attributed to the fact it's simpler syntax makes it easier to generate than the new C++ syntax.
Any guest worker system is indistinguishable from indentured servitude.
This is a checklist from any decent software project management book. Here's a good one written over 40 years ago:
Mythical Man Month
And the issue of managing a large code base can be handled by a modern VCS system like Perforce that let's fetch and work with only the portion of the code base you're interested in.
Python might be a special case with two versions, 2.7 and 3.4, being available. Although people are encouraged to use the 3.4 moving forward, a lot of people are still using 2.7. The next version of Python, 3.5, will compile with VS2015.
"You've written your own source control for this code [ +30 points of FAIL ]"
Thanks for providing the links. Where were you when I needed help two years ago?
What do you mean "locked to a single platform". I admit that I haven't tried it, but they give away the source code to VS 2015. Which is pretty much why RedHat it trying to claim that code which used to be owned by a single company is a point of failure. It's another business swipe at MS. RedHat is running for the hills because pretty soon they'll lose all purpose.
Any guest worker system is indistinguishable from indentured servitude.
Here is a video of Tom giving roughly the same talk at the Southeastern Linuxfest in 2011. https://www.youtube.com/watch?...
I wanted to run my own social networking site just for me and my friends using a FOSS project, so I was excited about Diaspora, then I saw that it requires Node.js. I have no interest in setting my server up for that. I imagine this selection was made because developers think Ruby is cool and PHP is boring and lame. Unfortunately, whatever the justification was, to make Diaspora work you need to have, you know, Diasporas, but if the only people using the project are those that manage their own Node.js server, then the already puny market size of available Diasporas has just shrunk by several orders of magnitude. It really needed to be a project that could be installed on any generic LAMP server, but the developers are so rarely interested in this boring aspect (this is actually the case across many engineering fields, it's why companies hire marketers) that left to manage their own projects they fail to achieve their stated goals.
So I took a look at GNU Social, which is written in PHP. Unfortunately, they also fail the marketing test. The project seemed to revolve around making a 'federated' social networking system. However, the actual features of the social networking seemed to be trumped by trying to make the federated system work. From a marketing perspective, they put the cart before the horse. How many users want a circa 2009 facebook clone? I bet a fairly high number, but GNU Social doesn't even offer that level of functionality. The 'federation' of the system should be viewed more as a distribution element, so, you know, before going to distribution, you should have a product that people want to distribute, and GNU Social is not that.
I once took an excursion to Reddit, and later HN. Unlimited up/down voting sucks when dealing with a hive-mind.
Headline contains the word "your" and is therefore clickbait.
systemd is Roko's Basilisk.
People start projects because they want to create something cool, not because they want to be project managers. But if they don't manage the project it fails.
Obviously not on Windows.
What do you mean "locked to a single platform". I admit that I haven't tried it, but they give away the source code to VS 2015.
I don't think having access to the source code to VS 2015 is going to allow anyone to compile VS for any non-Windows platform. Not unless you have a few million man-hours available for porting and redesign (since much of the functionality present in VS wouldn't even make sense outside of Windows)
I don't care if it's 90,000 hectares. That lake was not my doing.
And not just software. Look at security as well. And so many other computer-related areas.
For me it's more like ... someone "learned" one way of handing it when s/he was working ALONE.
Then that person never learned that the practices need to be changed when you are part of a TEAM.
And releasing your code to the public is being part of a team.
You are right. It's not an open source project. All it does is open its source. C'mon. Bridging API frameworks is where "it's at" today.
Any guest worker system is indistinguishable from indentured servitude.
My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub
I struggled to find what in the list actually applied to Chromium - However the one thing that definitely applies to Chromium is the massive code base... Chromium source really is fucking massive (even if you just clone with depth=1) and it takes a fuck load of time to do your first compile.
I really don't see how using one git host over another is going to solve that, once the host has reasonable resources (pretty easy in this day and age) then the users connection is the bottle neck, and it makes the build process slow... this creates a significant barrier for contributors. I don't particularly like the idea of a massive code base, it's well known that more code == more bugs, and it just makes it harder to comprehend the whole thing... I don't really know if it's necessary from chromium or if it's just grown large with the speed of development, i'd like to know if anyone has some insight into this.
this a slow news day? The list was crafted in 2009!
Political debates have me rolling my eyes so much I think I got optical whiplash. I should sue. - Foamy The Squirrel
Should not be that challenging to compile any version of Python with newer VS. Just because the binary builds used 2008 doesn't mean you couldn't crank out a build with a newer version.
Can I go on the record as saying how SHITTY Python 3 is, btw? I mean wtf? If your language requires all kinds of weird distinctions between unicode and non unicode strings such that it takes nontrivial work to make that shit work between modules and different calls, your language is shit. I don't have to deal with that nonsense nearly as much in .NET and it supports Unicode.
I'm sticking with Python 2.7.
That's a problem with human nature, not just devs. We are not Vulcans. Humans are impatient, egotistical, fixate on the wrong factors, and often just plain random; and most don't know it or care.
I know some well-educated people who are complete idiots outside of their narrow specialty. I'm probably an idiot also in ways I don't even realize (please don't educate me in replies). My head-model of the world is perfectly logical and consistent to me, but it's probably highly lossy against the real world.
Gee, it's almost as if we are merely upright apes who happen to be able to talk and write. (I would have said "hairless", but I'm hairier than the orangutans I see in the zoo.) They fling poo, we fling nukes.
Table-ized A.I.
mongodb? Use something that's been tried and true for at least 10 years. Go with MySql or PostGresql and screw the noSql toys until they mature and have decent docs.
Let pioneers take the arrows, the rest of us stay in proverbial Boston, which has infrastructure and seasoned specialists, and get shit done. And we have nice lawns to kick fanboys off of.
Table-ized A.I.
== Size ==
* The source code is more than 100 MB. [ +5 points of FAIL ]
* If the source code also exceeds 100 MB when it is compressed [ +5 points of FAIL ]
125372299 Jul 22 00:36 linux-4.1.3.tar.gz
== Source Control ==
* You've written your own source control for this code [ +30 points of FAIL ]
git was written originally for Linux kernel!
== Building From Source ==
* Your source is configured with a handwritten shell script [ +10 points of FAIL ]
Even worse: handwritten C program: scripts/kconfig/mconf that is compiled and run by "make menuconfig".
== Bundling ==
* Your source only comes with other code projects that it depends on [ +20 points of FAIL ]
* If you have modified those other bundled code bits [ +40 points of FAIL ]
Compression and encryption libraries, for example.
== Libraries ==
* Your code only builds static libraries [ +20 points of FAIL ]
* Your source does not try to use system libraries if present [ +20 points of FAIL ]
Not even glibc!
== Code Oddities ==
* Your code depends on specific compiler feature functionality [ +20 points of FAIL ]
Lots of places where gcc-specific code is used.
== Licensing ==
* Your code does not have per-file licensing [ +10 points of FAIL ]
=== FAIL METER ===
Total: 180, highest possible:
135+ points of FAIL: So much fail, your code should have its own reality TV show.
Pretty much off-topic: What is OSX .zip format, as opposed to .zip format? I don't think I have ever seen anything (software releases or other) that I'd know was a "special" OS X zip.
If it is common sense why do we have to pound it into people time and time again? Or do we just never learn?
Everything is obvious when you understand it?
If common sense was common... would people still say patently stupid things like "you can lead a horse to water but you can't make it drink" (want a bet?), "doesn't have the horse sense to stay out of the rain" (clearly never owned horses, they will seek shelter from rain - in some conditions - it's the broad brush of stupid, though some horses are smarter than others), "you'll catch more flies with honey than with vinegar" (try that with a fly trap).
I could go on... oh wait, I just did.
Dear coward
Sometimes the latter, but sometimes, even if you were to follow all his points, there's just not enough interest in the project to make it sustainable. It all sounds so easy coming from a developer who works at one of the biggest open source companies there is that can devote enough resources to a project to make it successful. Try asking some independent developers of a successful project what they attribute their success to and they may have a different list altogether.
He hasn't always worked for Red Hat. They are a good list of rules to use if you want to see where your project could be improved. I suspect you have some obscure definition of success, and/or missed the implicit definition of FAIL.
Try asking independent developers for a list of general reasons why projects fail to get wider usage, grow, and last - I have (that's an old list), the resulting list is pretty much the same, only the scores seem to change.
That list is well out of date. The most recent version is here.
== Size == * If the source code also exceeds 100 MB when it is compressed [ +5 points of FAIL ] 125372299 Jul 22 00:36 linux-4.1.3.tar.gz
FAIL. linux-4.1.3.tar.xz 22-Jul-2015 00:36 79M [ -5 points of FAIL]
== Source Control == * You've written your own source control for this code [ +30 points of FAIL ]
FAIL Like quoting the largest archive choice for the source code, you choose the wrong meaning of "own". That's not the meaning implicit in Tom's list - nor does it fit his reasoning (using an obscure versioning system limits use and development).
"There is no publicly available source control (e.g. cvs, svn, bzr, git)" . [ -30 points of FAIL]
== Building From Source == * Your source is configured with a handwritten shell script [ +10 points of FAIL ]
FAIL. It's not handwritten. It could be. There's difference. [ -10 points of FAIL]
Even worse: handwritten C program: scripts/kconfig/mconf.c that is compiled and run by "make menuconfig".
FAIL, most source code is handwritten. C source is not a script
== Libraries == * Your code only builds static libraries [ +20 points of FAIL ]
FAIL Demonstrably Linux supports dynamic libraries and can (often is) built without static libraries only. [ -20 points of FAIL]
* Your source does not try to use system libraries if present [ +20 points of FAIL ]
FAIL. Unless you can produce a citation that shows that Linux will try and use system libraries when the same libraries have been statically compiled (yes, even glibc). [ -20 points of FAIL]
== Code Oddities == * Your code depends on specific compiler feature functionality [ +20 points of FAIL ]
FAIL. If that were the case gcc would be the only possible compiler. It's demonstrably not. [ -20 points of FAIL]
== Licensing == * Your code does not have per-file licensing [ +10 points of FAIL ]
FAIL. Unless you've got a citation for that. I'm sure M$ would love to hear of this invalidation of the kernel license. [ -10 points of FAIL]
=== FAIL METER === Total: 180, highest possible: 135+ points of FAIL: So much fail, your code should have its own reality TV show.
Let me fix that for you:-
=== FAIL METER ===
20 points of FAIL
5-25 points of FAIL: You're probably doing okay, but you could be better. (um, maybe Linux is doing OK - one of two people use it)
Your score:-
=== FAIL METER ===
115 points of FAIL
95-130 points of FAIL: HONK HONK. THE FAILBOAT HAS ARRIVED!
"you can lead a horse to water but you can't make it drink (want a bet?)"
Short of violence and/or other behavior that most would consider animal abuse? Go figure that normal people exclude that as a valid solution to the 'problem'.
"doesn't have the horse sense to stay out of the rain"
(clearly never owned horses, they will seek shelter from rain -[...]"
"Horse sense" is a synonym for "good sense" or "sound judgment". The implication is that horses WILL stay out of the rain, and otherwise exhibit good sense. You mis-understood the proverb completely; and it means the opposite of what you think.
"you'll catch more flies with honey than with vinegar" (try that with a fly trap).
Fine you win one, sort of... if you get to pick the species of fly in question. Yes certain species of fruit flies are attracted to the scent of vinegar. Other species not so much.
"you can lead a horse to water but you can't make it drink (want a bet?)"
Short of violence and/or other behavior that most would consider animal abuse? Go figure that normal people exclude that as a valid solution to the 'problem'.
The limitations of your experience define the frame. Most animals like salt - but feel free to call PETA if I put a tiny dab inside a horse's lip.
If that horse has colic I will be cruel in order to get that horse to drink. The crueltry is relative. If you did think that unreasonably cruel to do to a horse with colic then you'd be deserving of a horse whipping.
"doesn't have the horse sense to stay out of the rain" (clearly never owned horses, they will seek shelter from rain -[...]"
Careful you don't hurt your back lugging those goal posts around.
"Horse sense" is a synonym for "good sense" or "sound judgment".
The implication is that horses WILL stay out of the rain, and otherwise exhibit good sense.
[golf clap] Such a wit(ling).
...means exactly or nearly the same". In your desperate bounds from bank to bank you pass over the obvious. WILL != do. So the "implication" - is wrong. Hint: a little rain won't hurt you (cold will).
The point I tried to make earlier, which you ignored, is that common sense is not common - because it's counter intuitive. Your reaction reinforces that.
It'd be simpler, and more accurate, to just say "they don't possess the intelligence to act in their own best interest" don't you think? (of course that may raise the question of why "they" feel the need to use a more obscure choice of words)
You mis-understood the proverb completely; and it means the opposite of what you think.
I don't think so (you'll ruin you eyes squinting like that). Context is the point. Is it context that eludes you? Or just the meaning of "a thread" and the ability to follow it's tangents?
"you'll catch more flies with honey than with vinegar" (try that with a fly trap).
Fine you win one, sort of... if you get to pick the species of fly in question.
If you think it's about scoring points you're doing two things wrong. I don't see it as winning points. I take it personally - if I test my opinion and find I'm wrong I consider that a win. No prizes for foolish pride. In this case I've won nothing. On the other-hand, maybe you could. Just a thought - it won't kill you.
And no, "I" don't get to pick the species of fly, that needs to be defined by the person trotting out the proverb like it is some valuable piece of wisdom instead of frontier gibberish passed off as a precept.
Yes certain species of fruit flies are attracted to the scent of vinegar. Other species not so much.
With the obvious exception of blow flies, horse flies, march flies, house flies, little house flies, cluster flies, meat flies, and most other flies found around homes and farms. I qualified my statement - fly traps. I never managed to catch any with honey (other insects, yes) if there was other traps nearby with vinegar. The second best mixture proved to be sugar and vinegar (better than just sugar).
That's not a unique finding. How hard is it to check, especially compared to a knee-jerk defence of what you've always assumed to be true but never tested?
It's demonstrably more truthful to say "you catch more flies with shit than with honey". The interesting question would be why do so many people trot out such proverbs like that which aren't self-supporting? The answer is often that the proverb is self-serving. They wish to make a proverb a precept. The devil is a gentleman whose speech is honeyed. The truth is not pretty and those that find it challenges their over-investment in a emotional belief resort to an ad-hominem argument, red herring, strawmen, or convoluted interpretations and appeals to authority in lieu of a valid argument.
>not because they want to be project managers. But if they don't manage the project it fails.
This. Creating something is fun. Managing is not. But, you HAVE to manage it, or it becomes some ridiculous unwieldy sword made of butter and you can't explain why it's made of butter.
Carpe Canem - Seize the Dog
I should have put large <humour> ... </humour> tags around my comment.
I found quite ironic that a project that could have been _formally _ assigned so many fail points has not failed in any meaningful sense of fail.
Python and extensions have to compile to the same version of Visual Studio to work together. You need VS2008 for version 2.7 if you want to stay compatible with everyone else. A lot of people are still sticking to 2.7 for now.
I should have put large <humour> ... </humour> tags around my comment.
I found quite ironic that a project that could have been _formally _ assigned so many fail points has not failed in any meaningful sense of fail. If were to look at my posting history my reply was notably gentle.
I figured it twelve to a dozen on whether you were taking the piss (that's Oz-speak for good clean fun). Most of the fail points you gave had already been made in the comments on the old version of the list - including some truly brainless one's you missed (no documentation, no website, no bug tracker).
Humour tags would have just spoiled the fun. Gotta think of the larger audience. [smile]
For a good laugh at mega-TV-series FAILS - one of our consultants sent me a link to this. (I knew there was a reason we pay him the big bucks) We need a bigger FAIL index
What is this code generator you talk of? The only thing I can think of is a compiler for a higher-level language. This may be extremely useful, but it has to take some input, and that input has to have an unambiguous meaning that describes a problem somehow, and therefore that input is a program that was written by a programmer.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
Nowhere in the list does QUALITY come into play aside from having a bug tracking system. Needs to add "No publicly transparent process in place for bug fixing [ +100 points of FAIL ]" and also "New features always take precedence over bug fixing [ +50 points of FAIL ]". I guess for both open and closed source software projects quality is still considered optional and the first thing to get crossed off the list. Sadly, users put up with that. I also wonder how the size limit is measured. Does that exclude code comments? I like code that has a lot of comments explaining in plain simple language what the next 3 to 5 lines of code do. It is much easier to decipher than trying to compile a programming language intended for machines in my head. Lastly, that list arrogantly assumes that there is no open source software created on Windows and for Windows. Even using closed source runtimes that are freely available and freely distributable are not a problem. In the end one big metric for success is always ROI. That applies to open or closed source applications. If it is a pain to install and use forget about it. So let me propose one more thing for the list: Not providing a commonly used binary package format for your software for the targeted platform (most users don't want to deal with make install and compiler output!) [ +50 points of FAIL ]
You are right. It's not an open source project. All it does is open its source.
I never said it wasn't open source. My comment was about whether or not it's "locked to a single platform". I assert that without a herculean porting effort, it effectively is.
C'mon. Bridging API frameworks is where "it's at" today.
I'll believe it when I see it. I guess getting it to run under WINE might be doable, but then again that was equally doable (at least in principle) when the source was closed.
I don't care if it's 90,000 hectares. That lake was not my doing.