Why Your Software Project Is Failing

An anonymous reader writes: At OSCON this year, Red Hat's Tom Callaway gave a talk entitled "This is Why You Fail: The Avoidable Mistakes Open Source Projects STILL Make." In 2009, Callaway was starting to work on the Chromium project—and to say it wasn't a pleasant experience was the biggest understatement Callaway made in his talk. Callaway said he likes challenges, but he felt buried by the project, and reached a point where he thought he should just quit his work. (Callaway said it's important to note that Chromium's code is not bad code; it's just a lot of code and a lot of code that Google didn't write.) This was making Callaway really frustrated, and people wanted to know what was upsetting him. Callaway wanted to be able to better explain his frustration, so he crafted this list which he called his "Points of Fail."

Comments

a little more proof reading please

[Anonymous Coward]

"he should jut quite his work"

not a lot, just a little

Re: a little more proof reading please

[Reason58]

This is /. Having horrific "editors" is to be expected. I wonder what it pays to click accept on a story every hour or so.

Re: a little more proof reading please

[__aaclcg7560]

Actually, the quoted paragraph was copied and pasted with that mistake from the article.

Re: a little more proof reading please

[ArcadeMan]

If I had a Bitcoin for every time I've seen the quiet/quite error, I'd be rich.

But quit/quite? This is the first time I see that. That's how bad the editors are on this website.

Re: a little more proof reading please

[Anonymous Coward]

editors? Surely they're just the names of some manky PERL scripts?

Re: a little more proof reading please

[MobileTatsu-NJG]

This whole situation makes me sic.

Re: a little more proof reading please

[Nidi62]

In Slashdot's defense, I've seen much more glaring errors on actual journalist sites such as CNN. So it is a growing trend, not just here.

Re: a little more proof reading please

[Virtucon]

i found it quite amusing, quit using the grammatical issue as a club.

Re: a little more proof reading please

[pj2541]

Have you seen such things on actual journalist's sites, or only on CNN? For that matter, are there any actual journalists left? It seems most so called journalists can't even read or write English any more.

Re: a little more proof reading please

[phantomfive]

NYT, WSJ do a lot of good work (no one ever did all good work). A lot of smaller regional newspapers do good journalism, but they are smaller and less useful.

Re: a little more proof reading please

[Anonymous Coward]

Don't quite jut yet.

Re: a little more proof reading please

[Anonymous Coward]

According to Soylent news, DICE is selling Slashdot.

Re: a little more proof reading please

[Anonymous Coward]

the first time I've seen that

self-serving list

[Anonymous Coward]

His list isn't particularly insightful, just generally common-sense. Some of his items are rather self-serving and is funny coming from a Red Hat employee.

If more than 50 percent of your contributors work for one company
If you don't have governance or you depend only on the company that released the code for governance

Re:self-serving list

[plopez]

If it is common sense why do we have to pound it into people time and time again? Or do we just never learn?

Re:self-serving list

[Anonymous Coward]

Sometimes the latter, but sometimes, even if you were to follow all his points, there's just not enough interest in the project to make it sustainable. It all sounds so easy coming from a developer who works at one of the biggest open source companies there is that can devote enough resources to a project to make it successful. Try asking some independent developers of a successful project what they attribute their success to and they may have a different list altogether.

Re:self-serving list

[tomhath]

People start projects because they want to create something cool, not because they want to be project managers. But if they don't manage the project it fails.

Re:self-serving list

[Demonoid-Penguin]

If it is common sense why do we have to pound it into people time and time again? Or do we just never learn?

Everything is obvious when you understand it?

If common sense was common... would people still say patently stupid things like "you can lead a horse to water but you can't make it drink" (want a bet?), "doesn't have the horse sense to stay out of the rain" (clearly never owned horses, they will seek shelter from rain - in some conditions - it's the broad brush of stupid, though some horses are smarter than others), "you'll catch more flies with honey than with vinegar" (try that with a fly trap).

I could go on... oh wait, I just did.

Re:self-serving list

[Demonoid-Penguin]

Dear coward

Sometimes the latter, but sometimes, even if you were to follow all his points, there's just not enough interest in the project to make it sustainable. It all sounds so easy coming from a developer who works at one of the biggest open source companies there is that can devote enough resources to a project to make it successful. Try asking some independent developers of a successful project what they attribute their success to and they may have a different list altogether.

He hasn't always worked for Red Hat. They are a good list of rules to use if you want to see where your project could be improved. I suspect you have some obscure definition of success, and/or missed the implicit definition of FAIL.

Try asking independent developers for a list of general reasons why projects fail to get wider usage, grow, and last - I have (that's an old list), the resulting list is pretty much the same, only the scores seem to change.

Re:self-serving list

[vux984]

"you can lead a horse to water but you can't make it drink (want a bet?)"

Short of violence and/or other behavior that most would consider animal abuse? Go figure that normal people exclude that as a valid solution to the 'problem'.

"doesn't have the horse sense to stay out of the rain"
(clearly never owned horses, they will seek shelter from rain -[...]"

"Horse sense" is a synonym for "good sense" or "sound judgment". The implication is that horses WILL stay out of the rain, and otherwise exhibit good sense. You mis-understood the proverb completely; and it means the opposite of what you think.

"you'll catch more flies with honey than with vinegar" (try that with a fly trap).

Fine you win one, sort of... if you get to pick the species of fly in question. Yes certain species of fruit flies are attracted to the scent of vinegar. Other species not so much.

Re:self-serving list

[Demonoid-Penguin]

"you can lead a horse to water but you can't make it drink (want a bet?)"

Short of violence and/or other behavior that most would consider animal abuse? Go figure that normal people exclude that as a valid solution to the 'problem'.

The limitations of your experience define the frame. Most animals like salt - but feel free to call PETA if I put a tiny dab inside a horse's lip.
If that horse has colic I will be cruel in order to get that horse to drink. The crueltry is relative. If you did think that unreasonably cruel to do to a horse with colic then you'd be deserving of a horse whipping.

"doesn't have the horse sense to stay out of the rain" (clearly never owned horses, they will seek shelter from rain -[...]"
Careful you don't hurt your back lugging those goal posts around.

"Horse sense" is a synonym for "good sense" or "sound judgment".

The implication is that horses WILL stay out of the rain, and otherwise exhibit good sense.

[golf clap] Such a wit(ling).
...means exactly or nearly the same". In your desperate bounds from bank to bank you pass over the obvious. WILL != do. So the "implication" - is wrong. Hint: a little rain won't hurt you (cold will).
The point I tried to make earlier, which you ignored, is that common sense is not common - because it's counter intuitive. Your reaction reinforces that.
It'd be simpler, and more accurate, to just say "they don't possess the intelligence to act in their own best interest" don't you think? (of course that may raise the question of why "they" feel the need to use a more obscure choice of words)

You mis-understood the proverb completely; and it means the opposite of what you think.

I don't think so (you'll ruin you eyes squinting like that). Context is the point. Is it context that eludes you? Or just the meaning of "a thread" and the ability to follow it's tangents?

"you'll catch more flies with honey than with vinegar" (try that with a fly trap).

Fine you win one, sort of... if you get to pick the species of fly in question.

If you think it's about scoring points you're doing two things wrong. I don't see it as winning points. I take it personally - if I test my opinion and find I'm wrong I consider that a win. No prizes for foolish pride. In this case I've won nothing. On the other-hand, maybe you could. Just a thought - it won't kill you.
And no, "I" don't get to pick the species of fly, that needs to be defined by the person trotting out the proverb like it is some valuable piece of wisdom instead of frontier gibberish passed off as a precept.

Yes certain species of fruit flies are attracted to the scent of vinegar. Other species not so much.

With the obvious exception of blow flies, horse flies, march flies, house flies, little house flies, cluster flies, meat flies, and most other flies found around homes and farms. I qualified my statement - fly traps. I never managed to catch any with honey (other insects, yes) if there was other traps nearby with vinegar. The second best mixture proved to be sugar and vinegar (better than just sugar).

That's not a unique finding. How hard is it to check, especially compared to a knee-jerk defence of what you've always assumed to be true but never tested?

It's demonstrably more truthful to say "you catch more flies with shit than with honey". The interesting question would be why do so many people trot out such proverbs like that which aren't self-supporting? The answer is often that the proverb is self-serving. They wish to make a proverb a precept. The devil is a gentleman whose speech is honeyed. The truth is not pretty and those that find it challenges their over-investment in a emotional belief resort to an ad-hominem argument, red herring, strawmen, or convoluted interpretations and appeals to authority in lieu of a valid argument.

Re:self-serving list

[boomer_rehfield]

>not because they want to be project managers. But if they don't manage the project it fails.

This. Creating something is fun. Managing is not. But, you HAVE to manage it, or it becomes some ridiculous unwieldy sword made of butter and you can't explain why it's made of butter.

I disagree with some of these points

[Anonymous Coward]

What's wrong with open sourcing previously closed source projects?
I guess linux fails since Linus wrote his own source control for it.

Re:I disagree with some of these points

[Anonymous Coward]

What's wrong with open sourcing previously closed source projects?

Ever seen the java or openoffice source? Needs ten years of cleanup before it is even slightly sane, or can be built without attorneys going AWOOGA?

Re:I disagree with some of these points

[TheRaven64]

It depends a lot on the codebase. Codebases tend to accumulate cruft. Having people refactor them because their requirements are different to yours can help, as can having a project developed without key product ship dates as the driving force. The bigger barrier is culture though. It's really hard to have a group of developers that have been working on a project for 10 years in private move to developing in public. In the list, he actually gives different numbers of fail points, more for projects that were proprietary for longer than they were open, which makes a lot more sense than the summary in the 'article'.

The one that I disagree with is 'Your source builds using something that isn't GNU Make [ +10 points of FAIL ]'. I disagree for two reasons. The first is that it implies using GNU make features, which likely means that you're conflating building and build configuration (which should gain some fail points). The projects that I most enjoy hacking on use CMake and Ninja for building by default (CMake can also emit POSIX Makefiles that GNU Make can use, but I take his point to mean that gmake is the only command you need to build, so the CMake dependency would be a problem). LLVM still more or less maintains two build systems, though the autoconf + gmake one is slowly being removed in favour of the CMake one. If I make a small change, it takes Ninja less time to rebuild it than it takes gmake to work out that it has nothing to do if I don't make any changes.

I'd also disagree with 'Your code doesn't have a changelog' - this is a GNU requirement, but one that dates back to before CVS was widely deployed. The revision control logs now fill the same requirement, though you should have something documenting large user-visible changes.

Re:I disagree with some of these points

[preaction]

Most of the version control commit messages I've seen do not describe at all the reason and purpose of the changes. No, VCS are no substitute for changelogs.

Re:I disagree with some of these points

[TemporalBeing]

The one that I disagree with is 'Your source builds using something that isn't GNU Make [ +10 points of FAIL ]'. I disagree for two reasons. The first is that it implies using GNU make features, which likely means that you're conflating building and build configuration (which should gain some fail points). The projects that I most enjoy hacking on use CMake and Ninja for building by default (CMake can also emit POSIX Makefiles that GNU Make can use, but I take his point to mean that gmake is the only command you need to build, so the CMake dependency would be a problem). LLVM still more or less maintains two build systems, though the autoconf + gmake one is slowly being removed in favour of the CMake one. If I make a small change, it takes Ninja less time to rebuild it than it takes gmake to work out that it has nothing to do if I don't make any changes.

Agreed. Any cross-platform projects needs to be able to support multiple build systems, whether you like it or not. Tools like CMake/QMake/Qbs/Ninja/etc are just essential.

I'd also disagree with 'Your code doesn't have a changelog' - this is a GNU requirement, but one that dates back to before CVS was widely deployed. The revision control logs now fill the same requirement, though you should have something documenting large user-visible changes.

Agreed. You have version control. That's all you really need. ChangeLogs are generally outdated and for releases (where the VCS won't be available) a dump of the logs in some form should be sufficient, provided you are writing meaningful log messages. IOW, good VCS practices are a must.

Re:I disagree with some of these points

[Demonoid-Penguin]

Dear coward

What's wrong with open sourcing previously closed source projects?

I'd have thought that obvious. Licensing difficulties. The more difficulties a project has the greater the chances it won't succeed in the long term. (if only I could think of a simple analogy you could comprehend)

I guess linux fails since Linus wrote his own source control for it.

That's the problem with guessing. Git wasn't developed just for the kernel project. Fun fact: "own" has more than one meaning. Words can be tricky like that. e.g. linux, and kernel.
Even if the kernel (Linux) didn't use git - that's only 30 points of FAIL, just enough to make a baby cry.

Logic can be tricky as well. You seem to have conflated your premise with your conclusion. Some deductive logic might of been good (and a schema) - you seem to have leaped to a conclusion. A valid premise would not be "it's all wrong".

tl;dr your deductive argument is unsound. (and not deductive)

Re:I disagree with some of these points

[Demonoid-Penguin]

I'd also disagree with 'Your code doesn't have a changelog' - this is a GNU requirement, but one that dates back to before CVS was widely deployed. The revision control logs now fill the same requirement, though you should have something documenting large user-visible changes.

He's referring to another criteria of success - user adoption. Revision control logs don't fill the same requirement if a user wants to know what's new. That plays some part in determining whether users will upgrade. If they don't, then additional development is rather pointless.

Note that he only gives not having GNU Make a FAIL score of 10. Which seems about right - it has a small effect on the entrance level for developers and wide spread adoption of the project. It's a critical factor only if your project has accumulated a bunch of other fails.

Re:I disagree with some of these points

[TheRaven64]

If people can't write good commit messages, what makes you think that they'll write good commit messages in a separate document?

Re:I disagree with some of these points

[preaction]

They won't. The release manager has to. Right now, at work, that role is mine, so I, I have to.

Slashdot summary, as usual, misses the point

[Idarubicin]

If we're going to talk about Callaway's Points of Fail, and create a link in the Slashdot summary that *looks* like it takes you to that list, then perhaps there should actually be a link to the list.

Callaway's original Points of Fail blog post.

You know, instead of the usual Slashdot way of pointing to an article wrapper that talks briefly about some of the points and then eventually links to the real list.

Re:Slashdot summary, as usual, misses the point

[plopez]

That would require little timothy actually expend effort.

Re:Slashdot summary, as usual, misses the point

[gstoddart]

But it would spare us the badly written original article.

he thought he should jut quite his work

Re:Slashdot summary, as usual, misses the point

[Anonymous Coward]

Heaven forbid timothy actually demonstrate he has a brain.

Re:Slashdot summary, as usual, misses the point

[T.E.D.]

OTOH, my fascist firewall blocks blog posts such as Callaway's, so I really appreciate the hop through an unblocked source. I take it from context that article covers some stuff that isn't in the blog post, as well.

Re:Slashdot summary, as usual, misses the point

[Idarubicin]

OTOH, my fascist firewall blocks blog posts such as Callaway's, so I really appreciate the hop through an unblocked source. I take it from context that article covers some stuff that isn't in the blog post, as well.

You're thinking of this as an either-or situation, when it really isn't. Hyperlinks are cheap. There's no reason for the summary not to clearly say, e.g."Here is the original blog post in its entirety, and here is an article which discusses some points from the blog post along with some other stuff." If they can't even manage that, then the link should at least clearly indicate that it isn't to the content described in the summary.

Instead, the Slashdot summary fails to link to the original blog post and implies misleadingly that the link in the summary actually does do so.

Correct link to TRA

[Demonoid-Penguin]

His list, instead of the link to a blog with an article about the list. That blog post is interesting - though the picture of the author scratching is just weird. Was that taken at a lice convention?

Re:Correct link to TRA

[Anonymous Coward]

Livejournal? Have I traveled through time?!?

Re:Correct link to TRA

[matfud]

It was written in 2009

Re:Correct link to TRA

[TheRaven64]

An alarming number of those hold for Chromium and they all stem from one core issue: Google developers do not understand how to design APIs. A lot of the bundled projects could be entirely separate repositories and shipped as shared libraries if they did, but gratuitous API churn means that they have to keep copies of things like v8 and Skia for Chrome and build the whole thing at once. It's fine to do the aggregate build thing if you want LTO, but it should be a performance optimisation, not a requirement of the software engineering workflow.

Re:Correct link to TRA

[smooth wombat]

Google developers do not understand how to design APIs.

Wrong. Wrong. Wrong! Google developers get paid buttloads of money, more than you or I could hope to make. These are the elites, the 1% of the 1%.

Because they are so highly paid the problem cannot lie with them since we are repeatedly told if you pay developers what they're worth you will get excellent results. Just like paying CEOs of companies who go running to Uncle Sam to protect them from their own incompetence.

The problem must lie elsewhere. Look harder.

Re:Correct link to TRA

[Anonymous Coward]

Wrong. Wrong. Wrong! Google developers get paid buttloads of money, more than you or I could hope to make. These are the elites, the 1% of the 1%. Because they are so highly paid, they do not care...

The sad truthful version of the funny...

Re:Correct link to TRA

[Hognoxious]

Google developers do not understand how to design APIs.

Or UIs.

It's all halo effect. I think Google got search right (or less wrong than everyone else at the crucial time - just when AltaVista tried to turn into a portal and became utter poo) and have pretty much ridden that, with financial and reputational cross subsidies, ever since.

Sooner or later they'll run out of credit, one kind or the other.

Re:Correct link to TRA

[boomer_rehfield]

Thanks for pointing that out, I didn't even notice the date until you mentioned it.

Editors.

[Anonymous Coward]

Callaway said he likes challenges, but he felt buried by the project, and reached a point where he thought he should jut quite his work

Apparently editing the summaries is a challenge too.

We're NOT jerks! Main reason #1!

[Anonymous Coward]

This bug is critical, I am posting the issue for several reasons:

âWe're not jerks

https://github.com/dotnet/core...

Here's the list

[xxxJonBoyxxx]

>> 1) If your codebase is too big, it's going to limit who's going to be able to download your code.
>> 2) There is no good reason in 2015 for a FOSS project to not have public source control. This helps people contribute and determine the health of your project based on the date of the last commit.
>> 3) If your source control has no web viewer and/or no documentation, these two are obvious things to have
>> 4) Code that doesn't build is worse than no code! You need documentation on how to build the project from the source.
>> 5) Use build tools
>> 6) Bundling is not going not be maintainable. Bundling leads to forking.
>> 7) Forcing people to install only in a specific directory

My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub, TFS online or one of the popular choices most FOSS projects already seem to use. (e.g. How would an experienced developer get these problems in the first place?) #4-6 are entry-level build issues. #7 refers to a best practice (let people pick their install directory) that's been commonplace in the industry for at least 15 years.

I see he's employed by Red Hat. Does this list as news suggest that Red Hat's internal development processes are immature too?

Re:Here's the list

[plopez]

What depresses me bout software is how often we JUST DO NOT LEARN! Yes I am shouting. I am frustrated by the situation. Software development seems to be riddled with arrogant know nothings who think they can cut corners or reinvent the wheel because doing the right way isn't "7337".

Software Development is not an Engineering discipline by any means, at best it is a craft, because the hard lessons are not explicitly taught to newbies who are not evaluated on how well they follow those practices and tests them on them as part of a core knowledge base. Which is how real Engineering disciplines do it.

Re:Here's the list

[Anonymous Coward]

Teet?

Re:Here's the list

[Anonymous Coward]

I think the point here is doing a very basic list like the Joel Test:
http://www.joelonsoftware.com/...

He is not looking here in advice for seasoned developers.

Re:Here's the list

[Anonymous Coward]

#1-3 is solved by using GitHug

And he's pointing out why projects fail -- failure to take advantage of widely popular tools like github, and trying to roll your own, or distributing a source bundle via zip with no other supporting version control -- which, as somebody who does platform support for a company that customizes a Linux distro, a SHOCKING number of reasonably popular open source projects do -- is just stupid.

As far as code that doesn't build - ever hear of apache thrift? I've literally downloaded official releases that *do not build.* I've been a good doobie and submitted patches and bug reports back upstream, but Apache Thrift is a very popular project, and it's still tripping over a lot of these basic, basic, basic freebies.

And don't even get me started on the complete bustedness of their debian packaging scripts.

Re:Here's the list

[Anonymous Coward]

He has a bunch more further down too, including "requiring support for Microsoft Visual anything". Which is 100 points of fail.

If you ask me, not having support for Microsoft Visual anything is however 2,000 points of fail, and not being compilable on Windows at all without using some cross-compiler/Cygwin/MinGW is 5,000 points of fail.

Unfortunately that includes everything made by GNU.

Re:Here's the list

[Kjella]

Mainly he's confusing a project which uses an open source licence for a project that wants a community based development model. Google doesn't release the Chromium source so that they can get contributions, they do it to be open so that nobody can claim they're creating another proprietary web like IE did with their closed source, non-standard implementation. And that is all. I mean, he's talking about the source code to the world's most popular browser so it's obviously a very narrow definition of "failure", I doubt neither Google nor the users see it that way.

Re:Here's the list

[Anonymous Coward]

Software development is also riddled with arrogant know nothings who want to demand that their way is the "right way" and everybody has to follow their policies, coding standards, etc. etc. despite the fact they're largely just thrown together from personal preferences and biases and are just as unscientific as (while costing more than) not having policies at all. These people fight to be the ones to write the coding standards, then they have a nerdgasm over their newfound power. They're typically the less able programmers, too. The able ones don't have time for that bullshit.

There's a reason software isn't like civil engineering. When you can model your entire design on one piece of A4 paper and prove that it stays up using a handful of basic physical principles, shit's easy. If we did software like that we'd have really really elegant versions of "hello world" by now, and very little else.

Re:Here's the list

[phantomfive]

My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub, TFS online or one of the popular choices most FOSS projects already seem to use. (e.g. How would an experienced developer get these problems in the first place?)....I see he's employed by Red Hat. Does this list as news suggest that Red Hat's internal development processes are immature too?

He wrote the list based on things he'd seen in Chromium, so it's Google's problems. Here is the full list. Not surprising, since they used to jam all their code into a single repository.

(It's hard to fault them for a 100+MB source code download though, unless there's a lot of redundancy in the code).

Re:Here's the list

[Darinbob]

That's not the real list, follow the link to the link.
http://spot.livejournal.com/30...

Re:Here's the list

[slew]

Clearly you don't understand engineering. Engineering isn't just "model your entire design". Engineering is decomposing your problem into problems that are "spec-able". For example, build your bridge out of steel and bolts. You don't have a model of bolts in your design, you have a spec for bolts that you use in your design that is testable (performance and tolerance) and then you use parts hierarchically in your design. The bolt is designed separately and is made out of some alloy that has specs and is tested (performance and tolerance)...

The problem with most software isn't that it can't be modelling and rely on basic physical principles, it's that many projects fail to take specs and testing seriously, and the specs that exist don't address performance and tolerance (aka, error handling). If software did this, things would be more engineered.

Right now many software artifacts are similar to the prehistoric bridges that cross chasms in jungles in third world countries. They work, people cross them every day, but things were made empirically so nobody knows what might cause them to fail, so it's hard to rely on them.

It's not that bridges that were built 100 years ago were "better", but they were actually built to specs and of course survive to this day (which can't be said for the prehistorical variety). However, improved bridges are continually desired so we use better parts and build even better bridges today because modeling allows us to get tighter specs on the parts that make up bridges and the stresses that we are putting on those parts.

But doing all that requires better engineering discipline not dismissing it as a something that isn't applicable. Engineering is an useful approximation of the physics (an approximation which always gets improved over time), not a practice of physics.

Re:Here's the list

[Darinbob]

In the real world you can learn but you are most often prevented from putting the learning into practice. After 30 years programming, I still spend 95% of my time dealing with other people's code and "maintenance". The opportunity to do things the right way usually was passed years before I joined.

Actually real engineering disciplines screw up too and for some of the same reasons as software. Quick and dirty hacks in hardware driven by unrealistic deadlines from upper management, last minute bandaids applied because it's too expensive to redesign, confusing document control, lack of knowledge transfer if someone leaves the team, etc.

Also remember that the list is about open source software. Some of the things that are wrong there aren't wrong with proprietary software or internal tools, distributed teams have different requirements from teams that are all in the same aisle (ie, open source is greatly improved if there's a web based source code control browser, but that loses importance if everyone on the team already has a license of a the source code control system's GUI tool).

Open Source also means lots of college students or recent grads, full of excitement to get stuff done but without the real world experience to know how to go about it; and full of hobbyists who start off strong then slow down and stop; etc. For example, hobbyist has only a home PC with Windows, and for budget/time reasons does not want to bother with portability to hundreds of different types of systems; or the hobbyist is the sort who thinks Microsoft Visual Something is a really good state of the art tool and bases the entire project on that.

Re:Here's the list

[Anonymous Coward]

[...] riddled with arrogant know nothings who want to [...]

Grammar police here. That should say "arrogant know-nothings" with a hyphen.

Re:Here's the list

[Tablizer]

The problem with most software isn't that it can't be modelling and rely on basic physical principles, it's that many projects fail to take specs and testing seriously

Most requesters (users) don't really know what they want UNTIL they actually see something concrete, and then realize it didn't fit what they had in mind. We don't need engineering, we need mind-readers. If users had enough time to sit and be thoroughly interviewed about needs and preferences, they wouldn't need automation to begin with.

And further, how to make software maintainable in the longer run is highly disputed largely because it depends on "wetware" and unknowns, such as developer perception of code, and unknowable future domain changes.

It's more akin to writing technical documentation than to building a bridge: how do you write documentation that's clear to the audience, but flexible enough that it doesn't have to be largely reworked for every change.

There is no magic modularity formula: domain issues inherently intertwine (or can intertwine in the future even if not at original launch.) You can't hide intertwining, you have to find a way to manage it well.

Re:Here's the list

[Demonoid-Penguin]

[...] (It's hard to fault them for a 100+MB source code download though, unless there's a lot of redundancy in the code).

That extracts to over 3GB, so yes, there's a hell of a lot of redundancy.

Re:Here's the list

[Anonymous Coward]

>(It's hard to fault them for a 100+MB source code download though, unless there's a lot of redundancy in the code).

It's not if you remember back when full operating systems used less than 100MB. My browser needs more than an OS?

Re:Here's the list

[cowdung]

We don't need engineering, we need mind-readers. If users had enough time to sit and be thoroughly interviewed about needs and preferences, they wouldn't need automation to begin with.

And further, how to make software maintainable in the longer run is highly disputed largely because it depends on "wetware" and unknowns, such as developer perception of code, and unknowable future domain changes.

It's more akin to writing technical documentation than to building a bridge: how do you write documentation that's clear to the audience, but flexible enough that it doesn't have to be largely reworked for every change.

There is no magic modularity formula: domain issues inherently intertwine (or can intertwine in the future even if not at original launch.) You can't hide intertwining, you have to find a way to manage it well.

To deal with this what we do is go quickly to the UI.. once you show them they can give you better feedback. There's also some research supporting this.

http://www.construx.com/Though...

Re:Here's the list

[Tablizer]

Not really. They often have to use it with actual data to relate to what it's doing with it. The data flow matters.

Sounds inept.

[Anonymous Coward]

If your code depends on Microsoft Visual Anything (you get 100 points of fail)

Riiiiight. Like there are _totally_ no highly successful open source .NET or Windows native C/C++ projects out there.

We really need to get past this whole neckbeardy thing of thinking that using only 30 year old editors as your IDE is something real professionals actually do.

Re:Sounds inept.

[__aaclcg7560]

If you wanted to compile the source code for Python 2.7 on Windows, you needed Microsoft VS2008 to do so. You couldn't use VS2010 or later. VS2008 is hard to come by and difficult to work with.

Re:Sounds inept.

[Anonymous Coward]

Geez, I wonder what Callaway tells all the other Red Hat employees that work on GTK+ and GNOME to use instead of Visual C++ in order to create Windows version of those projects? That's a really stupid point.

Re:Sounds inept.

[__aaclcg7560]

Python might be a special case with two versions, 2.7 and 3.4, being available. Although people are encouraged to use the 3.4 moving forward, a lot of people are still using 2.7. The next version of Python, 3.5, will compile with VS2015.

Re:Sounds inept.

[Anonymous Coward]

Yes, Visual Studio 2008 is very hard to come by, especially in 2015.

Re:Sounds inept.

[__aaclcg7560]

Thanks for providing the links. Where were you when I needed help two years ago?

Re:Sounds inept.

[Anonymous Coward]

Working on his Google fu.

Re:Sounds inept.

[__aaclcg7560]

Obviously not on Windows.

Re:Sounds inept.

[Anonymous Coward]

Should not be that challenging to compile any version of Python with newer VS. Just because the binary builds used 2008 doesn't mean you couldn't crank out a build with a newer version.

Can I go on the record as saying how SHITTY Python 3 is, btw? I mean wtf? If your language requires all kinds of weird distinctions between unicode and non unicode strings such that it takes nontrivial work to make that shit work between modules and different calls, your language is shit. I don't have to deal with that nonsense nearly as much in .NET and it supports Unicode.

I'm sticking with Python 2.7.

Re:Sounds inept.

[__aaclcg7560]

Python and extensions have to compile to the same version of Visual Studio to work together. You need VS2008 for version 2.7 if you want to stay compatible with everyone else. A lot of people are still sticking to 2.7 for now.

meh

[superwiz]

He sounds like another text-only monkey. He is of the generation that thinks that code still needs to be written by hand. Basically, he is doesn't know why is incompetent and he is proud of it.

Re:meh

[Anonymous Coward]

I'm curious; why do you think that? Is it just the slam against "Microsoft Visual Anything"?
Note that he gives points of FAIL for configuring source "with a handwritten shell script" (+10), "editing flat text config files" (+20), and "by editing code header files manually" (+30).

Re:meh

[superwiz]

"editing flat text config files"

All while working for RedHat. RPM relies on shell scripts and doesn't have a reliable rollback/commit mechanism.

Is it just the slam against "Microsoft Visual Anything"?

But yeah, this obvious attempt at slamming business competition under the guise of technical know-how is oh, so 1995 (which was 20 years ago). But, in todays world, we have gotten to the point when it is not only easier, but more reliable to generate code than to write it by hand. And while they have some learning curve, visual code analysis tool are still better than text-only ones. Even the resurgence of C can be mostly attributed to the fact it's simpler syntax makes it easier to generate than the new C++ syntax.

Re:meh

[Anonymous Coward]

The "needs Visual to build" *was* a good point, up until Microsoft started pretty much giving away their tools. Why?

1) It's locked to a single platform;
2) It often required a very expensive tool, which is a huge limiting factor for open source adoption - very few people buy VS.NET, and just use what their company issues them a license for.

Now that Microsoft's opening up their tools, I think we're going to see a lot of really good open source work taking off in that space... but making those choices *does* limit who can and will take part in your community, and if you're trying to attract people to work for your project for free, then arbitrarily limiting your pool of potential contributors is not a good idea.

Re:meh

[Anonymous Coward]

It's like he doesn't know that there are a lot of open source projects that are cross-platform, including ones that other Red Hat employees work on like GTK+ and GNOME. Both rely on Visual C++ to compile for Windows.

Re:meh

[superwiz]

What do you mean "locked to a single platform". I admit that I haven't tried it, but they give away the source code to VS 2015. Which is pretty much why RedHat it trying to claim that code which used to be owned by a single company is a point of failure. It's another business swipe at MS. RedHat is running for the hills because pretty soon they'll lose all purpose.

Re:meh

[Jeremi]

What do you mean "locked to a single platform". I admit that I haven't tried it, but they give away the source code to VS 2015.

I don't think having access to the source code to VS 2015 is going to allow anyone to compile VS for any non-Windows platform. Not unless you have a few million man-hours available for porting and redesign (since much of the functionality present in VS wouldn't even make sense outside of Windows)

Re:meh

[superwiz]

You are right. It's not an open source project. All it does is open its source. C'mon. Bridging API frameworks is where "it's at" today.

Re:meh

[david_thornley]

What is this code generator you talk of? The only thing I can think of is a compiler for a higher-level language. This may be extremely useful, but it has to take some input, and that input has to have an unambiguous meaning that describes a problem somehow, and therefore that input is a program that was written by a programmer.

Re:meh

[Jeremi]

You are right. It's not an open source project. All it does is open its source.

I never said it wasn't open source. My comment was about whether or not it's "locked to a single platform". I assert that without a herculean porting effort, it effectively is.

C'mon. Bridging API frameworks is where "it's at" today.

I'll believe it when I see it. I guess getting it to run under WINE might be doable, but then again that was equally doable (at least in principle) when the source was closed.

There are LOTS of projects with these problems

[dwheeler]

"How would an experienced developer get these problems in the first place?"

A lot of projects do not follow widely-accepted best practices... even if they are experienced... and that is a problem!

A remarkable number of OSS projects fail to have a public source control system (#2). That includes many established projects that everyone depends on. Actually, a number of OSS projects - and projects that people THINK are OSS but are not (because they have no license) - fail many of these points. It's not that Red Hat's internal processes are immature; Tom was trying to bring in software from someone else (Google in this case) and was fed up by the poor practices from people who should know better.

Yes, #7 refers to a best practice (let people pick their install directory) that's been around for at least 20 years and probably much longer, but it's still widely NOT followed.

Anyway, that's Tom's point; there are a lot of widely-accepted best practices that are NOT followed, and that needs to change.

the usual suspects apply.

[nimbius]

As a software dev for closed source, our problems are creeping into open source at an alarming rate. Standups, Kanban, Scrum, swim lanes, and other political middle management bullshit is making it harder and harder to as theo de raadt once said, "shut up and hack."

The other issue is runaway devs. Gnome and KDE turned into piss pots almost overnight because they followed lockstep with whatever was trending. gnome grew hotspots that were clickable and draggable in an attempt to appeal to tablets, and KDE's widget framework turned into a swirling vortex of lights and colours that chewed through ram like none other. And the "fuck it lets move on" mentality has got to stop. Pottering epitomizes the swinging dick Linus so rightly kicked after his team was called out for set it and forget it code that ultimately broke more things and didnt play nice.

bottom line: dont lose focus in stability and function.

Re:the usual suspects apply.

[lessthanjakejohn]

hmm? In one paragraph you say 'shut up and hack' and in the next you criticize Poettering for doing exactly that?

Re:the usual suspects apply.

[preaction]

Standups, Kanban, Scrum, swim lanes, and other political middle management bullshit, but not for me. For all the _bad_ coders. Not me.

Re:the usual suspects apply.

[phantomfive]

hmm? In one paragraph you say 'shut up and hack' and in the next you criticize Poettering for doing exactly that?

Specifically, he criticized Poettering for not taking care of features he'd created.

Re:the usual suspects apply.

[Anonymous Coward]

Of course, we should start doing what needs to be done now, right, off the bat, hands down and no beating around the bush!

Every1 with me on this???!!?

Re:the usual suspects apply.

[david_thornley]

I don't consider myself a bad coder (and I don't think anybody else does, either), but I really liked Scrum when I had a chance to use it. It may be that we were attacking a reasonably small problem that was fairly well-understood up front, and it may be that we were actually doing a form of agile rather than just being told to write the code without any design.

Re:We're NOT jerks! Main reason #1!

[plopez]

I saw it on the Reg.

Not spongeworthy

[Virtucon]

I can't believe the billions of bits that died in the production and subsequent redistribution of this article. We're all dumber now for reading it.

I shall now drown my sorry in beer to see if I can't placate the inexorable struggle of those brain cells trying to make sense of this diarrhea and those others that are saying "WTF, Why did you read this?!? This is another let me share a bunch of drivel shit article. It's like Show and Tell in Kindergarten, just ignore it!"

Re:Not spongeworthy

[Anonymous Coward]

You use a lot of flowery words but your grammar sucks.
 
And if you're still drinking beer then you're an amateur.

Where's the video?

[robi5]

The summary starts with:

"At OSCON this year, Red Hat's Tom Callaway gave a talk [...]"

The summary has a link, which points to the article, which says:

"At OSCON this year, Red Hat's Tom Callaway gave a talk [...] My favorite part of this talk was Callaway's passion for the items on the list."

So where is the video. The list felt a bit bland so I also got the notion that the video would be more informative.

Because you need SCRUM!

[Anonymous Coward]

Your software project is failing because you're not using SCRUM!

SCRUM! will solve all your problem!

If you're not using SCRUM! you're DOOMED! DOOMED! I tell you!

bad quality

[Anonymous Coward]

Of course, the software project fails due to lack of quality assurance.

Like the Microsoft standard - end users are the QA team (send us the crash report, pls..)

funny I should see this right now

[Anonymous Coward]

I'm trying to set up a sharded mongodb cluster at work. Mongodb seems to be actively hostile to my finding *previous* manuals - I'm on CentOS 6, which prepackages 2.4, and all I've been able to find is the manual for 3.0.4, which does not do me a lot of good, given the changes they've made.

So, as the subject of the OP said about documentation, make sure folks can FIND PREVIOUS RELEASE documentation.

Oh, yes... and more broadly, DO NOT EVER make changes in a subrelease that breaks the previously running software (I remember python 10 years ago....), nor, if it's bundled, DO NOT include, in a subrelease update, a package updated a full release that breaks everything currently running (I'm looking at you, EPEL, with torque going from 2.5 to 4.2 a couple months ago....)

                  mark

Re:funny I should see this right now

[Tablizer]

mongodb? Use something that's been tried and true for at least 10 years. Go with MySql or PostGresql and screw the noSql toys until they mature and have decent docs.

Let pioneers take the arrows, the rest of us stay in proverbial Boston, which has infrastructure and seasoned specialists, and get shit done. And we have nice lawns to kick fanboys off of.

Project Management 101

[ajedgar]

This is a checklist from any decent software project management book. Here's a good one written over 40 years ago:
Mythical Man Month

And the issue of managing a large code base can be handled by a modern VCS system like Perforce that let's fetch and work with only the portion of the code base you're interested in.

Because you're reading slashdot

[Anonymous Coward]

Get back to coding! *whipcrack*

Linux gets +30 Points of FAIL

[ajedgar]

"You've written your own source control for this code [ +30 points of FAIL ]"

Re:Linux gets +30 Points of FAIL

[prefec2]

You get -60 points of FAIL if you can convince the rest of the world to use your new and shiny version control system.

He just forgot that.

Link to video

[DRichardHipp]

Here is a video of Tom giving roughly the same talk at the Southeastern Linuxfest in 2011. https://www.youtube.com/watch?...

What about market adoption aspects

[MyFirstNameIsPaul]

I wanted to run my own social networking site just for me and my friends using a FOSS project, so I was excited about Diaspora, then I saw that it requires Node.js. I have no interest in setting my server up for that. I imagine this selection was made because developers think Ruby is cool and PHP is boring and lame. Unfortunately, whatever the justification was, to make Diaspora work you need to have, you know, Diasporas, but if the only people using the project are those that manage their own Node.js server, then the already puny market size of available Diasporas has just shrunk by several orders of magnitude. It really needed to be a project that could be installed on any generic LAMP server, but the developers are so rarely interested in this boring aspect (this is actually the case across many engineering fields, it's why companies hire marketers) that left to manage their own projects they fail to achieve their stated goals.

So I took a look at GNU Social, which is written in PHP. Unfortunately, they also fail the marketing test. The project seemed to revolve around making a 'federated' social networking system. However, the actual features of the social networking seemed to be trumped by trying to make the federated system work. From a marketing perspective, they put the cart before the horse. How many users want a circa 2009 facebook clone? I bet a fairly high number, but GNU Social doesn't even offer that level of functionality. The 'federation' of the system should be viewed more as a distribution element, so, you know, before going to distribution, you should have a product that people want to distribute, and GNU Social is not that.

Re:What about market adoption aspects

[Anonymous Coward]

Um, dude ... Paul, Node.js has nothing to do with Ruby.

Just thought you should know.

Re:What about market adoption aspects

[MyFirstNameIsPaul]

Yeah, argh, lol... for some reason my brain interchanges Redis and Node.js, and I have no idea why it does that, other than they are two things (among many others) I've decided I don't have time to learn how to manage.

Clickbait

[wonkey_monkey]

Headline contains the word "your" and is therefore clickbait.

Mod parent up.

[khasim]

What depresses me bout software is how often we JUST DO NOT LEARN!

And not just software. Look at security as well. And so many other computer-related areas.

Software development seems to be riddled with arrogant know nothings who think they can cut corners or reinvent the wheel because doing the right way isn't "7337".

For me it's more like ... someone "learned" one way of handing it when s/he was working ALONE.

Then that person never learned that the practices need to be changed when you are part of a TEAM.

And releasing your code to the public is being part of a team.

Re: GitHub doesn't solve massive

[tomxor]

My first thought on reading this is that this guy started coding this year. #1-3 is solved by using GitHub

I struggled to find what in the list actually applied to Chromium - However the one thing that definitely applies to Chromium is the massive code base... Chromium source really is fucking massive (even if you just clone with depth=1) and it takes a fuck load of time to do your first compile.

I really don't see how using one git host over another is going to solve that, once the host has reasonable resources (pretty easy in this day and age) then the users connection is the bottle neck, and it makes the build process slow... this creates a significant barrier for contributors. I don't particularly like the idea of a massive code base, it's well known that more code == more bugs, and it just makes it harder to comprehend the whole thing... I don't really know if it's necessary from chromium or if it's just grown large with the speed of development, i'd like to know if anyone has some insight into this.

original list

[ihtoit]

this a slow news day? The list was crafted in 2009!

Humans

[Tablizer]

Software development seems to be riddled with arrogant know nothings who think they can cut corners or reinvent the wheel...

That's a problem with human nature, not just devs. We are not Vulcans. Humans are impatient, egotistical, fixate on the wrong factors, and often just plain random; and most don't know it or care.

I know some well-educated people who are complete idiots outside of their narrow specialty. I'm probably an idiot also in ways I don't even realize (please don't educate me in replies). My head-model of the world is perfectly logical and consistent to me, but it's probably highly lossy against the real world.

Gee, it's almost as if we are merely upright apes who happen to be able to talk and write. (I would have said "hairless", but I'm hairier than the orangutans I see in the zoo.) They fling poo, we fling nukes.

Linux kernel FAIL score.

[Luarvic]

== Size ==
* The source code is more than 100 MB. [ +5 points of FAIL ]
* If the source code also exceeds 100 MB when it is compressed [ +5 points of FAIL ]
    125372299 Jul 22 00:36 linux-4.1.3.tar.gz

== Source Control ==
* You've written your own source control for this code [ +30 points of FAIL ]
    git was written originally for Linux kernel!

== Building From Source ==
* Your source is configured with a handwritten shell script [ +10 points of FAIL ]
    Even worse: handwritten C program: scripts/kconfig/mconf that is compiled and run by "make menuconfig".

== Bundling ==
* Your source only comes with other code projects that it depends on [ +20 points of FAIL ]
* If you have modified those other bundled code bits [ +40 points of FAIL ]
    Compression and encryption libraries, for example.

== Libraries ==
* Your code only builds static libraries [ +20 points of FAIL ]
* Your source does not try to use system libraries if present [ +20 points of FAIL ]
    Not even glibc!

== Code Oddities ==
* Your code depends on specific compiler feature functionality [ +20 points of FAIL ]
    Lots of places where gcc-specific code is used.

== Licensing ==
* Your code does not have per-file licensing [ +10 points of FAIL ]

=== FAIL METER ===
Total: 180, highest possible:
135+ points of FAIL: So much fail, your code should have its own reality TV show.

What is OS X .zip format?

[Anonymous Coward]

* Your releases are only in .zip format [ +5 points of FAIL ]

* Your releases are only in OSX .zip format [ +10 points of FAIL ]

Pretty much off-topic: What is OSX .zip format, as opposed to .zip format? I don't think I have ever seen anything (software releases or other) that I'd know was a "special" OS X zip.

Re:Luarvic FAIL score.

[Demonoid-Penguin]

That list is well out of date. The most recent version is here.

== Size == * If the source code also exceeds 100 MB when it is compressed [ +5 points of FAIL ] 125372299 Jul 22 00:36 linux-4.1.3.tar.gz

FAIL. linux-4.1.3.tar.xz 22-Jul-2015 00:36 79M [ -5 points of FAIL]

== Source Control == * You've written your own source control for this code [ +30 points of FAIL ]

FAIL Like quoting the largest archive choice for the source code, you choose the wrong meaning of "own". That's not the meaning implicit in Tom's list - nor does it fit his reasoning (using an obscure versioning system limits use and development).
"There is no publicly available source control (e.g. cvs, svn, bzr, git)" . [ -30 points of FAIL]

== Building From Source == * Your source is configured with a handwritten shell script [ +10 points of FAIL ]

FAIL. It's not handwritten. It could be. There's difference. [ -10 points of FAIL]

Even worse: handwritten C program: scripts/kconfig/mconf.c that is compiled and run by "make menuconfig".

FAIL, most source code is handwritten. C source is not a script

== Libraries == * Your code only builds static libraries [ +20 points of FAIL ]

FAIL Demonstrably Linux supports dynamic libraries and can (often is) built without static libraries only. [ -20 points of FAIL]

* Your source does not try to use system libraries if present [ +20 points of FAIL ]

FAIL. Unless you can produce a citation that shows that Linux will try and use system libraries when the same libraries have been statically compiled (yes, even glibc). [ -20 points of FAIL]

== Code Oddities == * Your code depends on specific compiler feature functionality [ +20 points of FAIL ]

FAIL. If that were the case gcc would be the only possible compiler. It's demonstrably not. [ -20 points of FAIL]

== Licensing == * Your code does not have per-file licensing [ +10 points of FAIL ]

FAIL. Unless you've got a citation for that. I'm sure M$ would love to hear of this invalidation of the kernel license. [ -10 points of FAIL]

=== FAIL METER === Total: 180, highest possible: 135+ points of FAIL: So much fail, your code should have its own reality TV show.

Let me fix that for you:-
=== FAIL METER ===
20 points of FAIL
5-25 points of FAIL: You're probably doing okay, but you could be better. (um, maybe Linux is doing OK - one of two people use it)

Your score:-
=== FAIL METER ===
115 points of FAIL
95-130 points of FAIL: HONK HONK. THE FAILBOAT HAS ARRIVED!

Re:Luarvic FAIL score.

[Luarvic]

I should have put large <humour> ... </humour> tags around my comment.

I found quite ironic that a project that could have been _formally _ assigned so many fail points has not failed in any meaningful sense of fail.

Re:Luarvic FAIL score.

[Demonoid-Penguin]

I should have put large <humour> ... </humour> tags around my comment.

I found quite ironic that a project that could have been _formally _ assigned so many fail points has not failed in any meaningful sense of fail. If were to look at my posting history my reply was notably gentle.

I figured it twelve to a dozen on whether you were taking the piss (that's Oz-speak for good clean fun). Most of the fail points you gave had already been made in the comments on the old version of the list - including some truly brainless one's you missed (no documentation, no website, no bug tracker).

Humour tags would have just spoiled the fun. Gotta think of the larger audience. [smile]

For a good laugh at mega-TV-series FAILS - one of our consultants sent me a link to this. (I knew there was a reason we pay him the big bucks) We need a bigger FAIL index

Major omission in list

[MoarSauce123]

Nowhere in the list does QUALITY come into play aside from having a bug tracking system. Needs to add "No publicly transparent process in place for bug fixing [ +100 points of FAIL ]" and also "New features always take precedence over bug fixing [ +50 points of FAIL ]". I guess for both open and closed source software projects quality is still considered optional and the first thing to get crossed off the list. Sadly, users put up with that. I also wonder how the size limit is measured. Does that exclude code comments? I like code that has a lot of comments explaining in plain simple language what the next 3 to 5 lines of code do. It is much easier to decipher than trying to compile a programming language intended for machines in my head. Lastly, that list arrogantly assumes that there is no open source software created on Windows and for Windows. Even using closed source runtimes that are freely available and freely distributable are not a problem. In the end one big metric for success is always ROI. That applies to open or closed source applications. If it is a pain to install and use forget about it. So let me propose one more thing for the list: Not providing a commonly used binary package format for your software for the targeted platform (most users don't want to deal with make install and compiler output!) [ +50 points of FAIL ]