Avoiding Mistakes Can Be a Huge Mistake

theodp writes "No doubt many will nod knowingly as they read Paul Graham's The Other Half of 'Artists Ship', which delves into the downside of procedures developed by Big Companies to protect themselves against mistakes. Because every check you put on your programmers has a cost, Graham warns: 'And just as the greatest danger of being hard to sell to is not that you overpay but that the best suppliers won't even sell to you, the greatest danger of applying too many checks to your programmers is not that you'll make them unproductive, but that good programmers won't even want to work for you.' Sound familiar, anyone?"

Comments

Perhaps

[Thelasko]

Perhaps programmers that have consistently good code should have some value placed on them. We'll call it "Karma". Programmers with good Karma get audited less often than others. If they fail an audit, they loose some "Karma" and have to write a bunch of excellent code to get it back.

Re:Perhaps

[Anonymous Coward]

Code reviews teach the reviewers as much as they check on the author. Why would you deny the lesser programmers the joy and experience of looking at good code?

Re:Perhaps

[YourExperiment]

Do programmers also loose karma for being fast and lose with their spelling?

Re:Perhaps

[Anonymous Coward]

Do programmers also loose karma for being fast and lose with their spelling?

/irony

Re:Perhaps

[Pogue Mahone]

That's exactly the kind of check that is harmful, according to the article. Who determines what is "excellent"? Against what benchmark? Who performs the audits? Who checks that you have spelled "lose" correctly?

Re:Perhaps

[enjo13]

How do you identify "good code"? That's one of the great problems we have as software developers. Quantifying 'good' code is extraordinarily difficult. Code reviews do an excellent job of identifying clever code, but rarely capture the full utility of what is being written. You may think you know good code when you see it, but over the course of my career I've become convinced that is not true at all.

Really the problem is that the only way to truly measure code quality is by seeing how it runs in a production environment. Even then I can easily quantify the quality of the teams overall output (does it work? does it work consistently?), but tracing that back to an individual programmer is often nearly impossible. Systems tend to interact with each other, and placing blame is not an exact science. The gulf between 'good' and 'good enough' is not nearly as wide as it seemed when I was a novice programmer.

Great code almost never breaks. Good code works most of the time. Poor code is another matter.

Poor code is easy to spot. Poor code never works. It's ugly. It's complex. It's stateful. It's jump off of the screen and practically begs to be put out of its misery.

That's precisely why companies have processes and checks. They are an attempt to catch marginal code and make it 'good enough'. The problem, as the article points out, is that in the process they often inspire great coders to deliver marginal code themselves.

The secret is to spot (through some mixture of science and art) great programmers and provide them with the freedom to write great code. If circumstance requires you to hire marginal programmers, then by all means put the process in place to make sure that what they do doesn't detract from the work your best and brightest are doing. Separate them as best you can. Limit how their systems interact.

But whatever you do... don't limit your best programmers, as they are far more valuable than hundreds of poor ones.

Re:Perhaps

[ChromeAeonium]

Do programmers also loose karma for being fast and lose with their spelling?

/irony

They can be docked karma that way, but when they're not sure about something, they can cover their asses and submit anonymously. That way, if something totally whooshes over their heads, they're in the clear. They can later correct their own dumbass mistakes unanonymously and whore karma instead of losing it. What a perfect system!

Re:Perhaps

[syousef]

Perhaps programmers that have consistently good code should have some value placed on them. We'll call it "Karma". Programmers with good Karma get audited less often than others. If they fail an audit, they loose some "Karma" and have to write a bunch of excellent code to get it back.

That's awful in so many ways.

For starters look at how poorly Karma works here. It serves to re-enforce awful sheep mentality. Just try putting down Google, Apple or Linux. Or try praising Microsoft.

Next what you're proposing creates a negative feedback loop. A developer codes well and gets through a few audits. Now they're trusted, they can afford to let things slip for some time before anything is caught. There's less incentive to keep producing good code, and there's more of a chance that an error will slip through. No one is perfect and mistakes will happen. The way to protect against them is to ensure there's some redundancy, and taht is exactly what a code review provides.

Also consider retention rates and the average time a developer spends at a company. Does an expert or lead programmer start off having every little thing reviewed? Who's qualified to do that? Or are they trusted based on heresay and a resume? If so how long will it take to find a dud programmer?

Next consider what effect it will have on the morale of a struggling programmer, or one that doesn't cope well with reviews. Especially a junior one whose abilities can be salvaged. A co-operative might work, but constantly giving more and more high pressure code reviews is just about guaranteed to break such an individual.

Finally, you should realize that such "karma" already exists informally and that making it a more formal process achieves very little. In other words developers very quickly get a feel for what the strengths and weaknesses of another developer are.

Re:Perhaps

[sammyF70]

Google is evil incarnated, Apple is style-over-function overpriced junk, and "The Year of The Linux Desktop" ain't coming soon.

Microsoft, on the other hand, has some really funny employees (and Reversi).

/em ducks

Re:Perhaps

[ScrewMaster]

Do programmers also loose karma for being fast and lose with their spelling?

/irony

They can be docked karma that way, but when they're not sure about something, they can cover their asses and submit anonymously. That way, if something totally whooshes over their heads, they're in the clear. They can later correct their own dumbass mistakes unanonymously and whore karma instead of losing it. What a perfect system!

Huh. Too bad Slashdot doesn't have a system like that.

Re:Perhaps

[ifdef]

We have a more informal system where I work. Whenever anybody checks in some code, whoever wants to automatically gets an email to notify them (for example, I am set up to receive notifications for any changes in a few directories where I do most of my work). Anybody who wants to then reviews the change. If there is nothing to comment on, it's completely transparent, and the person who checked in the changes is not even aware that they got reviewed. If there IS something to comment on, most likely someone will talk to the original coder (or send them an email), saying something to the effect of "by the way, did you consider such-and-such?", or maybe even "good idea!"

The system keeps track of who reviewed what change. As a check on the process, if there is any change that has been in the system for x days but has not been reviewed by at least m developers and n testers, the original coder gets an automatic email saying "please arrange for someone to review your code."

This is, of course, in addition to the automatic emails that get sent if a change actually BREAKS something. Those get dealt with right away.

So the review process does no harm to anybody's morale, except in the cases where that person really is producing bad code.

Strangely enough, it seems to work quite well.

As for your other point, about "A developer codes well and gets through a few audits. Now they're trusted, they can afford to let things slip for some time before anything is caught. There's less incentive to keep producing good code": I don't want anybody with that attitude in my group, period. I am not in any kind of supervisory position, but if there is someone on the team who only produces good code because of some "incentive", and the team lead doesn't do anything about that, then I don't want to work for that team lead, either. I will vote with my feet, and I don't think I would be the only one. And then this would become, no doubt, a much more "normal" software department, instead of an amazing one.

Re:Perhaps

[kbrasee]

/irony

/woosh

/irony

Error: Cyclic Dependency

Re:Perhaps

[lgw]

Excellence can be determined in various ways, often through documentation, the great allergen of programmers. If you can't explain it, it isn't really done.

Suppose you have a programmer who's 10x as fast as your benchmark standard guy at producing tested, debugged code that meets requirements, but slower at writing docs. Do you make him write docs, or attach a junior programmer to him for that purpose?

Of course, somewhere up the ladder a software engineer's job becomes mostly writing docs (regardless of whether you go management or architect), but still - tying that anchor around the neck of your best dev is exactly the sort of thing Paul Graham was on about.

Re:Perhaps

[lgw]

My fundamental point is not that documenation and paperwork should be avoided, but that in some cases it's just a stupid allocation of resoruces to have certain developers do that work. In the old days, IBM would pair a tech writer with *every* programer, so that really good docs could be produced with a minimum of distraction to the coding. It's hard to know whether that produced the best code (though there's a lot to be proud of in 70s mainframe code), but it produced lots of *really* good documentation.

Re:Perhaps

[networkBoy]

wow, a Beowulf cluster of /irony

A fine balance

[pwnies]

Obviously you can't go too much in the other direction either. The checks are there for people who write code seen on http://www.thedailywtf.com/ who actually click the flashing banner that says they've just won lots, and for those who open up the .exe they found in the email that contains 'instructions on how to get a bigger pen15 2day'.
It's like anything else in life. The sins of one hurt everyone.
There will always be people who get shampoo in their eyes, and because of that these checks will always exist.

More checks!

[I.M.O.G.]

From the article:

Whenever someone in an organization proposes to add a new check, they should have to explain not just the benefit but the cost. No matter how bad a job they did of analyzing it, this meta-check would at least remind everyone there had to be a cost, and send them looking for it.

So bureaucracy has a cost in that it places lots of checks on things, and the solution to that is adding more checks?

Sounds like solid bureaucracy to me!

Death March

[micromuncher]

I find the premise of the essay wrong. Go read "Death March" by Ed Yourdon (http://www.yourdonreport.com/). Most of the time problems aren't processes - they're people.

"Programmers, though, like it better when they write more code. Or more precisely, when they release more code. Programmers like to make a difference. Good ones, anyway."

This is a red flag. Coders that just code are part of the problem of a death March. Who has worked with wunderkind that churns out 16 hours of useless bug ridden code? That refuses to write unit tests because they slow him down? And at the end of the project look back, the MetricsReloaded tells you "Yes, that developer wrote the most code, but it is also the most defective and has the least coverage?"

Good processes are adaptive. Good people are agile. You can't build skyscrapers on spec. I am so annoyed by people that push a methodology or ideology that cannot also cite the specific historical evolution of software processes.

Agreed, Too Much Oversight Kills

[cgifool]

My group is a prime example. We all worked for a startup that generally released a new version of our application about 3 times per year. Over a few years we had developed a nice lean development process that involved documenting our design, but only in enough detail to be able to fairly accurately estimate the development effort (in X days, X weeks, or X months).

Based on the estimates, the biz dev group would then pick and choose features to make up 3 months dev + test time.

This worked great, and we pretty much never had a late shipment and few bugs.

Then we got acquired by a giant 3-letter company with huge amounts of development process and tons and tons of "standards", and immediately were ordered to begin a 16 month release consisting of removing all open source and complying with standards. All their architects routinely veto our decisions and our design documents must be very very detailed and approved via heavyweight process before implementation can begin. 24 months later we're still in development, only recently the last design document was finally approved; at the moment it seems we'll be about 12 months late in total.

Now they're asking us why we have so many tests planned, and making us remove half of them. Supposedly quality is a major priority, but they have no testing group; only people to enforce standards. All tests and test cases are written and implemented by the developers themselves.

Dont even get me started about the outsourcing issues.

As a programmer I want to avoid mistakes...

[syousef]

What I don't want is:

- To be reprimanded for every little mistake I make, or worse be put in a position where a little mistake on my part can cause a huge, expensive and/or very visible problem

- To be forced to comply with procedures that do not in fact improve quality but do require 90% of my time leaving me with 10% time to program

- To have no creative input into my code.

There are good ways to achieve similar goals without the above antics. Continuous integration comes to mind. Well qualified specialist testers for User Acceptance is good too. Avoiding mistakes in a way that is programmer friendly will actually improve morale and an employer more desirable. The trouble is that too many employers try to equate software manufacture with mass production factory work in every way and treat their programmers accordingly. If you look at the kind of work a programmer enjoys vs the kind a factory worker is expected to do, no wonder they leave or won't join.

Nothing changes. Really.

[onescomplement]

Paul, as usual, backs into the key argument.

This keeps coming up in various shapes and forms but the fact of the matter is that brilliant, high producers aggregate in places; and so do idiots.

Tom DeMarco ran a study of this in the 80s wherein teams were asked to solve the same problem. He expected a scatter-plot. It was a 45 degree line between the people who knocked the problem off and those who were clueless.

What didn't matter:

Platform. Language (except assembler, those folks were _lost_.) Operating System.

What did matter:

Team coherence and capability.

Design and planning; raw ability to design and plan as a coherent team. And not just a bunch of losers following a Pythonesque "Book of Common Knowledge."

(I have been to many "Does the witch weigh less than wood" meetings...)

Look at the back cover of Boehm's "Software Engineering Economics." What he _measured_ was that team capability overarchs everything. Period.

I would also ask you to look at the surface exposure of development. Folks who develop on the shoulders of many giants can and should be trying lots of stuff, because that's why platforms are built.

Folks working closer to the core (the OS, drivers, fundamental code) don't change as quickly, nor should they.

I've worked as a hatmaker (sheer, unbridled creativity with fancy ribbons and flowers and such) for high-end ladies and I've sat, confounded by bad documentation for UARTS.

Two different regimes.

Re:I have mixed feelings.

[russotto]

At the big company I worked at, someone added some features to code that I wrote. It broke my code. I wanted to go in and fix it. Why not? I knew how it worked. I couldn't without a defect written by a tester.

Annoying, but easy to handle. Testers love to find bugs, whether it be for the joy of crushing a young programmer's spirit, or for the look of fear in the eyes of the product manager as the release date approaches. Point a tester towards the bug, and he'll go right ahead and write a defect, possibly cackling evilly as he does so.