Alan Cox on Writing Better Software

Andy Gimblett writes "Alan Cox recently gave a talk in which he discussed some current and emerging techniques for producing higher quality software. Some of these will be familiar to Slashdot readers, such as validation tools, type checking, etc, but others seem heavily influenced by his recent MBA. In particular, he has a lot to say about Quality Assurance in the software world, and the kinds of things we should be doing (and some people are doing) to make better software. Story and lots of quotes at Ping Wales, and video at IT Wales."

Comments

Quality

[mfh]

he has a lot to say about Quality Assurance in the software world

Quality Assurance in 4 easy steps!

Dear Managers,

1. Listen
2. Close your mouth
3. Plan everything around #1
4. Profit!!! (notice there is no line with ??? because you listened!)

Re:Quality

[mfh]

And what then when the customer wants some hard evidence about what has been done? What then? What will you provide to them? How does your irrational hatred of managers actually assure quality in concrete steps?

A good manager does all the detail work and keeps track of everything. My post was about what they haven't been doing much of lately.

I don't hate managers, and to suspect that I might by reading my previous post means you failed to understand it.

Quality in business comes from many different people in any given project. Software quality can be adversely affected by morale, and you guessed it -- programmers have some of the lowest morales in the business world today; they are often not respected by management, they have the highest workloads, and they have no time for social lives.

If managers listened, and remained silent or inquisitive instead of arbitrary or antagonistic, software quality would be up.

Re:Quality

[acvh]

"no time for social lives"? like they would know what to do if they did?

but seriously, good managers manage. Bad managers threaten, cajole, bribe or whine. software is either a product, which means that management is monitoring to be sure the product is what they can sell; or software is a tool, in which case the manager must ensure that the tool works.

either way, successful software is a combination of good programming and good management.

Re:Quality

[angel'o'sphere]

The human issue is just a really small part of the software quality.
Trust me. All problems in software engineering are human issues.
It helps nothing if a guy in the team is a hero or super hero or the brightest expert if the human factros are a mess.
All problems in software projects are simple:
1) lack of understanding -- people do not understand the problem domain
2) lack of insight -- people do not realize they suffer from (1)
3) lack of problem aware ness -- people do not realize that there ARE problems and that they have to SOLVE them
4) blaming other people -- as soon as it gets obvious that (1) to (3) is happening, a solution is not searched but a guy to shift the blame on

This are the core problems. All so called Software Process or Software Engineering tools and habits only soften those problems. A issue tracking tool, e.g. makes it harder to shift blame. As the author of the issue, the assigner and the asignee are known. Same for version control.

As soon as the simplest tools and the simplest habbits get established, people start working together. Working together or not working together. Thats the big point.

How good a certain guy at certain technology is, thats less important. That guy can learn or can be placed at a different work area. If you have such a guy you made your first mistake anyway allready. That again was a human issue (he tricked you in believing he is good, or you misjudged). So cope with it buy educating him or fix it by replacing him.

It's fashionable to bash managers And rightfull so!
For a software project you need no managers! At least not a manager in the traditional sense. A managers job is to get the staffing and the budget done right. Furher he has to be able to understand the development speed/quality of the team and communicate the teams results to the budget owners or customers. Most managers don't do that but make technical descissions about stuff they have no clue about! E.G. my latest project we used a JDO engine (of a so called market leader lol, well the simple piece of software costed Euro 500). We had hughe problems in certain areas of the software and spend about Euro 50.000 in internal development costs in working around those problems.
The team wanted to switch to Hibernate. The manager descided: NO! This is in spite the fact that we made a thouroughfull investigation about costs and benefits and showed we start saving money after 3 weeks and would have speeded up the development process by spending less time in DB mapping etc.
Thats just a simple example. So, we had a human issue again! The coders failed to communicate! The manager failed to understand!
The project was close to run overtime, because of that issue. It was over budget because of that issue.

Nevertheless Alan Cox is right in his analysis. Better coding habits and better strategies and better tools and better languages of course have an impact. The limit however are allways the humans.

angel'o'sphere

Re:Quality

[penglust]

What you say is mostly true. Managers often get on my nerves too. Especially at crunch time. I also know a few developers that actually expect the code they write will never fail.

What ever the case, good management will know when to get involved and when not to. There are always signs of problems. As an example: Many years ago I worked at a company that was second marketing several different Unix machines under their own name. We did a lot of work to make them as compatable as possible to give customers a real choise. A whole group of tests were thrown together to try and test this. At any time over 80 percent produced some kind of failure messages on all or some of the different platforms. This was in the late 80's and 2 consultants were brought in to look at the test and failures. They were supposed to fix the tests where needed and report on real problems. 3 or 4 months later, on a saturday, I was trying to use the tests to validate some compatability changes I had made. They all passed. A couple of months before nothing had passed. I expected some success but not complete by a long shot. Every test I needed had simply been commented out in the latest snapshot of the tests. They had done nothing at all. When confronted they had no excuse but "we are still identifing tests that fail". They were gone about 2 hours later.

Point being, I think most quality failures are from both managment and development being secretive, self serving and practicing a lot of CYA. I guess these are the reasons why I like OSS. Secretive and CYA has a very hard time occuring and self serving comes about because the programmer gets reconized for both managing and developing on a project far above industry standard quality levels.

If you want an example of managment gone bad take a look at ISO 9000 time sinks.

Re:Quality

[jc42]

1. Listen
2. Close your mouth ...

One problem with this: By listening with your mouth closed, you run a very real risk of misunderstanding what you're hearing. Without feedback to verify that you're understanding, you are highly likely to do things completely wrong.

Funny example: A few years ago, I was implementing an interactive web site, and we had a nice test server machine in the lab. In one discussion with The Mgt, I casually mentioned that for testing, I thought we needed to run another server.

I didn't hear any response, so I went ahead and cloned the apache server that we were using, and fired it up on a different port. 10 minutes work, and we proceeded to test.

A while later, I discovered just in time that the managers had heard me say that we needed a second server machine, and were ordering another one. After a bit of discussion, I realized that to them, a server is a piece of hardware, while to us software guys, it is a piece of software.

I managed to explain to them that, no, we didn't need a new machine. I'd already set up a second web server on the old machine, and it was working fine. There was still time to cancel the hardware order.

But still, they had done a bunch of unnecessary work, and had almost spent a good sum of money unnecessarily, all because they had listened carefully without asking questions. If they had asked what hardware the new server needed, I'd have realized quickly that there was a misunderstanding, and we could all have saved a bit of time.

Listening without interacting, and acting on your understanding of what you heard, can lead to a lot of serious problems.

Re:Quality

[fyngyrz]

Quality Assurance in 6 Easy Steps:

1. Fire the managers
2. Fire the beancounters
   (steps 1 and 2 may be reversed if required)
3. Fire anyone who even mentions the word "committee"
4. Keep labs open 24/365. Apply metered, but 100% flexible, work hour rules for programmers (ie, "You have to work 40 hours/week, but it doesn't matter which 40 hours.)
5. Provide superb comfort and amenities to programmers
6. Create significant bonus program tied directly to number of problems found in each programmer's code per release cycle. Zero problems, max bonus, more problems, less or no bonus. Stir to taste.

Enjoy good results.

Re:Quality

[penglust]

Been there, done that. I could not agree more. But I have had a number of good managers in the past. Did a stint as manager myself at one time. I backed off because, like you, I just plain like to be productive and I think like an engineer. This may it difficult to deal with those above me.

My direct manager right now is acutally one of the best I have seen. The other 3 or 4 layers above are lost and clueless most of the time. Their buzz word at the moment is ISO 9000.

I think your comments bring up another issue. F**k up move up is often not just a saying. I have made friends and lost friends in the past for my opinions but if one of my fellow programmers is just plain incompetent, lazy, or an ass hole then they need to go. I don;t encorage moving them to get them out of my hair. I have done this before and somehow I always ended up having to deal with them again.

There is also the type that are just biding their engineering time till they can become a manager. I have generally found these guys to be less than worthless. They are usually garanteed to make the worst managers.

These are the criteria I usually find good in a manager and tried to use as one:

1. Have a good broad software background. If working at the kernel level (including driver) have a basic understanding of hardware

2. Understand how software impacts the potential customers and their needs. Books have been written on this, most of them wrong.

3. Know your employees and their stengths and weaknesses. Use this to build a team that can and will perform.

4. Trust the team but keep track of everything anyways.

5. Process is a must but it should be pretty lightweight. Make it enforcable and do not waiver in ensuring the necessary pieces occur.

6. Insulate your engineers from most of the stuff above. Not all, just most of it. Usually controlling the flow will suffice (often this involves rumor squashing).

OK enough for now.

Unit testing?

[JohnGrahamCumming]

It's quite surprising to me that there's no mention here of unit testing, or any sort of automated testing where the engineers who wrote the code actually write tests.

It should, by now, be clear that "code that doesn't have tests, doesn't work", and that if XP has done anything for us, it's to focus on writing tests. I've seen this in action and it works.

John.

Re:Unit testing?

[bloggins02]

Unit testing is essential, but it's not a panacea. In particular, beware of two pitfalls:

1) "The unit tests passed, so it works." This assumption is flawed on several levels. First, and most fundamentally, even if all unit tests pass, there is still the issue of whether your software works as a whole. Software often has "emergent logic" and UI scenarios that are difficult or impossible to test (after all, that's not what unit testing is for, but some people seem to think it is).

Second, just because a test passes, doesn't always mean the API works. This is especially important if you didn't write the tests yourself. Just because a unit test CLAIMS it tests X, doesn't mean it does. Is the test complete? Any false positives? Is the test just a skeleton that was intended to be implemented later but never was? I've had all these bite me in the past.

2) "That particular test has NEVER passed, so there's something wrong with the test. We just ignore it now." Bzzzt! Wrong! There's a REASON it never passed. It's either not implemented properly, just a stub that fails waiting for someone to write an implementation, or maybe you just think the feature it tests actually works. Look closer. The test might be trying to tell you something.

If you are careful with unit tests, they can be very rewarding and useful (especially for regression testing, where they are invaluable), but put too much confidence in them or depend on them to do the kind of overall testing they were never designed to do, and you will fail long before your first test does.

Re:Unit testing?

[cerberusss]

What people should realize, is that it can cost quite a bit of effort to build a unittest. I.e. if you write some code that reads several rows from a database and as a result writes a bunch of rows in another table, you need to run scripts before and after your unittest. It's of course all doable, it's just that it's sometimes more work than writing the method itself.

Re:Unit testing?

[Unkle]

But that's really the reason it's not done all that often. Developers think that the unit test will be a waste of time. The problem is, nobody codes perfectly. Finding a bug during unit testing is much better than finding it in design validation testing. Indeed, on many projects I have worked on, issues that could have been found with adequate unit testing were not found until after release.

Unfortunately, when schedules get tight, it's things like unit testing (and testing in general) that get cut. The more emphasis we get on the importance of QA the better our industry will be.

Re:Unit testing?

[Anonymous Coward]

Study Mock objects

Re:Unit testing?

[Anonymous Coward]

Anyone ever notice DO-254 (That pesky avionics software development guideline) never mentions unit testing? There's a reason for that.

A solid requirements based development process and requirements based testing/verification process is the key to large high quality software. In my opinion, formalized unit testing tends to hide errant system level behavior. Sure, it aids an individual developer in understanding their code works as they intended, and should stay a vital part of low level development. But I've come to realize, it just doesn't help to reduce big errors in system level design.

Re:Unit testing?

[OP_Boot]

the engineers who wrote the code actually write tests

Do NOT get the engineer who wrote the code to also write the test.

It's fairly fundamental - the engineer who wrote it will have a prejudiced view of what should/will work.

Get someone else to do it and get a valuable fresh insight.

Re:Unit testing?

[mark_lybarger]

you only need to validate the functionality in the method itself is working. from your sig it seems you're using java. junit with an embedded axion database allows for extremely easy testing of a method that uses a database connection for its work. of course, you have to setup your test by creating a table, and inserting any base data. getting that basic framework is as simple as writing the method itself. you'll probably double your time in writing the actual method. i'd think the return on that investment is "priceless".

Re:Unit testing?

[rumblin'rabbit]

I disagree.

First, it is not necessary for unit tests to be absolutely complete to be useful. Anything that finds a hitherto-undiscovered flaw is valuable - extremely valuable.

Second, that there are bugs in a unit test is not, in practise, that big of a deal. At worst it means is you haven't tested a unit as thoroughly as you think you have. Unfortunate, but not a disaster. Perhaps two people should write their own units tests for a single module, and then compare the bugs they found in the module. An interesting experiment I think.

My experience with unit testing is that it greatly speeds up the development process. Trying to get a half dozen classes all up and working simultaneously is an arduous task that can take god-knows-how-long. Trying to get one class up and working through unit tests is generally straightforward and predictable.

And the quality of the component goes up. Waaaaaay up. It is rare when I write a unit test and don't catch at least one bug, thus easily paying for itself.

It is not a case of unit testing versus reviewing. It should be a case of unit testing and reviewing.

Re:Unit testing?

[richieb]

A solid requirements based development process and requirements based testing/verification process is the key to large high quality software.

You tacitly assume that it is possible to get solid requirements. When writing avionics software, I'm sure you can, because the problem is well understood and we have good science/math to back it up (eg. GPS nav system).

But in most sofware projects it is impossible to create requirement ahead of time, mostly because the problem you are trying to solve is new and we don't understand it well enough yet.

Are there requirements for the web browser? Were they created before the code was written? WHat about requirements for MS Word?

Re:Unit testing?

[dubl-u]

But I've come to realize, it [unit testing] just doesn't help to reduce big errors in system level design.

That's like saying that street maps don't tell you what the continent looks like. It's technically true, but it seems to miss the point.

Unit tests are for testing relatively small chunks of work. If you want to be sure the pieces work together, you do testing at higher levels. I think both are necessary for a solid system.

Personally, I think of my high-level tests as executable requirements. Every time I check in code, a server makes sure that all our tests, both unit and end-to-end, run happily. That's a big step up from the traditional requirements phone book that most people don't use except as an aid to yelling at someone.

2 words

[otisg]

unit tests

not a panacea, but it does go far.

Re:2 words

[ceeam]

That's nice thing when you do your project bottom-to-top, i.e. when you have some engine tossing around your data and stuff, and then write UI on top of it. (UI is not very unit-testing-friendly, is it?)

Unfortunately, IMHO, most of software is done as this - let's put pretty gadgets around, cool nice icons, and then we'll do the job in event handlers somehow. Here it all breaks loose.

Why it is done that way is easy to see - managers/supervisors are not interested in you doing smth behind-the-scenes for weeks/months without showing "cool stuff" to them.

Re:2 words

[Unkle]

I actually had a coworker marked down on his yearly review last year for wanting to write re-usable code. Our manager's (very VERY flawed) opinion was that, though it might be nice to have the re-usable code, just write it for this specific task because it's just easy and fast.

The kicker is, this year that same manager wants to re-use the code that my coworker was origionally going to write.

Re:2 words

[johannesg]

Ha, that's nothing. Long ago I used to work for a company that thought it was a good idea to NEVER reuse code because then you would reuse all the bugs as well. OTOH, they reasoned, if you wrote it from scratch you wouldn't be copying old bugs, thus this was a safer thing to do.

I'll leave the results as an exercise for the reader...

Re:2 words

[sootman]

And for those who don't like exercise, I suggest Joel. Samples:

"The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they've been fixed. There's nothing wrong with it. It doesn't acquire bugs just by sitting around on your hard drive. Au contraire, baby!... it has grown little hairs and stuff on it and nobody knows why. Well, I'll tell you why: those are bug fixes.

One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.

Each of these bugs took weeks of real-world usage before they were found. The programmer might have spent a couple of days reproducing the bug in the lab and fixing it. If it's like a lot of bugs, the fix might be one line of code, or it might even be a couple of characters, but a lot of work and time went into those two characters. When you throw away code and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work."

"free" ?

[mirko]

I see that the word "Free" doesn't appear in this story's synopsys.
How would Alan apply his quality methods to projects which member might never meet due to geographical contingencies ?

Good code...

[TrollBridge]

...isn't just for end-users! If you anticipate that others will be working on future versions/releases of your software, good commenting can make the job SOOOOO much easier for the next codemonkey.

I'd say commenting is doubly important in OSS projects, as it involves many sets of eyes trying to comprehend what you coded.

Re:Good code...

[Dr. Evil]

Commenting must be 100% accurate else it is detrimental to understanding the code.

Sometimes code changes don't result in updated comments...

Once I find an inaccurate comment in somebody's code, I have to start rewriting or deleting all the comments because I can't trust them anymore.

Re:Good code...

[Megasphaera Elsdenii]

Mod parent comment (pun intended) up ... comments are evil. Code should
first and foremost be self-documenting, by the
choice of proper variable names and subroutine
names. Only comment things that are not obvious,
like tricks that are employed. The point is that
comments are not read nearly as much as names,
and get stale more quickly, rendering them useless.

Re:Good code...

[johannesg]

That, and good naming of variables / functions / classes. To clear up any possible confusion: that means that the name has some bearing on what the thing in question is doing for you...

Recently I had the misfortune to wade through a few hundred kilobytes of Java that was written by someone who thought he should 'abstract' everything as much as was humanly possible. Sounds good, right? Well... It turns you can do a lot of harm that way, too. I don't think he had a single function in there that _wasn't_ called something like SetProperty(), GetValue(), DoFunction(), etc. There was absolutely no way to guess what it was doing based on the name of the functions. Naming of classes and variables wasn't much better. After looking at it for a couple of hours I don't think I could have guessed what it was trying to do if I hadn't already known that beforehand.

So, next time you are writing software, feel free to get in touch with reality and name stuff after what it is supposed to be doing. Nice long names please, no abbreviations unless you go over 30 or 40 characters. Down with CmtPmt2Db(), down with SCUPD(), and down with GetPropertyValueInterfaceCaller()!

Because, be honest: those mean nothing, while CommitPaymentToDatabase(), ScreenUpdate(), and GetXLocation(), have intuitive meanings we all understand...

Re:Good code...

[gidds]

Comments tend to state the obvious and not tell you what you really want to know.

This is a reason to write better comments, not to avoid them completely!

Of course comments that are flippin' obvious, or wrong, do no good to anyone. But some sorts of comments can be incredibly helpful:

• Big-picture comments. Explaining what this function, method, class, file, or interface is doing; how it fits into the app or system; important design decisions. Also authorship and date information, if that's important.
• API comments. Describing the implicit (or explicit!) contract that a function, method, class or whatever has with its caller. When you should be calling this code, what the caller needs to have set up beforehand, what the code guarantees to do in return, any gotchas.
• Section comments. Within long functions, just a few words every 10-30 lines, breaking it up into logical sections, can make it much easier to understand the flow, or find the particular section you're looking for.
• Here-be-dragons comments. Describing tricks, shortcuts and other hacks. Better to make the code clear and obvious, of course, but there are times when a clever technique or bit of magic is needed, and for those rare occasions, a few words of explanation can make things so much clearer -- and prevent someone ignorantly 'fixing' it later!

Of course, there's some overlap. But these are the sorts of things that can make code so much easier to read, and don't take much effort to maintain. The big-picture and API comments tend to be common practice in literate languages like Java (JavaDoc) and Perl (POD), but too many people don't follow it -- and even where those comments aren't extracted automatically, they can still be incredibly useful.

My rule of thumb is: Try to make the code itself obvious. And comment everything else!

Re:Good code...

[ajs318]

You see, that's the point. There's no point being too clever -- you can use the programming language itself as a sort of "generality-of-purpose abstraction layer". After all, if a language is computationally complete then as a matter of definition, any task you might want to perform can be implemented using that language. There is no shame in writing a program to do just the thing it has to do. It's far better to be able to play one song all the way through from start to finish with no mistakes, than any number of fancy riffs and bits of verses.

Code that you've written once for one specific purpose can be picked apart and made to do something different later if you ever need to -- and you might not. Just keep the thought at the back of your mind that you may need to make something a little more general-purpose, so as to make sure you will be able to do it when the time comes. That will mean intelligent use of comments and choice of variable names. If you're following KISS principles anyway, it should be almost obvious what something is doing.

I once wrote a function to send a MIME-formatted e-mail with exactly two attachments, and did not feel even slightly guilty about it; because that was what it had to do. Sure, later I did need to be able to send three attachments, or just one; but it was much easier to extend something that had already proved itself sending two attachments, than to write something from the outset that could handle an arbitrary number of attachments. The bits that needed to be altered to use loops and arrays stood out much more clearly once I was seeing them hard-coded with separate statements and separate variables.

Extraneous generality of purpose is really just a form of cruft. It just makes your code harder to write {because you have to consider other things beyond the problem you were originally considering} and harder to test {because you have inevitably introduced more ways to break it}.

Re:Good code...

[iabervon]

Comments are valuable at a relatively high level, where you describe what the code is supposed to do. The code itself documents what it actually does, but there's no way for a person reading it later to determine if it is doing what it is supposed to do without a comment that explains the intent. For example: "// Long divide in place, junking the data, but leaving the reminder" is very useful to the person who comes along later and doesn't know that the code actually wants this odd result, and wouldn't have a chance of figuring it out if the code didn't compute it correctly due to a bug.

Alan Cox

[Anonymous Coward]

Who cares about what Alan Cox has to say? He's soooo 2.4

Code review and pair programming

[Frans Faase]

The most effective techniques for finding defects is still code review. It seems that one of the pair programming is a very good way of doing code review.

However the greatest problem with writing good software is still in the marketing. In order to sell/license software it needs to have features, and the lack of defect often does not count as "features".

Re:Code review and pair programming

[ZorbaTHut]

Funny, that. I recently started working at a company with mandatory code reviews. Here's a list of my recent experiences with it:

1) Checked in code. Spent fifteen minutes justifying design decisions. No changes made.

2) Checked in code. Code contained horrible horrible bug. Code reviewer didn't see it.

3) Checked in code. Defended my design against several more computationally expensive suggestions that were also more complicated. No changes made.

4) Listened to a friend gripe about having to spend a DAY AND A HALF repeating design reasons and fixing bugs introduced by his code reviewer "cleaning up" his code.

5) Received company-wide email about a build that flat-out didn't compile - apparently someone hadn't bothered compiling a patch, and had sent it to a code reviewer, who likewise hadn't bothered compiling it before authorizing it.

Now I'll admit that there are also a whole lot of "well, it only took five minutes, so it wasn't much of a waste" cases. But so far I haven't heard one person talking about how useful the mandatory code reviews are.

Maybe it's just an artifact of the kind of programmers working at this company, or the kind of code being worked on, but so far code reviews have been a net loss in my experience. I've taken to doing major changes in my own personal branch of the repository (which doesn't enforce the code-review requirement) and in a month or two I'll have 3000 lines of code for someone to look at - but at least I won't have nickel-and-dimed them to death with 120 100-line code reviews, 3/4 of which I will inevitably end up deleting entirely.

Note that I'm not saying "code reviews are bad" - what I am saying is that there's a time and a place for just about every technique, and there's also a time and a place where each technique is worse than useless. Pick your battles and pick your tools.

Re:Code review and pair programming

[Unkle]

Sounds like your company has a bad process. Having a formal code review after each check in is crazy. Reviewing code when a task is declared completed makes more sense, or even doing regularly scheduled reviews.

Our company has some loose rules (we're working on strengthening them) that state that checked in code must be unit tested. This is to prevent things like your #5. But we haven't gotten to code reviews yet. Being on the team that's working on our process, I'll remember your experiences when we get to code reviews.

Re:Code review and pair programming

[Alan Cox]

If you are bug hunting then the code review needs to be an explanation of what the code does. "We get a foo, we have to decide if its in US or UK format so we compute both checksums and .. oh umm.. I'll come back later'

It's very different to things like design reviews. They have their place too. A lot of things Linus rejects are really design review things. Its not uncommon to get a "Yes this needs fixing but do it this way instead". It works well providing the person saying that has good judgement.

Bad code review, bad tools, bad compilers and bad managers are _all_ useless

Re:Code review and pair programming

[someme2]

[... Examples of how code reviews slowed down this guy's work without doing any good ...]

If this happens to you on a regular basis then you are probably a better-than-average developer. Which is just fine as long as we find a way to make your work more average. And code reviews seem to do the job.

Seriously, the productivity spread between developers (20 times as effective... adding more people... Thank you Mr. DeMarco) is what a lot of very strict process models and practices (such as code reviews) really attack - and it is a good thing, too. Because otherwise developer workforce does not scale well.

The productivity spread is the enemy of plans, predictions, estimations and bureaucracy. Bureaucracy is the most successful human scaling mechanism.

In other news: Big cooperations have difficulty exploiting the individual genius of low-level members of their workforce.

Testing is good

[TreadOnUS]

But the real key is reducing the number of defects introduced into software. Testing only finds existing errors. If the number of errors are low from using good requirements, design and development practices then testing becomes less expensive and time consuming.

Re:Testing is good

[djmurdoch]

Testing only finds existing errors.

No, testing finds new errors. That's what test suites are for: to let you know when your change to X caused Y to break.

Testing and releasing software

[plcurechax]

My employer produces a large-ish software package, with 10 years of history and a small, 2-3 people, development team. Since I joined we have made massive strides in automating the build process, include some unit tests, and a few smoke tests in an automated process.

Well, the effort paid off. Before we supported one version of HP/UX, and one release of Linux, now we support HP/UX (still a pain), and 4 looking at going to 6 Linux version/distributions and it is less work to produce a release now than ever just a year ago.

Tools like automake, autoconfig, libtool, cvstrac and of course cvs have made my life bearable.

Thought is was having a sabbatical?

[Viol8]

Is he back on full time linux development now?

"Alax Cox gave a talk"

Was it in Welsh? :)

Text Of Article

[Anonymous Coward]

Launch of IT Wales Technical Computing Group- Thursday 23rd September 2004

IT Wales, working in partnership with Know How Wales, Knowledge Exploitation Centre and Cygnus Online, has unveiled plans for an exciting new programme of events specifically targeted at computing professionals from both business and academia.

During the launch breakfast, held in Sketty Hall Swansea, on Thursday 23rd September, Beti Williams, Director of IT Wales, outlined the aims and objectives for the group.

"The IT Wales Advanced Technical Computing Group will help to establish and promote Wales as a country with an international reputation for innovation and excellence in Computing and Information Communication Technologies. It will provide a platform for collaboration between computing professionals from business and academia, enabling them to drive forward the computing industry in Wales and stimulate awareness of the diverse applications of computing. The group will also be well placed to highlight the skills required to meet the future demands of the global computing industries."

Guest speaker Alan Cox, a graduate of the University of Wales Swansea and one of the most influential IT innovators in the world, offered the audience an entertaining and thought provoking insight into the current limitations and human failings of the software development process.

The full programme of events will commence in January 2005 and will link with organisations including the British Computer Society, Welsh E-Science Centre and Natural Computing Forum.

Details will be published through itwales.com and e-mailed directly to IT Wales business club members. For full video footage click on one of the links below

To register interest in receiving information from the Advanced Technical Computing Group e-mail details to info@itwales.com

Presentation:

Real Media File (31kbit, 7.5MB,
320x240res)

mpg video (120kbit, 25.5MB,
160x120res)

High quality DivX and 330kbit mpg files available here

DivX version also available in the business club members section

Questions:

Real Media File (31k, 3.3MB, 320x240 res)

mpg video (120kbit, 11.2MB, 160x120 res)

High quality DivX and 330kbit mpg files available here

DivX version also available in the business club members section

Paths to quality

[Anonymous Coward]

The links seem dead so I will add my .02. The programmer's view of the software should not be confined to the source code control system. Programmers should know how to install and implement the software they design and code. Programmers should have some personal experience supporting the software they code. NOTHING highlights the weaknesses of your software faster than being forced to work a support line/forum, etc. Establish personal relationships with employees in the sales force and management. You will be able to be more influential in the company if you do this. Establish personal relationships with your customers. You can use them to push your agendas. Paying customers have some of the most pull with your company. Use this to your advantage.

Slashdotted

[Anonymous Coward]

It's slashdotted..

Btw He didn't write it in Welsh did he? Coz Wales officially doesn't exist http://news.bbc.co.uk/1/hi/wales/3715512.stm

Code validation tools...

[tcopeland]

...are nifty. They can catch all sorts of stuff and produce lovely reports - or, well, at least functional reports. And running them nightly - or hourly - helps to ensure the code won't get out of sorts.

PLUG: Need to check Java code? Try PMD!

Keeping prototypes just that

[mark1348]

You can't help but be frustrated when dealing with projects that were obviously "proof of concept" which then evolved directly into the actual "product". Hard to resist but just plain stupid. I wish more open source projects had strict standards.

Interfaces and contracting...

[LeonardShelby]

Reusable, reliable, quality software begins with the interface. If someone cannot tell what a routine or module does with just a quick read, then all is lost. They'll cease to believe what the documentation (if any) says, and go on to write it themselves.

The solution is better interface design. Clear, concise naming without ambiguity. And including the specification is absolutely necessary. With the contract included as part of the interface, there is even less chance for error and/or any ambiguity. Testing is aided because the rules for calling a routine are right there with the routine interface and comment.

Unfortunately, most programming languages refuse to support contracting in any form, and most developers don't see the benefits. Until this hurdle is breached, quality software will not be achieved.

Steve

--

Re:Interfaces and contracting...

[LeonardShelby]

I'm not claiming silver bullet status. Far from it. I'm simply stating that software engineering requires contracting and better interface design techniques as part of its standard repetoire. Add these to the hard work and discipline you mention.

(Also note these are not tools, but techniques.)

Mirrors Here - Pages and Videos

[Kinetic]

As always, mirrors of the pages and the movies are here: MirrorDot. Enjoy.

There are different types of code reviews...

[Richard Steiner]

We used to implement code reviews at my former workplace, but they were more "sanity checking" peer reviews than anything else. They didn't take very long, and it gave one or two other people a chance to see how understandable or generally logical the changes were.

That type of thing doesn't work as well for large changes, but we found that for small ones it sometimes can be a useful thing.

Re:There are different types of code reviews...

[TreadOnUS]

This type of sanity checking is especially useful for training junior programmers. It can be very instructive for a senior programmer to sit down with a junior progammer and go through their code together. The primary purpose of a review should be to have a second set of eyes on the code but it is very valuable for training and communication as well.

Write the tests *first*

[wowbagger]

More importantly, write the tests FIRST. Then write the code, updating the tests for anything that is identified during the coding.

This has several important benefits:

1. You have to DEFINE what the module is to do so that you can write the tests. Granted, the first pass for the "tests" may simply return "failed" until something is written for the module, but at least you will have a chance to think about what you should be testing.
2. You actually DO write the tests, rather than blowing them off. If your manager says "Why aren't you working on $blarg?" you can say "I *am* working on $blarg" since the first step is writing the tests.
3. As you get funtionality working (as demonstrated by the tests passing) you can quickly determine if a later addition to the code breaks the working feature - and fix it while the change is still fresh in your mind (and hopefully BEFORE you commit your changes to the mainline code path (you ARE using a source code control system, aren't you?))
4. You can automate the testing of the system.

Re:Write the tests *first*

[gustgr]

What? Projecting the test cases before the software is written? If you plan to do some black-box testing this is acceptable, once you can plot the test cases based on the software formal specification. But I guess you know there are software testing approachs other than functional (black-box) testing. If you intend to do some structural testing (white box) it is impossible to write test cases before writing the software, once the testing requeriments are defined by analyzing the source code (that's why it is called white box testing).

Re:Write the tests *first*

[Lumpish Scholar]

If you intend to do some structural testing (white box) it is impossible to write test cases before writing the software ...

Three words: test-driven development.

Re:Write the tests *first*

[_|()|\|]

What? Projecting the test cases before the software is written?

It's an iterative process. You don't necessarily write a complete suite of tests for your interfaces before you start writing a single implementation. Someone in QA might think of a unit test as white box, but they tend to be black box from the perspective of the developer. You should be able to write a unit test before writing the unit.

The point of most unit tests is to verify an implementation's conformance to its interface. When you later change the implementation, you would likely invalidate a white-box test or, if it still happens to be valid, weaken its premise.

Re:Write the tests *first*

[angel'o'sphere]

If you intend to do some structural testing (white box) it is impossible to write test cases before writing the software, once the testing requeriments are defined by analyzing the source code (that's why it is called white box testing).

This is only partly correct. If you design your software via Use Cases and Scenarios or User Storiers, then the possible pathes are in gereat deals predefined. That means you can do black box tests with out any need of white box tests.
To check wether your black box test cover all the code you do a coverage analysis.

White Box tests finally have total different goal!

With doing black box tests, you do tests driven from the outside, usually according to some spec. What you most of the time don't know is wether the spec is closed (has no gaps, does not leave undefined inputs open) etc. The goal is to ensure all legal inputs work.
So you use inspections to spot places where all the black box tests likely won't reach into. E.G. the coverage tool will show you such lines of code. Now you make your white box tests to "cover" those lines as well. The goal then is to catch "illegal" inputs and to ensure the code does at least accept them gracefully.
The question what to do depends on your over all goals and risks. In business applicatins its usually "good enough" to have balck box tests which cover all lines of code.
In an avionic system, thats NOT enough. Here white box tests are much more important.
Finnaly, the way you test affects your system architecture. The way you ensure robustness affects your system performance. And so on ...
angel'o'sphere

Re:Write the tests *first*

[TheLink]

"More importantly, write the tests FIRST. Then write the code"

But write which tests first? There are so many possible. Usually there are more ways for something to malfunction than for it to function correctly.

How do you write a test to check that a banking application does not allow a customer to cancel other customer's cheques? How do you write a test to check that someone didn't allow sql injection? How do you write a test to ensure that a user account cannot do what it is not authorized to?

Since there are usually more ways to test something to check limits than to use it within the limits, your suggestion makes changes a lot more expensive. If you do the code first (and haven't finished writing the 10 tests for it) and people say "We don't like the way that works in practice", you haven't wasted time writing 10 tests. Whereas if you do it the other way round, they can only find out they don't like it after you've written the 10 tests AND the actual code.

Your suggestion probably works well for certain scenarios, but appears counterproductive for many others.

Re:Write the tests *first*

[hachete]

There are some more reasons:

1. You can be more confident when you begin refactoring mature code-bases. This for me is the clincher as code never stands still but the tests can be a constant. A permanent measure.

2. If it's an API, you have working examples to show people.

3. for years, I used informal, undocumented tests. Handing-over was always going to be bad. Now, hand-overs are a, uh, doddle.

4. Progress. Nothing like some concrete test results to show people. Most test suits show results as HTML. Put them on your intranet.

h.

Re:Write the tests *first*

[wowbagger]

First, you confuse "unit testing" with "application testing". You are writing the tests for a *part* of the app, not the whole. So, to use your example, you are writing the tests for parsing a customer's auto-pay entries - not the whole app.

SO:

Your first unit test is a simple "return pass"

You run your test framework and verify everything works.

Commit the changes into the main line branch.

You then change your unit test to "return fail". Run test framework and verify it fails. (NOTE: You do all of this IN YOUR PERSONAL DEVEL BRANCH, not in the mainline code.)

Now, define the first tests for your function - test that it does what is is supposed to do. In this example you would define some simple, correct auto-pay setups and submit them to your module, and confirm the module returns the correct data. At this point, you are now FORCED to give some consideration as to what the inputs, outputs, and API will look like - one of the benefits of this approach.

You now start on the module, creating a stub routine that does nothing.

Run the tests - they should fail as you are not yet "doing what you are supposed to be doing".

Implement the module. If, during this part you discover you have to change the API you also change the tests as needed. Run the tests as you go.

Eventually, you will have a pass - your module is "doing what is is supposed to be doing". Now you add more test data - different valid transactions, and different invalid transactions. You modify the test to pass IF the valid operations pass AND IF the invalid operations are detected and correctly failed.

Run the tests. Watch them fail as your module barfs on the bad data.

Fix the module to deal with the bad data. Run tests. Repeat until tests pass.

Do code review on module. Sally spots a potential problem. Add test case to test. Verify tests fail. Fix problem. Repeat until module passes review.

Merge module and test with mainline code branch and re-run tests.

Now, you have a set of tests for your "auto-pay setup" module. Meanwhile, the others who are implementing the "generate statement", "send bills", and suchlike modules have done the same, so your test framework validates each module. The idea is to limit the scope of failures - you prove each module behaves itself, so the number of possible failures AT THE APP LEVEL is greatly reduced.

Now, George finds a case that causes you module to barf, and writes a bug in the bug tracking system (you *ARE* using a bug tracking system, aren't you?).

You add the bad case to the tests, verify they break, commit change.

You fix the module, and verify the fix. Commit change, mark bug as FIXED.

Your Q/A guys verify that the run before the fix is broken, the run after the fix is OK, and VERIFY the bug, OR they suggest other possible tests that break the system. If so, repeat the last three steps.

Sure, this is no "silver bullet" that will eliminate all bugs. What this DOES do is catch the bugs as quickly as possible, when they affect as little of the project as possible, and VERIFIES that they are indeed fixed and STAY fixed.

Re:Write the tests *first*

[iabervon]

You don't catch SQL injection issues with unit testing; you catch them with type checking. A SQL injection occurs when someone uses a string plus quotes as a partial SQL statement. There should be a different type for partial SQL statements, and the operation to make one for a string constant out of a string should escape the string appropriately. Alternatively, you could just use prepared statements exclusively and make your database happy as well as making SQL injections totally impossible (as the database will be done parsing the SQL when it looks at the user-supplied data).

The best reason to write the tests first, in my opinion, is that this will give you a good idea of whether the API is any good. Many of the APIs I've wanted to be different obviously couldn't have been tested in advance, because writing the test would have made it clear that the API was just too much of a pain to use. If writing 10 example uses for your API is enough work to worry about, your design is bad, and people are probably going to not like how it works. Change it before either finishing the tests or writing the code (or, ideally, telling anyone else about it).

Re:Write the tests *first*

[Surt]

Peronsally, I don't like anything from XP except the testing ideas, and those don't originate with XP.

The process I like best for 'test-first' is actually 'almost-test-first'. Instead of just sitting down and writing tests, the process is:

1) Design a system that solves the problem.
2) Figure out what entry points are to the system.
3) Implement the interfaces for the entry points so that the public contract is defined.
4) Now you start writing tests that verify the fulfillment of the public contract.
5) Implement the interfaces in a way that satisfies the tests.

Our current project has about 7500 tests. The system is what I would rate as middleing complex. I average about 3 sideffects caught by tests per major checkin. That's a lot of subtle stuff no qa person had to chase down, and a lot of developers who aren't irritated that someone else broke their stuff.

Being on a chip

[AnuradhaRatnaweera]

Wonder if he also talked about being on a chip. ;-)

Depends on the language and context...

[Richard Steiner]

In the Fortran variant I was working in until a few years ago, local variable names were limited to five (5) characters, and many otherwise obvious functions were performed by a series of external subroutines with six-character names usually beginning with SY (for "System").

That meant that the code itself could be quite hard to follow without some sort of logical commentary accompanying it.

I tended to comment my code quite heavily in that context, since I'd already had experience with uncommented code in that context and knew how hard it could be to follow...

(Try reading Fortran that's all in column 7 and which is mainly composed of arithmetic IFs and CALL statements sometime and see how intuitive it is!)

When using more modern languages, I tend to comment less, sometimes a lot less. It usually comes down to a judgement call on my part as to whether or not I think the code would be easy for a newcomer to maintain...

Re:software dev

[110010001000]

Viewing your code in assembler? Comment every 3 lines? You must be kidding, or trolling.

The top 5 things are:
1) Meeting the requirements
2) Stability
3) Maintainability
4) Expandibility
5) Efficiency

Linus already covered this

[jaymzter]

"regression testing, what's that? If it compiles it works and if it boots it's perfect!" - Linus

'nuff said

Good practices

[vinukr]

These are some things i guess is necessary for good software
1) Reviews at all stages.(Reqs/design/dev)
2) While at development, u sure must know whats the most efficient way to code a design (which libraries are more suitable etc)
3) Unit testing and Integration testing (when the project is huge)

Some practices that managers can really use to take the pressure off the team
1) Try giving buffers to the team (seriously, it works)
2) Proper Code management (Lotsa rework and pressure come due to lack of this)
3) Proper tracking and status updates to the customers

Didn't know he used Gentoo...

[cs02rm0]

emerging techniques for producing higher quality software

MBA?

[Anonymous Coward]

So now we're taking software writing advice from PHBs now? :)

MDA?

[little1973]

At my company the management has started an MDA (Model Driven Architecture) project, because some developers presented it to them as the holy grail in software development. They use a GUI to make associations between classes and ASL for the code. They say that you are less likely to make a mistake because you code in a platform independent way (so, you do not have to pay attention to all the little details) and the translator is responsible to make the actual code.

Some of us more experienced developers do not think it is the holy grail. It looks like you can make as much mistake as in convetional languages. Also, development with a GUI (see at www.kc.com) is much more cumbersome.

Is there anyone who used MDA and ASL and has some experience about it?

Re:software dev

[Timesprout]

Surprising you dont seem to think good software should meet the users/customers requirements. That would be first on the list for me. Software must do what the user wants or it is a total failure as far as I am concerned. In general I would agree with your points. Code should be robust, testable, maintainable and reasonably performant. I prefer the source code to be documented well and autogenerate documenation from it rather than maintain duplicate documentation.

I am also a big fan of profiling code, and for server apps, proper load testing is and absolute must. It often only finds the low hanging fruit as performance bottle necks tend to be resource based eg the database is usually the 'culprit'. Profiling and code coverage/ code analysis are valuable tools though to identify critical paths and likely sources of problems and help developers gain a clearer picture of how the application actually works.

Re:Ping Wales slashdotted already.

[headshrinker]

it's working again after I upped the server processes a bit :)

my first slashdotting, fun.

Gareth

WHY, not what

[wowbagger]

The single best rule of comment is "Comment upon the WHY, not the WHAT".

Don't say:

double sum = 0.0; // zero sum
for (i = 0; i < len; ++i) // all samples
{
double signal = buf[i]; // get value
sum += signal*signal; // add signal^2 to sum
}
return sqrt(sum/len);


say:
// Compute RMS average of signal -
// compute the sum of the squares of all values,
// then compute the mean (sum/len), then the
// root of the sum - hence Root-Mean-Squared

double sum = 0.0;
for (i = 0; i < len; ++i)
{
double signal = buf[i];
sum += signal*signal;
}
return sqrt(sum/len);

In other words, tell me WHY this code added the square of the signal, not THAT it added the square of the signal.

Moreover:

1. Every directory of the project should have a purpose, and should contain a short README describing the purpose of the code in the directory.
2. Every file should have a header that describes the purpose of the file.
3. Every function should have a header describing the purpose of the function, as well as any inputs and outputs (including global, static, and class variables), any exceptions thrown, and any assumptions made.
4. Blocks of code within a (large) subroutine should have a descriptive block comment describing the overall goal of that block (e.g. "Now we walk the list of conversations and update the call stats").
5. Comments on a line of code shouldn't be needed normally - the code should speak for itself. Only pretty tricky things ("use a shift and add rather than a multiply as this saves 3 clock cycles") should need a comment at end of line.

If more folks would follow these rules it would be a HELL of a lot easier to follow their code.

NOTE: If you can say it in the code, do so - if you can specify the exceptions to your function via a "throw(int code)" type statement, then do so rather than using a comment.

Remember - the code tells the COMPILER and the programmer what is going on, the comments tell the programmer WHY it is going on.

MBA- Management

[nuggz]

Oh no, someone with an education in MANAGEMENT suggesting ways to MANAGE a production process.

Yes your average programmer/engineer might be able to manage a project. But why not take some of the expertise of a manager to make it a bit better?

If someone like Alan Cox should now be ignored as "some MBA toting PHB" how open minded are you?
I think Alan might have a bit of an idea how the software development process works.

If you're not even willing to consider their ideas, you're doing yourself quite a disservice.

Some tools I've found handy ....

[mike_diack]

For DOS/Windows/OS2 etc:

Gimpel Software's:
PC Lint (cheapish)
Flexelint (pricier)

Freeware (checks GDI leaks)
bear.exe (http://www.geocities.com/the_real_sz/misc/bear_.h tm)
gdiobj.exe (http://www.fengyuan.com)

Linux:
Electric Fence (free)
Valgrind (free)
Splint (free)

Books:
John Robbins books on debugging. Concentrates on Win 32 but useful ideas wise for any x86 platform.

And now the gags...

Tools I've not found helpful.....
Rational Rose!
Microsoft's beloved COM!

QA != testing !!!

[gosand]

I have said it for years, and I'll keep saying it.

Quality Assurance is not testing.

Testing is testing, and can run the gamut from unit to use case, from integration to system, from acceptance to beta. But QA is not testing. A lot of people call testing QA, but it is not the same thing.

Testing is what you do when you get the code. QA is everything that you do throughout the software development cycle to ensure that you have quality software. This can include code reviews, process audits, statistical analysis, etc.

I have been doing QA and testing for 11 years now. I have a degree in computer science, and I CHOSE to do this career. You may be able to get away with ignoring QA professionals and still produce high-quality software. But not all software projects are equal. QA is probably the most overlooked part of software development, testing the second.

Re:Unit testing first

[johannesg]

Hye, cal lyour pear pro gramign budy! You rtypign skils a ren t up to scrtach wehny our alon e!

Man, if only there were unit tests for slashdot postings... Ah, wait, we have those (lameness filter!) and they don't help at all!

INTERFACES INTERFACES INTERFACES!

[argent]

Good interfaces is why UNIX originally took off. Back in the '70s when you had to actually call the OS rather than just doing (say) Fortran formatted I/O, you had to do stuff like:

Allocate a file control block. In assembly, this was usually done statically at compile time. In Fortran it was usually a global common.

Call something like SYS$INIT%FCB

Fill in half a dozen random fields in the FCB with magic numbers. Like, file mode, file type, access mode, device type, file name, file extension, file owner, file password, read key, write key, tape policy, ...

Call something like SYS$OPEN%FILE, which may involve two or more calls. If It was a device, you might need to call SYS$ATTACH%DEVICE, for example, or SYS$LOCK%FILE...

Repeat the above for an I/O control block. If the I/O control block doesn't contain a buffer, repeat the same for the buffer. You may need to allocate an event flag, or just pick one and hope that the same one wasn't used by another library. You may need to allocate a "mailbox" or other asynchronous I/O endpoint.

Call the variant of SYS$READ%BLOCK for the file type or device type you're using. You could be calling SYS$READ%BUFFERED or SYS$QUEUE%IO followed by SYS$WAIT%IO.

Do something with the buffer.

Manually increment the block offset in the I/O control block.

Repeat the reads until done.

Release the I/O control block, and any event flags or mailboxes.

Call SYS$CLOSE%FILE or SYS$RELEASE%DEVICE or whatever.

On UNIX you did this instead:

int fd;
char buffer[BUFSIZ];

fd = open("file name",mode);
if(fd != -1) {
while(nr = read(fd, buffer, BUFSIZ) > 0) {
do_something_with(buffer, nr);
}
close(fd);
}

This was revolutionary. Really. It's not perfect, but it's so much better than what we were using at the time that it's no wonder people couldn't wait for AT&T and re-implemented UNIX from scratch several times.

This is the same kind of improvement we need in our interfaces for GUIs, databases, network services, and so on. Even the Berkeley socket interface is too complex... all those details of address formats and address structures? Those shouldn't be there... you should be able to do this:

fd = open("/tcp/host.domain/http/options",mode);
// or at least fd = tcpopen("host.domain","HTTP",sockopts);
if (fd != -1) {
...
close(fd);
}

Yes, there's libraries that do this, but they all have the same socket()...gethostbyname()...connect() stuff under the hood. This should be handled at the system call level.

As for GUIs... Plan 9 is about the only one I've seen that looks like it's even trying to reach the level of clarity, safety, and consistency we need.

QA

[Brandybuck]

...but others seem heavily influenced by his recent MBA. In particular, he has a lot to say about Quality Assurance in the software world...

Software QA by normal people: Test the product.

Software QA by MBAs: Assure that twenty thousand meaningless documents are signed, perform audits to ensure that these documents are signed, provide mandatory training so engineers know how to sign these documents, award bonuses to those who sign the most documents, define productivity to be the number of signed documents in an engineer's cabinet.

Re:Type Checking = false promises

[dvdeug]

Heavy type-checking relies on the same theory that more buerocracy is the solution to quality.

The earlier you catch a bug, the easier it is to fix. That's been proven. Heavy type-checking catches bugs at compile time. Dynamic type-checking may leave bugs until the user runs over them.

Is the array row-first or column-first? If rows and columns are different types, then the compiler will tell me when I got it wrong. I don't find that chasing that bug later, when it could be anywhere in a thousand lines of code, is a more efficent way to quality. Sure, it means a few more casts when I want to add rows and columns, but I've never found that a problem.

Type-checking doesn't always make the code harder to read. In a type-checking language, "if (a==0)" lets me know that a is a number. In several non-type-checking languages, a could be just about anything. It's possible that you can't figure out what a is, even after looking around the code; it's possible that the else part of that statement may have to deal with a being a list, or a record, or a string. There is no local checks you can make to discover that; you may have to read every line of code to discover what type a is.

Quality assurance is dead.

[Pathetic Coward]

Any programmer that tries to "convince" management that quality assurance checking is desirable will be first on the outsource-to-India list.

Management doesn't want any backtalk about "quality". They expect the programmers to do things right the first time; that's what they're paide to do. When management has compliant workers in India that work cheap, follow instructions, don't talk back, and aren't around the manager's office geeking up the place, they're not going to bother with "quality assurance" insubordination. In particular, programming methodologies such as Extreme Programming that require greater management involvement in the coding process will be treated with scorn; management wants less involvement, not more. As to whether or not the code actually works - once it's out the door and bonuses are in hand, who cares?

Re:Quality assurance is dead.

[dubl-u]

In particular, programming methodologies such as Extreme Programming that require greater management involvement in the coding process will be treated with scorn; management wants less involvement, not more. As to whether or not the code actually works - once it's out the door and bonuses are in hand, who cares?

Well, for one, companies that want to make money. A company may be able to get away with shipping crap for a little while, but it's rare that somebody can do it for long. Why? It's not just that customers notice. A cruddy code base is hard to build on, so low initial quality means slow development for the life of the code base.

Once a manager has experienced doing a project right, it's pretty rare they want to go back to the old, painful, unproductive way of doing things.

Quality is not profit

[Anonymous Coward]

Stunned by the horrible quality of the software in the megacorp I work for, I was complaining to a dot-com millionaire friend of mine who got rich on a different company. It was enlightening.

He related the experience of meeting one of their VPs. Turns out the VP was famous for having realized that every customer service call was an opportunity to SELL MORE PRODUCT. Their profits went up. Wooo-hooo! Writing SHIT CODE actually made them MORE MONEY.

I realized it was the same with my company. On all the on-line forums I hear "I'd never buy any other brand. I once had this horrible problem, and their customer service was so responsive! They fixed the problem!" It never seems to occur to customers that if the product actually WORKED they wouldn't have needed to call support in the first place. Shit code boosts our profits, too. People get a warm fuzzy talking to good customer support, no matter how bad the technology is.

There are other ways to benefit from bad code, especially if you dominate the market. Releasing a new version? If you botch the backward compatibility it forces everyone to upgrade! Profit!

So, what's all this nonsense about better code?

Alan Cox doesn't get it...

[renehollan]

... which surprises me because he is one smart dude.

All his points are valid, but (a) dangerous when taken as gospel, and (b) miss the forest for the trees.

What kills software is complexity. I've been writing code professionally (i.e. getting paid for it) since I was 15. It's been 28 years. Starting with Dijkstra's "GOTOs Considered Harmful", I've seen fads to improve software reliability come and go: structured programming, object-oriented programming, garbage collection to handle memory leaks, etc; as well as programming languages prividing the syntactic and semantic sugar to support the fad du jure. Hasn't helped, has it?

The general problem is one of managing complexity: if you can "come from" anywhere to a snippet of code, how can you ensure that all your assumptions about the context you're in are valid? Similarly, if you can access a global variable, well, globally, how can you be sure it contains what you expect? How do you know all the ways your data can be accessed? How do you control when and where an object is destructed, or that all resources (memory being just one) are freed?

Either restrict who can do what, or restrict the assumptions you make about the things you are operating on.

Using a garbage collector restricts who can allocate and deallocate memory. Object oriented programming restricts who can muck with private members. Structured programming restricts how you can get somewhere. O.K. wise guy, how are you going to write a garbage collector without dealing with raw memory? Gee, you have to get your hands dirty. How are you going to deal with private members inside private member functions? Gee, looks a lot like functional programming with globals, doesn't it?. How, the heck are you going to compile that loop without a jump at the end? Gee, what was that about GOTOs?

The trend appears to be to try to find the "next great technique" which will provide the best bang for the buck when improving quality. All of them help. None of them enough. Frankly, this incremental approach is not going to solve the problem of defects that are a result of the product of human error and code complexity. We need to learn how to manage complexity on its own.

Are there any examples of success in managing complexity, and can we learn from them?

I think so.

The best example I can think of is a compiler. Look at how many different inputs it can handle and still produce correct machine code with a fairly low defect rate. I attribute this to the fact that its input is highly structured: it has to follow strict syntactic rulers. Thus, when compiling something, one has a great deal of knowledge about the context in which this occuring. Yes, you have to deal with semantic issues (types, declarations, etc.), but an unambiguous language should make this clear.

One can point out that programming languages are well-specified, so implementing a compiler is relatively easy. If only all requirements were so detailed. I don't think that's it, however: one can come up with very detailed specifications for a complicated system that's difficult to implement. A programming language, by contrast, can be syntactically expressed in a few pages of BNF.

Contrast this with a user interface, where various elements need to be enabled or disabled depending on what other elements have been previously activated, or used to gather data. How easy or difficult is it to "forget" to enable or disable a control? If you see a parallel with this problem and excessive use of GOTOs in a program, you're starting to get a feel for what I'm talking about.

What has happened in the past 25 years or so of programming language evolution is that the complexities of the past have been pushed and morphed into the complexities of the present. And tackling one in isolation (memory management) does nothing for a transformation of the same problem (resource management in general -- memory isn't the only thing that leaks), though it may provide the best bang defect-rate-reduction-wise

Re:Original ideas are highly overrated

[Alan Cox]

If you are giving a talk to people in small businesses do you want to give them a list of things they can do or have an esoteric discussion over the relative reliability data for C++ and Java exception models ?

I'm glad you don't think the content is original - that means you are one of the people who actually has some idea of how to write good code and automate/enhance the testing side. Unfortunately to a lot of people out there actually writing code in business the concepts are new.

This was Alan's morning talk to local businesses. I'm suprised it was even worthy of slashdot, when things like Indymedia disks being seized by the FBI and rackspace are apparently not even noticed by the slashdo crowd

Oh wait .. Rackspace are advertisers
8)

Alan

Where to begin...

[andymac]

OK, first off let me fess up: I am a QA manager, have been for near on 10 years now. I've worked on projects with MS, Adobe, IBM, Sun, yadda yadda yadda. All sorts of projects and languages, products and programmers.

Cox really does highlight some of the best practices out there, but he also skims over some key ones. The biggest problem I've seen out there (and I've done QA Management consulting for a good chunk of those 10 yrs, so I've seen a lot of organizations) is commitment by management. Most QA folk know that there will always be a challenge to find the right balance between schedule/budget, quality of product, and feature set of the product. Do you want it good? now? or stripped down? And most QA folk are willing to work within that mindset. But when management 1) does not appropriately staff QA activities; 2) does not appropriately fund QA activities & annual budgets; 3) does not make it damn clear to all staff that QA is a requirement for project and thus ultimately product success and if you don't like someone testing your code and logging bugs against it, you better move along pardner, the organization pays lip service to QA. Heck even when QA finds a horrible problem prior to release, so there's time to fix the problem, you'd think most folks would be happy (ok, not thrilled because that balance is skewed towards a schedule slip) that there was time to get in a fix. But no, 99 times out of 100, QA is slammed for holding up the process.

Independent testing is a must for an organization to have any real understanding of the quality of the product. Engineers cannot be the only folks testing their outputs. For oone, it's a very expensive way to test - I want geers designing & coding & unit testing not integration or system or release testing. QA folks will cost between 30-70% of a geers salary - why would you want the most expensive resource doing the (in most cases) less technically demanding work? And work they usually don't enjoy doing anyways (grumble grumble, I didn't sign up for this!)

OK, so this is a bit of a rant. I'm just dealing with my current senior management who say they want QA to manage and execute the independent testing, but turn around and find 95% of Engineereing who refuse to participate in the process.

I'll just go have a beer and forget about my problems...mmm... beer... drool drool.