Documentation As a Bug-Finding Tool

New submitter Sekrimo writes "This article discusses an interesting advantage to writing documentation. While the author acknowledges that developers often write documentation so that others may better understand their code, he claims documenting can also be a useful way to find bugs before they ever become an issue. Taking the time to write this documentation helps to ensure that you've thought through every aspect of your program fully, and cleared up any issues that may arise."

Comments

Common knowledge?

[O('_')O_Bush]

I thought everyone knew that documentation describes what you intended code to do, rather than what it actually does.

Re:Common knowledge?

[jgrahn]

I thought everyone knew that documentation describes what you intended code to do, rather than what it actually does.

The documentation tries to document what your intentions are, just like the code tries to implement them. The code can fail to do its job, and of course the documentation can too!

Re:Common knowledge?

[icebike]

I thought everyone knew that documentation describes what you intended code to do, rather than what it actually does.

Just as often, while writing documentation on code I just wrote, I've thrown up my hands, and thought "this is so ridiculous and embarrassing that I can't be associated with it", and I went back and re-wrote it to do it the right way. The act of documenting it revealed you left too much undone, or too many situations un-handled.

Any time your documentation reads like you are describing a game of twister you just know the code can't be worth documenting, or even keeping.

But as for finding bugs, I don't know. You may document exactly what you intended, and thought the code did, but still be wrong because of some corner case. Documenting it isn't likely to reveal all of these situations any better than the code itself. A

Re:Common knowledge?

[O('_')O_Bush]

Article summarised: "Thinking about what you write produces better results than not."

Re:Common knowledge?

[ZackSchil]

Just as often, while writing documentation on code I just wrote, I've thrown up my hands, and thought "this is so ridiculous and embarrassing that I can't be associated with it", and I went back and re-wrote it to do it the right way. The act of documenting it revealed you left too much undone, or too many situations un-handled.

This is by far more common for me as well. I'll find myself describing some really brittle setup process or some common operation that seems to take a thousand steps, get embarrassed over how bad it is, and then go fix the code. More than once, I've written sophisticated automatic parameter selection code because it was easier than finishing the documentation I had started explaining to the user how to set up parameters properly.

Re:Common knowledge?

[Anonymous Coward]

Tell that to the PHB that wants the code to ship YESTERDAY--bugs or no bugs!

To me, proper programming is and exercise in minimalism--get the most work you can out of the simplest and least least amount of code.

Then comments/documentation isn't so critical.

But that takes too long and time is money to the average PHB. So you get apps that barely work and must be 'updated' to work better and to be an income stream for the company that put said software out.

But when you have people DELIBERATELY writing needless, convoluted code in an effort to keep their job because they are the only one who can upgrade it--you have the situation we have to day which is addressed humorously by the The International Obfuscated C Code Contest

Re:Common knowledge?

[AuMatar]

If you really filled 30 positions in 2 months, your problem is likely in hiring shitty programmers. A most companies I've worked at, we made offers to 10-20% of the people who interviewed. Unless you're doing 5+ candidates a day, and all offers are accepting, you're murdering that rate. Some of that may be better offers or more efficiency, but it sounds like you're hiring a lot of mediocre people to fill seats.

Re:Common knowledge?

[TheRaven64]

There's a very old saying:

If the code and documentation disagree, then they're both wrong

The documentation itself is probably not the important bit. The thing that a lot of programmers seem to do wrong is getting the correct ordering of the thinking and coding steps mixed up. Writing documentation first means that you have to do the thinking before the coding, and that eliminates a whole load of problems.

Re:Common knowledge?

[Anonymous Coward]

Kudos to you if you can refactor the implementation in your head before you write it!

Re:Common knowledge?

[St.Creed]

Interesting sidestep: consider Literate Programming (http://en.wikipedia.org/wiki/Literate_programming). Donald E. Knuth advocates the approach of document, AND code, then compile the documented code. I have always been fond of this and try to use this in all my programming.

Just comment preconditions, postconditions and mock up the pseudo-code, then extend it.

I'd absolutely love it if Eclipse and Visual Studio supported this. However, most people don't know Literate programming so I doubt it's going to be in there anytime soon.

Re:Common knowledge?

[SimonInOz]

When I was involved in writing one of the first packet switching systems in Europe (AT&T, Belgium, 1979), we found a brilliant way to fix bugs was to explain the bug (and thus the operation of the program) to someone. They didn't have to do much, just nod and look interested.
Then usually about halfway though, the hapless coder (eg me) would go "Oh shit" ... and the listener cold then leave.
We called it the "tailors dummy" approach to debugging.

A bit like pair programming, only less labour intensive.

Re:Common knowledge?

[mickwd]

So let's get this straight: you just sacked 60% of your developers for not following certain rules?

That 60% of your developers had properly been informed that certain rules were important enough to follow that breaking them would probably mean dismissal, and then went ahead and broke them anyway?

That there was no process in place of informal verbal warnings / formal verbal warnings / formal written warnings that could have been followed before sackings?

That this wasn't discovered before it was widespread enough that it became necessary to sack 60% of your developers?

That you have decided you'll be more productive in future with only 40% of your new development workforce having any experience of your software whatsoever?

Your company has BIG problems.

Re:Common knowledge?

[gbjbaanb]

rubbish. At no point in any development process does thinking not enter the equation.

If you're doing something innovative, you'll be spending much more time thinking - re Edison's "genius is 1% inspiration, 99% perspiration" comment. He knew what innovation was!

Too many developers I know think that you can slap something together (calling it 'being creative' or 'innovative') and fail to do a professional job which ends up requiring massive amounts of maintenance, if it works properly at all. Those who really do good job think first, write down their ideas so they won't forget them, develop them, then write down the results so others can use them.

Re:Common knowledge?

[Glonoinha]

I read this as 'When I don't know what I'm doing, I hack at it until I get something working and then I document what I did.' Fair enough. At that point you are documenting it at a fairly detailed level (hopefully). This happens when software development is done as 'Art'.
When software development is happening as 'Science', odds are you can at least outline your intent and design before you start coding the solution.

I've done both. Computer Science usually produces better results than Computer Art.

Better than customer service

[kawabago]

as a bug finding tool. Lots of unhappy customers that way.

Indeed

[bazald]

Explaining your work is a great way to demonstrate that you actually understand it. As the article illustrates, perhaps the most critical part is going back and verifying that the code matches the explanation.

Re:Indeed

[paimin]

This is news? Wow, the human race is going downhill.

Re:Indeed

[grcumb]

Explaining your work is a great way to demonstrate that you actually understand it.

My standard development process is:

1. Document the method;
2. Write the tests for the method;
3. Write the code for the method;
4. Make corrections to each until everything works;
5. Move on to the next method.

My rationale is precisely that: I'm not really sure I know what I'm doing until I've described it, then figured out how my idea might fail.

Forgive my ignorance, but doesn't everyone do this?

Re:Indeed

[purpledinoz]

I think most people to this:

Write method.
Write unit tests.
Fix until it works.
Write documentation.

I generally find that code documentation is a lot better if the document first approach is used. In the above approach, the method is done, and the documentation becomes a chore.

Re:Indeed

[Jah-Wren Ryel]

Forgive my ignorance, but doesn't everyone do this?

Yep, pretty much every one does do that.
Where they tend to differ is in their particular definitions of "do."

IS0 9000?

[jayveekay]

Say what you do.
Do what you say.

The documentation becomes an error check on the code and vice-versa. If the 2 disagree, something needs to be fixed.

Re:False

[icebike]

For some methods of documentation this is very true. For some programmers that care about their work its very true.

But if you don't care about your code, you probably won't care about the documentation. In this case, I agree its False.

If you know the documentation you just wrote is a bag of lies but you turn it in anyway, because you know that the PHB won't understand it and couldn't check if it was true if his life depended on it, then you might as well junk the code you just wrote as well. Chance are it will break the minute you walk out the door.

Good code

[pwnyxpress]

If your code is good, it should be self documenting. Everyone knows what the variables 'foo' and 'bar' do.

So, someone rediscovered Literate Programming?

[Elf Sternberg]

Oh, come on, Literate Programming has been around for 30 years! Knuth made exactly this argument in his 1984 essay entitled, surprisingly enough, "Literate Programming!" Wikipedia asserts in it "Literate Programming" entry: "According to Knuth, literate programming provides for higher-quality programs, since it forces programmers to explicitly state the thoughts behind the program, making poorly thought-out design decisions more obvious. Knuth also claims that literate programming provides a first-rate documentation system, which is not an add-on, but is grown naturally in the process of exposition of one's thoughts during a program creation. The resulting documentation allows authors to restart their own thought processes at any later time, and allows other programmers to understand the construction of the program more easily."

Congratulations to Slashdot for posting about some kid rediscovering an ancient technology by a revered master of the craft. What's next? "Snot-nosed highschooler discovers GOTO is a bad idea?"

Documemtation Priority

[PolygamousRanchKid]

Manager, at the beginning of a project: "Forget the documentation! Just get it to run!"

Manager, at the end of a project: "Where's the documentation! You were lazy, and didn't write any!"

Documentation is at the ass-end of a project. The Manager's Manager wants to see something running. He doesn't accept paper as a currency. So documentation will always get low priority. And that ass-end will be hanging out and swinging in the breeze.

Someone could do a scientific study that proves that documentation cures cancer.

It will still get low priority in a software development project.

Knuth

[phantomfive]

I believe this is the method Knuth recommended, and formed the beginnings of the idea of literate programming.

This article from 1996 never gets old

[sn00ker]

Titled They Write the Right Stuff it looks at the coding practices at the company that wrote the control software for the space shuttles. If you want to know about documentation as a bug-finding tool, this is pretty much the holy grail.

Consider these stats : the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.
...
Take the upgrade of the software to permit the shuttle to navigate with Global Positioning Satellites, a change that involves just 1.5% of the program, or 6,366 lines of code. The specs for that one change run 2,500 pages, a volume thicker than a phone book.

The term "documentation" is subjective

[ciaran.mchale]

There is an old joke: "The definition of promiscuous is somebody who has more sex than you do". From reading TFA and some of the comments on slashdot, I get the feeling that the definition of documentation is equally subjective and self-serving for developers. Some developers think that writing documentation means adding comments to code. Others feel it involves writing Javadoc/Doxygen-style comments at the start of every class and method, and then generating HTML from that. Yet others feel that documentation hasn't been written unless it involves an architectural description.

When I am working on my own open-source projects, I feel that documentation isn't complete until I have written a few hundred pages of text that aim to be cover most/all of the following: (1) API reference guide, (2) programming tutorial, (3) user guide, (4) architectural guide, and (5) suggestions for "future work" that I hope other people will volunteer to do. Yes, I recognise that I am a bit extreme in the amount of effort I put into writing documentation. However, it does enable me to elaborate on the thesis of TFA: attempting to write such a comprehensive amount of documentation often highlights not just coding bugs, but also architectural flaws. This causes me to work in an iterative manner. I implement a first draft version of the code. Then I start documenting it, and when I encounter a part of the software that is difficult to explain, I realise that I need to re-architect the code base a bit. So I do that, and then get back to writing documentation, which causes me to notice another difficult-to-explain issue with the code. Working in this manner is slow, and I suspect it wouldn't work in a business with time-to-market pressures, but I find it gives excellent results in my own, non-time-pressured open-source projects. I touched on this issue in the documentation for one of my open-source projects.

Re:False

[David Gerard]

A Kuro5hin classic: HOWTO: write bad documentation that looks good.

Pick descrisptive variable names

[Required Snark]

This is another good documentation tool, and a way of avoiding bugs. It is surprisingly hard to do.

If you can't think of a good reasonably short and descriptive name then you don't understand the concepts as well as you should.

I only use variable names like i,j,k for loops. I use x1,x2,y1,y2 and similar names only for numeric values. This is applicable when I am implanting math algorithms. If I have a lot of similar variables differing by their last digit and I'm not doing equations, I know I am writing code that I won't be able to read later.

I tend to declare one variable per line, and describe what I am using it for as a comment. If I have a lot of variables I split them into groups, which I separate by blank lines.

I try and avoid reusing intermediate variable names, unless they are in different lexical scopes. It is fine to have similar name inside loops that doing similar work, but make sure that you are not confusing concepts when reusing variable names thie way.

I have been working on algorithms, and have stopped and spent an hour or more thinking about what to call the variables. I do this when I get confused. It always pays off. When you have a good descriptive name and you see it in it's use context, you can actually see the mistakes before you make them.

Re:Pick descrisptive variable names

[davide marney]

Even better:

1. When naming entities, pick the SAME names that the business people use when they talk about their domain. Do they call that thing you're working on an "XYZ Thingy"? Then that's the name you should give that entity in the code, formatted according to your conventions for names. Why? Because then your code aligns with the requirements, models, and documentation, no translation needed.

2. When naming methods, use GENERIC names for generic actions, or build names from combining generic action verbs with entity names, sort of the way the German language builds compound words. The entire web, for example, runs on a handful of method names, and the top three (HEAD, GET, POST) account for 99% at that.

This is the domain-driven design way to do it.

Theory doesn't always work in practice.

[Barbara, not Barbie]

The theory was, write the documentation, then code to the documentation.

In practice, that isn't sufficient to reduce bugs significantly, for several reasons.

1. As you develop something, you find that "getting from A to B" sometimes requires going via D instead of C;
2. Other times, you realize that the documentation doesn't completely capture the requirements, and you need to visit both C and D, and maybe Z
3. Still other times, you realize that A is entirely superfluous.
4. "Can you add/change this feature?"

The initial specification should never be too specific about implementation details - that's a mistake that too many people fall for, going with the illusion that they actually can nail down every detail of a non-trivial problem and just throw the spec at "code monkeys." They don't understand that a specification should only say what, not how. Writing "code documentation" before writing the code is writing the "how".

So they can end up with something that meets that spec, but doesn't work either as intended or just flat-out is wrong.

Unfortunately, code, then document (when and if you get around to it) is the reality because, unlike theory, reality is messy.

Re:Theory doesn't always work in practice.

[St.Creed]

You need to distinguish between functional and technical specs.

Functional specs are very usefull (if done even halfway right). Technical specs are a waste of time unless you assume by default that you're dealing with incompetents, in which case you're better off saving yourself time, money and aggravation and hire a better developer.

So I do write a lot of functional specs. Even now, in an agile environment, with HUGE time pressure and multi-million penalties for delivering late - just finishing up my final 40 pages (90 pages total, in this 4 week sprint). Why? Because not doing so will make the project much later. Good architecture (system design) specs will make the project about 20% more likely to deliver on time (citation if I can find it again) even if the programmers don't follow the guidelines (interesting, right?). This matches with my experience: if you write decent designs, you are more likely to find the pitfalls before they can bite you in the ass. If you cover all the bases and make sure the business has provided for all scenarios before you get there, your project will run smoother.

So docs may not prevent all the bugs. But it does prevent a large number of nasty stuff before it gets to the stage where it turns into a defect.

Re:Theory doesn't always work in practice.

[ThirdPrize]

Wrong. First you write the documentation, then you write the tests and then, if you have time, you write the code.

Actually, I think most people do this:

[petes_PoV]

Sit around trying to make sense of the requirement
Write some code that they think does what the requester really meant
Spend most of their time fighting the code management system and getting it to build cleanly
Pull an all-nighter just before the deadline so it doesn't crash when fed correct input
Toss it "over the wall" to the integration team
Refuse to answer any questions about it as they're not "too busy" on the next project

Have a nice feeling of satisfaction that they never have to do support on old code, as none of it ever gets into a production system
Get promoted frequently for meeting their targets on productivity and delivery times

Re:Someone teach this guy about circular buffers S

[donscarletti]

Crap coders tend to focus on incidental things that they feel will improve the quality of their code, rather than addressing the issues directly.

Re:I must be in bizarro world..

[St.Creed]

I understand, but at least do this:
- keep a change log. Even when I'm under extreme pressure I write down at least what the customer wants, and what I'm going to do about it. What did I change where.
- keep a decision log. Every time you have to interpret a request or design spec in a certain way, write it down.

Over time, these will keep you afloat.

doctest

[sad_]

python doctest ftw! - http://docs.python.org/py3k/library/doctest.html