Slashdot Mirror
Donald Knuth Rips On Unit Tests and More
eldavojohn writes "You may be familiar with Donald Knuth from his famous Art of Computer Programming books but he's also the father of TeX and, arguably, one of the founders of open source. There's an interesting interview where he says a lot of stuff I wouldn't have predicted. One of the first surprises to me was that he didn't seem to be a huge proponent of unit tests. I use JUnit to test parts of my projects maybe 200 times a day but Knuth calls that kind of practice a 'waste of time' and claims 'nothing needs to be "mocked up."' He also states that methods to write software to take advantage of parallel programming hardware (like multi-core systems that we've discussed) are too difficult for him to tackle due to ever-changing hardware. He even goes so far as to vent about his unhappiness toward chipmakers for forcing us into the multicore realm. He pitches his idea of 'literate programming' which I must admit I've never heard of but find it intriguing. At the end, he even remarks on his adage that young people shouldn't do things just because they're trendy. Whether you love him or hate him, he sure has some interesting/flame-bait things to say."
... looks like it falls into the same trap as COBOL. The idea that by making programming languages incredibly verbose, they will somehow become easier to use is a fallacy.
Using "MULTIPLYBY" instead of "*" isn't going to make your code easier to read.
Now, I've no problem with literate programming, but given that even semi-literate practices like "write good comments" hasn't caught on in many places, I think Don is flogging a dead horse by suggesting that code should be entirely documentation driven.
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
I have a lot of respect for Knuth as an algorithms guy, but anything he says about programming needs to be taken with a grain of salt. When he created the TeX language, he lost all credibility - designing a language in 1978 which makes structured programming almost impossible is just insane. TeX gets a lot of praise as being 'bug free,' but that's really only half true. The core of TeX is a virtual machine and a set of typesetting algorithms, both of which are very simple pieces of code (they'd have to be to run on a PDP-10). Most of the bits people actually use are then metaprogrammed on top of the virtual machine, and frequently contain bugs which are a colossal pain to track down because of the inherent flaws in the language (no scoping, for example).
If you want to learn about algorithms, listen to Donald Knuth and you will learn a great deal. If you want to learn about programming, listen to Edsger Dijkstra or Alan Kay.
I am TheRaven on Soylent News
Literate programming is an old friend for developers of functional programming languages. I see it like "code for the human mind": it provides a source code that is well adjusted to the needs of the developer, not just the machine.
It interleaves code and documentation in the same files, and provides a specialized compilator to tell the two kinds of codes apart. Just like Doxygen and Javadoc can extract the comments from a source code project, the "tangle" process can extract all the code from a Literate program and pass it to a clasic compiler.
Now that C and C++ seem to have a declining popularity, maybe we can look for better ways of getting away from the bare metal (which, don't forget it, is why those languages become popular at the beginning). Don't get me wrong, they served us well for 36 years, but I think it's time again to begin caring more for the developers' requirements and less for the hardware requirements.
Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
the prize was not US$1000. it started out very small. Knuth did indeed pay out, and indeed doubled it, several times. From wikipedia: "The award per bug started at $2.56 (one "hexadecimal dollar"[24]) and doubled every year until it was frozen at its current value of $327.68. This has not made Knuth poor, however, as there have been very few bugs claimed. In addition, people have been known to frame a check proving they found a bug in TeX instead of cashing it."
Using "MULTIPLYBY" instead of "*" is asinine, because both are equally descriptive. Putting a comment above the line telling people why you're doing it isn't.
The original prize was $2.56 (i.e. 2^8Â), and he doubled it every time someone found a bug until it reached $327.68. Over 400 bugs have been fixed in TeX, and that's just counting the core VM and typesetting algorithms - all of the rest is in metaprogrammed packages, many of which contain numerous bugs. I'm fairly sure that most programmers could write bug-free code if the only place where bugs counted was in a simple VM with a few dozen instructions that interpreted all of the rest of the code...
I am TheRaven on Soylent News
The reason for his dismissive attitude of unit tests - that he knows exactly how all of his code works, and what impact a change will have - is exactly the reason you need them. In the real world, most programmers do in fact have to share their code with others. You're not always going to know the ramifications of refactoring a particular block of code, even if you wrote it yourself. And if you can keep all of that in your head at once, either your program is trivial, or you are some sort of supergenius. Now while I think the TDD guys are a little bit overzealous sometimes with their "100% coverage or die" attitude, unit testing is still a good habit to get into, regardless of what Knuth thinks.
http://en.wikipedia.org/wiki/Knuth_reward_check
Many people save these (usually small) checks as souvenirs. My father -- a frugal mathematician -- received a few $2.56 checks from Knuth, and he promptly cashed each one.
The snippets have markup to indicate when some snippet needs to come textually before another to keep a compiler happy, but mostly this is figured out automatically. But in general, the resulting C code is in a different order than it appears in the source documentation. For instance, the core algorithm might come first, with all the declarations and other housekeeping at the end. (With documentation about why you're using this supporting library and not that, of course.)
He's right about unit tests... sort of. Just as most coders shouldn't be designing interfaces, most coders don't know how to test. It can often be more work writing the unit tests than writing the code.
If you have a function that multiplies two integers, most coders will write a test that multiplies two numbers. That's not good enough. You need to consider boundary conditions. For example, can you multiply MAX_INT by MAX_INT? MAX_INT by -MAX_INT? Etc. With real world functions you are going to have boundaries up the whazoo. In addition, if you have a function that takes data coming from the user, check for invalid cases even if another function is validating. Check for null or indeterminate values. Write tests that you expect to fail.
Don't blame me, I didn't vote for either of them!
The headline is misleading. Donald Knuth represents the epitome of the solitary super-programmer stereotype, so it's only natural that he sees no need for unit tests to catch mistakes or extreme programming to improve team development practices. I don't think he's necessarily saying that those things are without value for ordinary programmers.
That's a brave stance. He's old, but he hasn't reached his dotage yet. The good doctor has contributed more to the science of information than most, and almost certainly more than you.
One of the reasons why we're reinventing so much over and over with nuisances like VB and C# is that developers are architecting grand toolchains based on ideas that were in the 1960's proven incorrect. They get a lot profits from their workarounds, and then we burn it all down and start over because they all contain the same fatal flaws.
That would be because you haven't installed Vista on it yet.
Having watched this tragedy unfold for a quarter century I've often shook my head and wondered what y'all were thinking. And then I remember that I once thought my parents were fools too. If you can read TAOCP and understand a good fraction of it you will come away with a firmer foundation for the way all things work. It's a tough slog, though, and not everybody is capable.
Help stamp out iliturcy.
"I've only proven that it works, I haven't tested it" - Knuth
I somewhat disagree with what you and... *sigh* Monkeybaister posted. Yes, there are many times when long stretches of code should be broken out into functions. But I tend to do that mostly when the same bit of code is used in several different cases. The reason being is that when you start modularizing off all your while loops that are more than a dozen lines long, you create a whole new type of spaghetti code. I'm going to coin a term and call it "spaghetti-O code." You try to track down a bug and what would have been a straightforward couple pages of code now has all kinds of functions at different places in the code. As such, it can often make debugging or forming a mental map of the code much harder.
Seriously, if you're "religious" about unit testing and mock objects, then you really need to revise the way you live your life.
It's just a good habit to get into, if you take it seriously and don't just create tests that test silly little things like "is my text box centered where I slapped it on the form with gui form tool" type of stuff. That's kind of the point he's trying to make, that you program intelligently in the first place to avoid having an insane amount of redundant tests to pass each time you build.
I've been doing literate programming (well, as close as you can with C and its derivative languages) for a long time now. I've watched XP coders take that literacy and chop it all up because "it didn't look pretty enough". The idea with making something literate is to make it so clear that you can reduce the total numbers of tests needed to make that code pass to only ones that test the actual expected outputs of that function. That's something that intelligent coders who don't just follow the Agile rulebook, but apply it effectively, can do. I don't know how many times I'd see a piece of code that did one simple task, had one test to test the output of that test, then another coder drops 3 more tests because they "didn't feel comfortable with only one" without specifying WHY. That is how you get into having redundant tests that muck up your test infrastructure.
I'll forgive you for being a Java developer, but the fathers of C have always cited readability first (The C Programming Language ~1978). They don't call it "literate programming", which is simply a trendy buzzword, but the idea of programming for readability has been around for an extremely long time.
There are a few techniques to be mastered and using a language designed with parallelism in mind helps hugely with the picky details.
Assuming you work an 8 hour day, that means you are testing your code every 2 minutes and 24 seconds. Given that most of your tests will take this long to run (you've got a suite of them right?), that leaves you with zero time to actually do the work you are testing.
Frankly, if you are using Unit Tests you should be using them after major chunks of work, not in a trial and error fashion. Now if you were using them in a trial and error fashion - "lets change this, run the tests and see if they pass, no that didn't work, lets try this", etc, I could understand how you hit the 200 times per day mark. That is a completely respectable position to take. I used to work in this way during college. My several thousand unit tests do take 2-3 minutes to build. My estimate of 200 times was probably over shot and should be more like 50-75 times a day. I would also like to point out that I can continue developing while the tests run. I use Maven2 as a build tool and enjoy it immensely, it helps me do test driven development. Since you are obviously far superior to me, I will assume you know what this means but point it out to the rest of the idiots like myself. TDD is where you write your unit tests before you code. Then you satisfy your unit tests with code. When you need to change code, you change the tests and then you change the code to fix the tests. Crazy waste of time right? If you are coding in a trial and error fashion and using unit tests that way, I'd advise getting some tuition or changing career. Thanks, I love you too.
But the thing is that my employer loves my work and my code rarely breaks. Now why is that? Perhaps because I'm regression testing at all times? Perhaps it's because I take the time to think about things before I do them and, as a result, I really begin to understand what it is that I'm writing.
An added benefit is that I've found I can look at my or others unit tests and really understand what was going through their mind when the first wrote the method that I am expanding. It's quite interesting, but I'm sure you are a supreme being like Knuth and don't bother with such trivialities. I'm a highly productive individual
Here's a question: how much time do you spend working out what happened when your code breaks? TDD is a trivial amount of time compared to that. I am concerned about my software in the present and future. I wish others were also. What counts is that when you run the unit tests, they pass, and that they accurately test the conditions that need testing. I disagree with you. I run unit tests that fail all the time--on purpose.
I know it will most likely result in a swift abrasive response but I implore your highness to really spend some time understanding how unit tests can help the really stupid coders (like me).
My work here is dung.
It's also out of favor because of how much of the real world of programming works. My very small company does a lot of work for a very, very large company. At my small company, we have one layer of management - the owner of the company. Everyone else is in the level of "not an owner of the company."
At the large company, there are a multitude of layers of management. Any software they build require extensive specifications and documentation far in advance of laying down any code. The decide all the aspects of the software before it's written. At my company, the boss just gives us a general outline of what he's thinking about and ask us to feel out the idea. We use a RAD environment and will often have a first iteration within a week. This version tends to get completely, sometimes multiple times. We do not document any of this in advance because the usable version may differ so much from the original ideas.
At the large company, their projects tend to take years and years, go far over budget and typically are much less useful than they had originally hoped. As a bonus, they are usually bug-ridden and unstable. Many times they just eventually get canceled by the new layer of management, who then get awards for this "cost saving measure." At my company, our projects are typically finished far in advance for a tiny price. They are typically of very high quality, with very minor bugs which we fix rather quickly.
This large company frequently hires our company to build software rather than trying to do it internally. They are usually amazed at the things we can do.
Something like "literate programming" is completely anathema to how our company works. If we started trying to write specifications in advance of figuring out what product our clients actually want (as opposed to what they think they want at the start of the process).
Now, I will state that our company only works because we don't hire idiots or slackers. Also, I am fully aware that this is not a good way to, for example, design nuclear power plant software or a baggage control system. But for businesses, all that documentation and "thinking" can just cloak that fact that the people building the software don't know what they are doing.
The summary sounds like it was written by the headline-producing monkeys at Fox, CNN -- or hell, at the Jerry Springer show. Donald Knuth is not "playing hardball." Nobody needs to call the interview "raw and uncut," or "unplugged."
The interview has almost nothing to do with unit testing and the little Knuth does have to say about the practice is hardly "ripping."
When will people stop sullying peoples' good names by sensationalizing everything they say?
Knuth is a well-respected figure who makes moderate, thoughtful statements. From the summary, you'd think he was a trash-talking pro-wrestler.
After initially being a proponent, I've come to the same conclusion about unit tests myself. I don't think they're worthless, but the time you spend developing or maintaining unit tests could be more profitably spent elsewhere. Especially maintaining.
That's my experience, anyway. I suppose it's pretty heavily dependent on your environment, your customers, and exactly how tolerant your application is of bugs. Avionics software for a new jet fighter has a different set of demands than ye olde "display records from the database" business application. More applications fall in the second category than the first.
If you define "bug" to mean "unexpected undocumented behaviour", as Knuth seems to, then it's not surprising that there have been very few bugs claimed, since TeX is so very well documented.
But most people -- and certainly the majority of open source projects these days -- define "bug" as "undesirable behaviour"; and by that standards, TeX is chock full of bugs. To pick a couple of obvious examples, incorrect handling of ASCII 0x22 quotation marks, and treating "etc." as the end of a sentence. These aren't "bugs" to Knuth since the incorrect behavious is well documented, but by many people's standards they are.
What's purple and commutes? An Abelian grape.
No offence meant, but I think your preconceptions may be clouding your judgement here.
You claim that today's programming field is not about clever tricks and fast algorithms. I claim that if more people understood these old-fashioned concepts, we would have much better software today. For a start, anyone developing those "libraries implemented by specialists" you mentioned had better be very good, since a lot of other people's code is going to depend on them. Having worked in groups that develop various kinds of library, I can assure you that a little more general programming knowledge about clever tricks and fast algorithms wouldn't go amiss.
You claim that today's programming field is about big systems with many programmers. I claim that this is because management and technical leadership in most places isn't competent enough to divide up a big system in modular fashion and allow smaller, more flexible teams to solve the little problems before multiplying them all up to solve the big ones. Instead, the guys at the top tend to reduce all problems to the least common denominator, "throw enough people at it and we'll win eventually" philosophy. This explains how a small company with a few dozen employees can produce software that is better in every way than the competing offering from a larger company with hundreds of developers. You don't even need to have a dew dozen genius programmers; you just need to understand the concept that there are O(n^2) lines of communication between n individuals working in a single large team, but if your project is divided hierarchically then logarithms start coming into play, and if you can split a problem into several properly independent smaller ones this becomes a constant factor overhead. This elementary mathematics seems to be beyond a lot of senior management in the software business, and that has far more to do with the need to build monolithic systems maintained by zillions of developers than any actual project requirements do.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.