Slashdot Mirror
Donald Knuth Rips On Unit Tests and More
eldavojohn writes "You may be familiar with Donald Knuth from his famous Art of Computer Programming books but he's also the father of TeX and, arguably, one of the founders of open source. There's an interesting interview where he says a lot of stuff I wouldn't have predicted. One of the first surprises to me was that he didn't seem to be a huge proponent of unit tests. I use JUnit to test parts of my projects maybe 200 times a day but Knuth calls that kind of practice a 'waste of time' and claims 'nothing needs to be "mocked up."' He also states that methods to write software to take advantage of parallel programming hardware (like multi-core systems that we've discussed) are too difficult for him to tackle due to ever-changing hardware. He even goes so far as to vent about his unhappiness toward chipmakers for forcing us into the multicore realm. He pitches his idea of 'literate programming' which I must admit I've never heard of but find it intriguing. At the end, he even remarks on his adage that young people shouldn't do things just because they're trendy. Whether you love him or hate him, he sure has some interesting/flame-bait things to say."
... looks like it falls into the same trap as COBOL. The idea that by making programming languages incredibly verbose, they will somehow become easier to use is a fallacy.
Using "MULTIPLYBY" instead of "*" isn't going to make your code easier to read.
Literate programming is an old friend for developers of functional programming languages. I see it like "code for the human mind": it provides a source code that is well adjusted to the needs of the developer, not just the machine.
It interleaves code and documentation in the same files, and provides a specialized compilator to tell the two kinds of codes apart. Just like Doxygen and Javadoc can extract the comments from a source code project, the "tangle" process can extract all the code from a Literate program and pass it to a clasic compiler.
Now that C and C++ seem to have a declining popularity, maybe we can look for better ways of getting away from the bare metal (which, don't forget it, is why those languages become popular at the beginning). Don't get me wrong, they served us well for 36 years, but I think it's time again to begin caring more for the developers' requirements and less for the hardware requirements.
Singularity: a belief in the "God" idea with the "demiurge" relation inverted.
The reason for his dismissive attitude of unit tests - that he knows exactly how all of his code works, and what impact a change will have - is exactly the reason you need them. In the real world, most programmers do in fact have to share their code with others. You're not always going to know the ramifications of refactoring a particular block of code, even if you wrote it yourself. And if you can keep all of that in your head at once, either your program is trivial, or you are some sort of supergenius. Now while I think the TDD guys are a little bit overzealous sometimes with their "100% coverage or die" attitude, unit testing is still a good habit to get into, regardless of what Knuth thinks.
Amen about TeX (and even LaTeX). I consider myself pretty knowledgeable about many computing languages, but every time I've hit a non-trivial problem with making TeX do what I want, I've had to consult with a TeXpert (i.e. the utterly invaluable comp.text.tex). And, sadly, in almost every case the solution has been either insanely baroque, or there's been no real solution at all. LaTeX makes brilliant looking documents, but Jesus wept, it's hard to make your documents look like YOU want, as opposed to how it thinks they should look.
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
He's right about unit tests... sort of. Just as most coders shouldn't be designing interfaces, most coders don't know how to test. It can often be more work writing the unit tests than writing the code.
If you have a function that multiplies two integers, most coders will write a test that multiplies two numbers. That's not good enough. You need to consider boundary conditions. For example, can you multiply MAX_INT by MAX_INT? MAX_INT by -MAX_INT? Etc. With real world functions you are going to have boundaries up the whazoo. In addition, if you have a function that takes data coming from the user, check for invalid cases even if another function is validating. Check for null or indeterminate values. Write tests that you expect to fail.
Don't blame me, I didn't vote for either of them!
The reason I think literate programming doesn't catch on has mostly to do with the fact that a great many programmers don't bother to think through what they want to do before they code it: they are doing precisely what Knuth mentions he does use unit testing for -- experimental feeling out of ideas. Because they don't start with a clear idea in their heads, of course they don't want to start by writing documentation: you can't document what you haven't thought through. This is the same reason why things like design by contract don't catch on: to write contracts it helps to have a clear idea of what your functions and classes are doing (so you can write your pre-conditions, post-conditions and invariants) before you start hammering out code. The "think first" school of programming is very out of favour (probably mostly because it actually involves thinking).
Craft Beer Programming T-shirts
Making a program parallel will always be too hard for most programmers. But that's exactly why you don't have normal programmers do it... have the libraries do it automatically. Functions like qsort(2) are already black boxes, so they can be made to always run in parallel when the input is large enough. Other functions like say Ruby's .collect can run in parallel. For things like .each there can be a parallel and a sequential version that the programmer can pick which is appropriate.
But to do this we need operating systems that can efficiently and reliably schedule code across cores. Add an ability to 'bind' threads together, so that they schedule always at the same time but on separate real processors. This gives the program the ability know when running an operation split between these threads will always complete faster than sequentially, without vagaries of scheduling possibly starving one thread and making it run much slower.
Once you have this then you can automatically get some speedups from multiple cores on programs that are designed to only run sequentially, and more speedup on programs with just minor tweaks. You aren't going to get perfect scaling this way, but you will get substantial improvements at virtually no cost to the programmer.
The headline is misleading. Donald Knuth represents the epitome of the solitary super-programmer stereotype, so it's only natural that he sees no need for unit tests to catch mistakes or extreme programming to improve team development practices. I don't think he's necessarily saying that those things are without value for ordinary programmers.
That's a brave stance. He's old, but he hasn't reached his dotage yet. The good doctor has contributed more to the science of information than most, and almost certainly more than you.
One of the reasons why we're reinventing so much over and over with nuisances like VB and C# is that developers are architecting grand toolchains based on ideas that were in the 1960's proven incorrect. They get a lot profits from their workarounds, and then we burn it all down and start over because they all contain the same fatal flaws.
That would be because you haven't installed Vista on it yet.
Having watched this tragedy unfold for a quarter century I've often shook my head and wondered what y'all were thinking. And then I remember that I once thought my parents were fools too. If you can read TAOCP and understand a good fraction of it you will come away with a firmer foundation for the way all things work. It's a tough slog, though, and not everybody is capable.
Help stamp out iliturcy.
FVWM on Ubuntu Linux. Emacs with special modes using a homemade bitmap font. Mac OSX for Illustrator and Photoshop...
Now that's breadth AND depth.
I somewhat disagree with what you and... *sigh* Monkeybaister posted. Yes, there are many times when long stretches of code should be broken out into functions. But I tend to do that mostly when the same bit of code is used in several different cases. The reason being is that when you start modularizing off all your while loops that are more than a dozen lines long, you create a whole new type of spaghetti code. I'm going to coin a term and call it "spaghetti-O code." You try to track down a bug and what would have been a straightforward couple pages of code now has all kinds of functions at different places in the code. As such, it can often make debugging or forming a mental map of the code much harder.
The summary sounds like it was written by the headline-producing monkeys at Fox, CNN -- or hell, at the Jerry Springer show. Donald Knuth is not "playing hardball." Nobody needs to call the interview "raw and uncut," or "unplugged."
The interview has almost nothing to do with unit testing and the little Knuth does have to say about the practice is hardly "ripping."
When will people stop sullying peoples' good names by sensationalizing everything they say?
Knuth is a well-respected figure who makes moderate, thoughtful statements. From the summary, you'd think he was a trash-talking pro-wrestler.
After initially being a proponent, I've come to the same conclusion about unit tests myself. I don't think they're worthless, but the time you spend developing or maintaining unit tests could be more profitably spent elsewhere. Especially maintaining.
That's my experience, anyway. I suppose it's pretty heavily dependent on your environment, your customers, and exactly how tolerant your application is of bugs. Avionics software for a new jet fighter has a different set of demands than ye olde "display records from the database" business application. More applications fall in the second category than the first.
"my dualcore laptop really has no problem with that"
Moore's law says (well, indirectly at least) that machines from 2007 should be roughly 256 times as powerful as machines from 1995.
Somehow, the actual performance difference (starting the computer, starting a web-browser, editing text etc.) in running Win95 on hardware from it's time, compared to running Vista on todays hardware, seems to be nowhere near a 256-times improvement.
I can only conclude that while the hardware-industry have improved itself again and again, the software industry have ate almost all of those improvements, instead of giving it to the users.
I expected much, much more from Knuth than what I've just seen in that interview and after reading the design of MMIX.
Knuth dismisses multi-core and multi-threading as a fad and an excuse by manufacturers to stop innovating. I'm amazed someone of his intelligence has managed not to read up on exactly WHY this is happening:
So he dismisses the technical problems that manufacturers have been falling over for the last few years as merely a lack of imagination. No - parallelism is here to stay, and people need to realise it rather than just wishing up some magical world where physics aren't a problem.
He dismisses multi-threading as too hard. It isn't, if you're not unfair to the concept. Nobody is getting 100% out of their single-threaded algorithms. You always have stalls due to cache misses, branching, the CPU not having exactly the right instructions you need, linkage, whatever. Nobody EXPECTS you to get 100% of 1 CPU's theoretical speed. So why do people piss all over multi-core/multi-threading when it doesn't achieve perfect speed-ups?
If you achieve only a 50% speed-up using 2 cores compared to 1, you're done a good job, in my opinion. That means you could have dual-core 3GHz CPU or a single core 4.5GHz CPU. Spot which of those actually exists. Getting a 25-50% speed-up from multi-core is easy. The 100% speed-up is HARD. If you stop concentrating on perfection, you'll notice that multi-threading is a) actually not hard to implement, and b) worthwhile.
Then there's MMIX. Knuth thinks that simplicity has to work all the way down to the CPU design. Yes, but not simplicity by way of having instructions made up of 8 bit opcode and 3x8 bit register indexes. A CPU doesn't give a crap how elegant that looks. It's also BAD design - 256 registers makes for a SLOW register file. It'll either end up being the slow critical path in the CPU (limiting top clock speed) or taking multiple cycles to access. There's also no reason to have 256 opcodes. He should have a look at ARM - it manages roughly the same functionality with much less opcodes.
It almost pains me to see the MMIX design and how it's a) not original, b) done better in existing systems already on the market, e.g ARM, and c) doesn't solve any of the performance limit problems he complains about. What's going on with Knuth?
C and C++ only seems to be declining because they are the de facto standards and everyone knows how to use them. There is no need for news articles, blog entries, etc. They may not be hip but they power everything and are more used than anything else.
...which the compiler can't discover, because foreach describes a mechanism (looping through a sequence, in order), and not a high-level transformation.
Compare foreach with map. Map is a higher order function that takes a function and a collection, and results in a collection of the same size and structure as the original, but with each element replaced by the result of applying the supplied function to it. Note that the value of each element in the result depends only on the corresponding element of the input. It's trivial to parallelize map.
You can parallelize map easily because it has a favorable contract that specifies the relationship between its inputs and its outputs, and it just so happens that this contract is amenable to parallel execution. A smart compiler, upon seeing a use of map, can trivially tag it as a parallelism candidate.
But since foreach specifies a sequential looping mechanism, there are no guaranteed relations between the input and output (in fact, not even any simple way to determine what should be treated as inputs and outputs). When you write a foreach loop to perform the equivalent of a map, you're underspecifying the transformation you're performing on your collection, and overspecifying the mechanism. That's bad programming.
You mention Parallel LINQ, and this is very relevant. LINQ is based on operations similar to map, that transform sets into sets. LINQ queries, since they abstractly describe the relation between an input and the desired output, can be executed in a number of ways: (a) the system can translate them into SQL queries and send them to a database server to execute; (b) the system can execute them serially; (c) the system can execute them in parallel.
Are you adequate?
If you define "bug" to mean "unexpected undocumented behaviour", as Knuth seems to, then it's not surprising that there have been very few bugs claimed, since TeX is so very well documented.
But most people -- and certainly the majority of open source projects these days -- define "bug" as "undesirable behaviour"; and by that standards, TeX is chock full of bugs. To pick a couple of obvious examples, incorrect handling of ASCII 0x22 quotation marks, and treating "etc." as the end of a sentence. These aren't "bugs" to Knuth since the incorrect behavious is well documented, but by many people's standards they are.
What's purple and commutes? An Abelian grape.
No offence meant, but I think your preconceptions may be clouding your judgement here.
You claim that today's programming field is not about clever tricks and fast algorithms. I claim that if more people understood these old-fashioned concepts, we would have much better software today. For a start, anyone developing those "libraries implemented by specialists" you mentioned had better be very good, since a lot of other people's code is going to depend on them. Having worked in groups that develop various kinds of library, I can assure you that a little more general programming knowledge about clever tricks and fast algorithms wouldn't go amiss.
You claim that today's programming field is about big systems with many programmers. I claim that this is because management and technical leadership in most places isn't competent enough to divide up a big system in modular fashion and allow smaller, more flexible teams to solve the little problems before multiplying them all up to solve the big ones. Instead, the guys at the top tend to reduce all problems to the least common denominator, "throw enough people at it and we'll win eventually" philosophy. This explains how a small company with a few dozen employees can produce software that is better in every way than the competing offering from a larger company with hundreds of developers. You don't even need to have a dew dozen genius programmers; you just need to understand the concept that there are O(n^2) lines of communication between n individuals working in a single large team, but if your project is divided hierarchically then logarithms start coming into play, and if you can split a problem into several properly independent smaller ones this becomes a constant factor overhead. This elementary mathematics seems to be beyond a lot of senior management in the software business, and that has far more to do with the need to build monolithic systems maintained by zillions of developers than any actual project requirements do.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
One would hope that you get to the final system design by iterative prototyping, which you then clean up, refactor, and rewrite into a nice codebase that doesn't have the evolved and iterative cruft, and is suitably documented (it's this last step where literate programming an come, though you can introduce some incrementally as modules and subsystems get frozen and rewritten). Who knows, maybe you do just ship the first thing you can get to work. What I do know is that in the business world where time is money, the clean well documented final version is going to involve a lot less time (and hence a lot less money) to maintain and debug and patch. And what if I want a new version next year? Well I would certainly prefer to have a clean readable well documented codebase to work from than "whatever worked" code. Maybe that's just me though. Different people work differently, and I'm sure you do what works best for you.
Craft Beer Programming T-shirts
I've never seen any interesting and useful software that is ever "finished". You always need to add and change things, and there is always far more functionality wanted than you can produce.
If you clean up and refactor as you go, rather than at "the end", what you have described is Agile/XP development.