Slashdot Mirror
Book Review: The Art of Computer Programming. Volume 4A: Combinatorial Algorithm
asgard4 writes "Decades in the making, Donald Knuth presents the latest few chapters in his by now classic book series The Art of Computer Programming. The computer science pioneer's latest book on combinatorial algorithms is just the first in an as-of-yet unknown number of parts to follow. While these yet-to-be-released parts will discuss other combinatorial algorithms, such as graph and network algorithms, the focus of this book titled Volume 4A Combinatorial Algorithms Part 1 is solely on combinatorial search and pattern generation algorithms. Much like the other books in the series, this latest piece is undoubtedly an instant classic, not to be missing in any serious computer science library or book collection." Keep reading for the rest of asgard4's review.
The Art of Computer Programming. Volume 4A: Combinatorial Algorithms Part 1
author
Donald E. Knuth
pages
883
publisher
Addison-Wesley Publishing
rating
9/10
reviewer
asgard4
ISBN
0-201-03804-8
summary
Knuth's latest masterpiece. Almost all there is to know about combinatorial search algorithms.
The book is organized into four major parts, an introduction, a chapter on Boolean algebra, a chapter on algorithms to generate all possibilities (the main focus of the book), and finally 300 pages of answers to the many exercises at the end of every section in the book. These exercises and answers make this work an excellent companion for teachers of a university course.
The book begins with some introductory examples of combinatorial searching and then gives various definitions of graphs and directed acyclic graphs (DAGs) since a lot of combinatorial algorithms conveniently use graphs as the data structures they operate on. Knuth's writing style is terse and to the point, especially when he presents definitions and proofs. However, the text is sprinkled with toy problems and puzzles that keep it interesting.
After the introduction, the first chapter of the book (out of only two) is titled "Zeros and Ones" and discusses Boolean algebra. Most readers that have studied computer science in some form should be intimately familiar with most of the discussed basics, such as disjunctive normal forms and Boolean functions and their evaluation. The reader might be surprised to find a discussion of such an elemental foundation of computer science in a book on combinatorial algorithms. The reason is that storage efficiency is especially important for these types of algorithms and understanding the basic storage unit of computer systems nowadays (as the decimal computer is a definite thing of the past) is of importance.
After covering the basics of Boolean algebra and Boolean functions in quite some detail, Knuth gets to the most fun part of this chapter in my opinion: the section on bitwise tricks and techniques on integer numbers. Being a software engineer in the video games industry, I recognized a lot of the techniques from my day-to-day work, such as bit packing of data and various bit shifting and bit masking tricks. There is also a discussion of some interesting rasterization-like algorithms, such as the shrinking of bitmaps using Levialdi's transformation or filling of regions bounded by simple curves. The chapter concludes with Binary Decision Diagrams that represent an important family of data structures for representing and manipulating Boolean functions. This topic was also quite interesting to me since I have never been exposed to it before.
The second and main chapter of the book is titled "Generating All Possibilities". In this particular volume of the The Art of Computer Programming series, the only subsection of the chapter in this volume is on generating basic combinatorial patterns, or more specifically generating all n-tuples, permutations, combinations, partitions, and trees. We can expect more on this topic from Knuth in his continuation in Volume 4B and beyond.
The discussion on n-tuples starts out with a lengthy focus on Gray codes, which are binary strings of n bits arranged in an order such that only one bit changes from string to string.
A quite fun example for generating all permutations presented in this part of the book is alphametics, also sometimes known as verbal arithmetic — a kind of puzzle where every letter of a word stands for a digit and words are used in equations. The goal is to assign digits to letters in such a way that the equation is correct. A classic example is SEND + MORE = MONEY (the solution is left as an exercise for the reader).
The next section deals with generating all combinations. Given a set of n elements, the number of all possible combinations of distinct subsets containing k elements is the well-known binomial coefficient, typically read as "n choose k". One of the more interesting algorithms in this section of the book to me was generating all feasible ways to fill a rucksack, which can come in quite handy when going camping.
After combinations, Knuth moves on to briefly discuss integer partitions. Integer partitions are ways to split positive integer numbers into sums of positive integers, disregarding order. So, for example 3, 2+1, and 1+1+1 are the three possible partitions of the integer 3. Knuth, in particular, focuses on generating all possible integer partitions and determining how many there are for a given number. The book continues with a concise presentation of the somewhat related topic of set partitions, which refer to ways of subdividing a set of elements into disjoint subsets. Mathematically, a set partition defines an equivalence relation and the disjoint subsets are called equivalence classes; concepts that should be familiar to any mathematics major. Again, the focus is on generating all possible set partitions and determining how many partitions can be generated.
The main part of the book closes with a discussion of how to exhaustively generate all possible trees, which is a topic that I have never given much thought to. I am familiar with generating permutations, combinations, and partitions, but have never really been confronted with generating all possible trees that follow a certain pattern. One main example used throughout this part of the book is generating all possible strings of nested parentheses of a certain length. Such strings can be represented equivalently as binary trees.
Knuth's latest book is comprehensive and almost all encompassing in its scope. It compiles an incredible amount of computer science knowledge on combinatorial searching from past decades into a single volume. As such, it is an important addition to any computer science library. This book is not necessarily an easy read and requires a dedicated reader with the intention of working through it from front to back and a considerable amount of time to fully digest. However, for those with patience, this book contains a lot of interesting puzzles, brain teasers, and almost everything there is to know on generating combinatorial patterns.
On a final note, if you don't have volumes 1-3 yet you can get all volumes in a convenient box set .
Martin Ecker has been involved in real-time graphics programming for more than 10 years and works as a professional video game developer for High Moon Studios http://www.highmoonstudios.com/ in sunny California.
You can purchase The Art of Computer Programming. Volume 4A: Combinatorial Algorithms Part 1 from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
The book begins with some introductory examples of combinatorial searching and then gives various definitions of graphs and directed acyclic graphs (DAGs) since a lot of combinatorial algorithms conveniently use graphs as the data structures they operate on. Knuth's writing style is terse and to the point, especially when he presents definitions and proofs. However, the text is sprinkled with toy problems and puzzles that keep it interesting.
After the introduction, the first chapter of the book (out of only two) is titled "Zeros and Ones" and discusses Boolean algebra. Most readers that have studied computer science in some form should be intimately familiar with most of the discussed basics, such as disjunctive normal forms and Boolean functions and their evaluation. The reader might be surprised to find a discussion of such an elemental foundation of computer science in a book on combinatorial algorithms. The reason is that storage efficiency is especially important for these types of algorithms and understanding the basic storage unit of computer systems nowadays (as the decimal computer is a definite thing of the past) is of importance.
After covering the basics of Boolean algebra and Boolean functions in quite some detail, Knuth gets to the most fun part of this chapter in my opinion: the section on bitwise tricks and techniques on integer numbers. Being a software engineer in the video games industry, I recognized a lot of the techniques from my day-to-day work, such as bit packing of data and various bit shifting and bit masking tricks. There is also a discussion of some interesting rasterization-like algorithms, such as the shrinking of bitmaps using Levialdi's transformation or filling of regions bounded by simple curves. The chapter concludes with Binary Decision Diagrams that represent an important family of data structures for representing and manipulating Boolean functions. This topic was also quite interesting to me since I have never been exposed to it before.
The second and main chapter of the book is titled "Generating All Possibilities". In this particular volume of the The Art of Computer Programming series, the only subsection of the chapter in this volume is on generating basic combinatorial patterns, or more specifically generating all n-tuples, permutations, combinations, partitions, and trees. We can expect more on this topic from Knuth in his continuation in Volume 4B and beyond.
The discussion on n-tuples starts out with a lengthy focus on Gray codes, which are binary strings of n bits arranged in an order such that only one bit changes from string to string.
A quite fun example for generating all permutations presented in this part of the book is alphametics, also sometimes known as verbal arithmetic — a kind of puzzle where every letter of a word stands for a digit and words are used in equations. The goal is to assign digits to letters in such a way that the equation is correct. A classic example is SEND + MORE = MONEY (the solution is left as an exercise for the reader).
The next section deals with generating all combinations. Given a set of n elements, the number of all possible combinations of distinct subsets containing k elements is the well-known binomial coefficient, typically read as "n choose k". One of the more interesting algorithms in this section of the book to me was generating all feasible ways to fill a rucksack, which can come in quite handy when going camping.
After combinations, Knuth moves on to briefly discuss integer partitions. Integer partitions are ways to split positive integer numbers into sums of positive integers, disregarding order. So, for example 3, 2+1, and 1+1+1 are the three possible partitions of the integer 3. Knuth, in particular, focuses on generating all possible integer partitions and determining how many there are for a given number. The book continues with a concise presentation of the somewhat related topic of set partitions, which refer to ways of subdividing a set of elements into disjoint subsets. Mathematically, a set partition defines an equivalence relation and the disjoint subsets are called equivalence classes; concepts that should be familiar to any mathematics major. Again, the focus is on generating all possible set partitions and determining how many partitions can be generated.
The main part of the book closes with a discussion of how to exhaustively generate all possible trees, which is a topic that I have never given much thought to. I am familiar with generating permutations, combinations, and partitions, but have never really been confronted with generating all possible trees that follow a certain pattern. One main example used throughout this part of the book is generating all possible strings of nested parentheses of a certain length. Such strings can be represented equivalently as binary trees.
Knuth's latest book is comprehensive and almost all encompassing in its scope. It compiles an incredible amount of computer science knowledge on combinatorial searching from past decades into a single volume. As such, it is an important addition to any computer science library. This book is not necessarily an easy read and requires a dedicated reader with the intention of working through it from front to back and a considerable amount of time to fully digest. However, for those with patience, this book contains a lot of interesting puzzles, brain teasers, and almost everything there is to know on generating combinatorial patterns.
On a final note, if you don't have volumes 1-3 yet you can get all volumes in a convenient box set .
Martin Ecker has been involved in real-time graphics programming for more than 10 years and works as a professional video game developer for High Moon Studios http://www.highmoonstudios.com/ in sunny California.
You can purchase The Art of Computer Programming. Volume 4A: Combinatorial Algorithms Part 1 from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
I already have the entire O'Reilly library, plus selected volumes from the "for dummies" and "...in 21 days" series. Why do we need another lousy computer book? This one doesn't even appear to cover anything useful like HTML coding or Adobe software.
After all the Foo for Dummies books that review on /. and rate a 10/10, Donald Knuth just gets a 9/10? Sad...
Lorem ipsum dolor sit amet, consectetuer adipiscing elit.
Much like the other books in the series, this latest piece is undoubtedly an instant classic, not to be missing in any serious computer science library or book collection.
During a job interview I was given a test. Some questions/problems were good, other were not. One of the not-so-good questions presented 8 or so sorting algorithms and asked for their run time complexity (O notation). I answered bubble sort and quick sort and then added that I bought Knuth vol 3 so I didn't have to memorize such trivia. I'm not sure the engineer who created and graded the test liked the answer but the manager of the team (not an engineer) loved the answer after I explained what Knuth vol 3 was. I got hired.
Someone actual read The Art of Computer Programming? Are you sure it wasn't just sitting on your shelf?
Knuth's books are awesome, not just for the content (which would itself be a bargain at quadruple the price) but also for the sheer intimidation factor.
However, I've got to admit: the volumes I'm most looking forward to -5, 6, and 7- are yet to come. This bothers me, because with the way Volume 4 keeps growing, I'm no longer convinced that he's going to live long enough to finish the series, not because of any slowness on his part but because the work just keeps getting bigger and bigger. Has he made arrangements for others to finish the series in case the worst happens?
This new book combines all of the previously-published Volume 4 fascicles from 2005 to 2009, all of which I bought last year and am still reading. Those fascicles are:
All the volumes combined are a true masterpiece for the computer science community. I do not know of many other fields in the sciences where the core ideas, both theoretical and practical, are wrapped up so well. The only comparison I know of is The Merck Manual for physicians. If anyone knows of definitive and comprehensive readings for other engineering fields like EE, CivilE, or ChemE, I'd like to know of them.
Your comment makes me sad. You're missing so much really beautiful stuff that will help you in ways that you can't even imagine, and you don't even know it.
They'll be a nice addition to the other pristine volumes on your "personality bookshelf" ...
Be forewarned. Some of us who have fairly pristine looking copies today once pooled resources and read a shared copy back in the day.
Only students of advance academic computing theory can actually glean anything from them ... Very different for our "instructables" DIY culture.
I think it is a little more approachable than you suggest. I read it sophomore year of a computer science program, while some proofs were beyond my abilities the concepts and algorithms were not. Basically these volumes can be used as practical references to algorithms and concepts. Part of the popularity of the books is due to its making grad school level work practical, you get the fancy math with greek letters :-) and you get assembly language level implementations.
Also I've know a few DIY'ers who read university level materials on their own initiative. YMMV.
Like I said above, the people selling to the client understand little, so nothing complex is sold to them. Besides, you can't really use an algorithm your boss doesn't understand, can you? Knuth is the enticement into the world of programming, but the reality is usually grinding out crap for businesses.
Democracy Now! - your daily, uncensored, corporate-free
You set E to both 0 and 1.
Listen to my music.
YOU CAN'T HANDLE THE KNUTH!
A stunning result probably too recent to have made it in to the book under review: we now have a closed-form formula for the partition numbers.
I never could get more than 10-20 pages into Gravity's Rainbow. And you left out A Brief History of Time.
I forget whether the "advanced academic computing theory" course I first used Knuth in was "CS100" or "CS201", probably the latter. But that was 30+ years ago, and kids these days get to college having been exposed to a bit more than BASIC in high school. And self-taught programmers these days probably don't bother with assembler language unless they're trying to automate toasters (so the "instructables" DIY crowd) or write viruses.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
More like the Art of Failing, in your case.
I respect Knuth for his stand that a man of such technical sophistication does not have to be at odds with faith. In fact, he wrote a pictorial Bible study:
http://www.amazon.com/3-16-Bible-Texts-Illuminated/dp/0895792524
Knuth as quoted from a reviewer:
"it's tragic that scientific advances have caused many people to imagine that they know it all, and that God is irrelevant or nonexistent. The fact is that everything we learn reveals more things that we do not understand... Reverence for God comes naturally if we are honest about how little we know."
Back in 1983, when I was still in school, I published an article in Dr. Dobb's Journal on how to perform various binary operations efficiently. I also sent a letter to. Knuth describing one algorithm in particular: An efficient means of calculating a weighted sum of the bits in a word.
The minute I put the letter in the mailbox I regretted bothering Knuth with such a trivial matter. I was greatly relieved when there was no response; I assumed the letter had circular-filed.
Then about three years ago I got a phone call from someone working with Knuth. They informed me that after 25 years my letter was about to become an exercise in volume 4A, and asking how I wanted my name to appear in the index. And now the book is out, and there it is: Section 7.1.3, exercise 44.
It goes without saying that I was delighted by what happened. But even more than that, I am in awe of the level of scholarship behind this work, where such a little thing as this algorithm was tracked for almost three decades.