Slashdot Mirror
Book Review: Software Build Systems
MassDosage writes "Software Build Systems by Peter Smith is a well written, albeit rather lengthy book that covers the intricacies of systems used to build software. It tries to do this in a technology-neutral manner where possible but covers existing tools like Ant, Make and Scons as a means of illustrating the various concepts covered instead of just focusing on theory. The real world examples illustrate building Java, C/C++, C# and Python software and cover scaling up from small builds with tens of source files up to massive builds with tens of thousands. All of the technologies used are introduced in some depth which newcomers should find useful but experienced build developers may want to skim over." Read below for the rest of MassDosage's review.
Software Build Systems: Principles and Experience
author
Peter Smith
pages
583
publisher
Addison Wesley
rating
7/10
reviewer
MassDosage
ISBN
978-0-321-71728-3
summary
Principles and Experience
Software Build Systems weighs in at a hefty 580 odd pages and covers a lot of ground. It is targeted at developers, managers and build engineers and while there is definitely something for all of these groups, each of them will most likely find themselves skipping sections which are either not of interest to them or are too basic. It is also important to note that this book is not a hands-on tutorial of how to set up a build system from scratch but instead uses a variety of detailed examples to illustrate its concepts with pointers to external documentation for those wanting to actually implement a complete build system of their own.
The book is clearly divided into parts which move from "The Basics" to "The Build Tools" and then on to "Advanced Topics" and finally "Scaling Up". "The Basics" really is basic and most build engineers and developers worth their salt can probably skim read most of this. The fundamentals of C/C++, Java and C# are also covered illustrating a major theme in this book in that it tries to be programming language neutral and just use the different languages as examples to explain various concepts. While this is an inclusive, worthwhile aim it does mean that someone only interested in say, building Java software, will find large parts of the book irrelevant. This applies particularly to the chapters on the various build tools where Make, Ant, SCons, CMake and Eclipse are all covered in painstaking detail. The pros and cons of each of these tools are discussed in a non-biased manner using real world examples where possible. Anyone who has actually written a build system using one of these tools probably won't find much they don't already know but for a manager or someone new to any of these technologies the author provides a a good introduction with mentions of similar tools and pointers for finding out more about each of them.
"Advanced Topics" is where things start to get interesting as concepts like dependency graphs, change detection and version management that the build tools rely on to function are discussed in depth. A lot of the advice in these chapters feels like it comes from practical experience and the best parts are where tool-neutral tips are provided. There is a bit too much detail here as well as the odd digression which feels unnecessary in a book of this length. Do we really need to know the details of lex and yacc in order to create a build system? The author clearly has an understanding of open source development and competently discusses the wild world of building software that may run on a plethora of machines and platforms that the developers have little or no control over as well as software that is built in a more clinical manner for a limited set of environments.
Build systems for massive software projects are covered in "Scaling Up" and the author acknowledges that this probably isn't relevant for everyone. I however think that the first chapter in this section ("Reducing complexity for end users") is the best in the whole book and applies to all build systems, regardless of size or technology. A better title for this chapter would have been "Best practices for build systems" as it doles out plenty of good tips such as how to automatically detect dependencies, what not to keep in a source control system, when to abort a build (early), why to ensure there is always a way to clean up all build-generated artifacts and so on. The gist here is to try to reduce complexity wherever possible and the advice is all very well reasoned and practical. The book wraps up by covering methods for reducing the size of a build and ways to speed up and optimize builds.
Overall this is a very well written, edited and structured book but it does suffer from attempting to cover too much and going into detail on topics which aren't going to be of interest to everyone. A prime example of this is the section on packaging technologies where I doubt that someone concerned with creating Debian packages will find the information on the Nullsoft Scriptable Install System very useful, or vice versa. The same applies to the varying levels of technical detail in the book — a manager may find the introductions to concepts like compiled versus scripted languages enlightening but to most developers this will be old hat. Conversely the intricacies of how Make calculates its dependency graph is probably interesting to a build engineer but most managers will be out of their depth.
This isn't the kind of book most people will read from cover to cover. Instead I recommend skimming through the sections that aren't immediately applicable and just focusing on the parts that discuss the particular build technologies the reader is interested in as well as the more technology neutral parts towards the end. It is obvious that years (if not decades) of real world experience have been distilled in Software Build Systems. It is just a shame that this process wasn't a bit more focused as this could have been a great book, instead of just a good book with some great sections and some sections that most readers will find themselves skimming over.
I was given a copy of this book free of charge by the publisher for review purposes. They placed no restrictions on what I could say and left me to be as critical as I wanted so the above review is my own honest opinion.
You can purchase Software Build Systems: Principles and Experience from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
The book is clearly divided into parts which move from "The Basics" to "The Build Tools" and then on to "Advanced Topics" and finally "Scaling Up". "The Basics" really is basic and most build engineers and developers worth their salt can probably skim read most of this. The fundamentals of C/C++, Java and C# are also covered illustrating a major theme in this book in that it tries to be programming language neutral and just use the different languages as examples to explain various concepts. While this is an inclusive, worthwhile aim it does mean that someone only interested in say, building Java software, will find large parts of the book irrelevant. This applies particularly to the chapters on the various build tools where Make, Ant, SCons, CMake and Eclipse are all covered in painstaking detail. The pros and cons of each of these tools are discussed in a non-biased manner using real world examples where possible. Anyone who has actually written a build system using one of these tools probably won't find much they don't already know but for a manager or someone new to any of these technologies the author provides a a good introduction with mentions of similar tools and pointers for finding out more about each of them.
"Advanced Topics" is where things start to get interesting as concepts like dependency graphs, change detection and version management that the build tools rely on to function are discussed in depth. A lot of the advice in these chapters feels like it comes from practical experience and the best parts are where tool-neutral tips are provided. There is a bit too much detail here as well as the odd digression which feels unnecessary in a book of this length. Do we really need to know the details of lex and yacc in order to create a build system? The author clearly has an understanding of open source development and competently discusses the wild world of building software that may run on a plethora of machines and platforms that the developers have little or no control over as well as software that is built in a more clinical manner for a limited set of environments.
Build systems for massive software projects are covered in "Scaling Up" and the author acknowledges that this probably isn't relevant for everyone. I however think that the first chapter in this section ("Reducing complexity for end users") is the best in the whole book and applies to all build systems, regardless of size or technology. A better title for this chapter would have been "Best practices for build systems" as it doles out plenty of good tips such as how to automatically detect dependencies, what not to keep in a source control system, when to abort a build (early), why to ensure there is always a way to clean up all build-generated artifacts and so on. The gist here is to try to reduce complexity wherever possible and the advice is all very well reasoned and practical. The book wraps up by covering methods for reducing the size of a build and ways to speed up and optimize builds.
Overall this is a very well written, edited and structured book but it does suffer from attempting to cover too much and going into detail on topics which aren't going to be of interest to everyone. A prime example of this is the section on packaging technologies where I doubt that someone concerned with creating Debian packages will find the information on the Nullsoft Scriptable Install System very useful, or vice versa. The same applies to the varying levels of technical detail in the book — a manager may find the introductions to concepts like compiled versus scripted languages enlightening but to most developers this will be old hat. Conversely the intricacies of how Make calculates its dependency graph is probably interesting to a build engineer but most managers will be out of their depth.
This isn't the kind of book most people will read from cover to cover. Instead I recommend skimming through the sections that aren't immediately applicable and just focusing on the parts that discuss the particular build technologies the reader is interested in as well as the more technology neutral parts towards the end. It is obvious that years (if not decades) of real world experience have been distilled in Software Build Systems. It is just a shame that this process wasn't a bit more focused as this could have been a great book, instead of just a good book with some great sections and some sections that most readers will find themselves skimming over.
I was given a copy of this book free of charge by the publisher for review purposes. They placed no restrictions on what I could say and left me to be as critical as I wanted so the above review is my own honest opinion.
You can purchase Software Build Systems: Principles and Experience from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
I wonder how much of the complexity of these things is due to people wanting to be able to do an incremental build of just part of the whole thing.
now we need to go OSS in diesel cars
Especially if you're into trying a little of everything.
A unique way to learn a language: http://languageloom.com
good review but what about continuous integration? Is this books already outdated or was this important part of software build omitted from the book.
Jehovah be praised, Oracle was not selected
I think this is quite useful. This topic is super-important and highly neglected in most Universities and colleges. Newbies where I work as an embedded developer learn very fast that compiling things is actually a lot harder than just writing code.
Why not covering Maven? It's got a totally different way of building applications, together with some concepts like artifacts, dependency, artifact repository, that are not present in other build systems.
I think that it is worth at least a mention.
I have seen build environments that made life so much better. But in most cases I have seen complex build environments that could be described as nothing more than make work projects. When you have a dual processored 8-12 core desktops incremental build times are generally pretty damn short unless you are compiling something massive. So no savings there. Where build environments can help is when you are working with a cluster of monkeys who would simply screw the build up every time they added their so called contributions. After discovering Git I really came to realize that many tools are designed to prevent stupidity at the cost of efficiency. Git would be a great example of a tool that promotes efficiency but won't keep the morons at bay. For many smaller projects build environments are generally moron prevention tools.
But at a certain level of complexity they do really act as a traffic cop of sorts.
The summary doesn't mention cmake or maven. That's bizarre. I hope the actual book covers those; those are really common and useful.
- David A. Wheeler (see my Secure Programming HOWTO)
I doubt that someone concerned with creating Debian packages will find the information on the Nullsoft Scriptable Install System very useful, or vice versa.
While I love living in the world of web apps, where I can simply point people at the latest Chrome or Firefox for their chosen platform, I would certainly find both of these things helpful if I ever need to develop a non-trivial cross-platform native application. In fact, if you're going to cover Debian packages at all, any developer interested in those is probably also (unfortunately) interested in some sort of installer for Windows.
But this is, of course, why I prefer web apps. Otherwise I'd also have to deal with building a Mac .app or .mpkg, whatever iOS and Android support, and an RPM at least, if not also packages for Arch, Gentoo, and others, plus a simple tarball with some sort of install script or instructions for anyone I left out.
Don't thank God, thank a doctor!
Every product I've ever worked on is built with a shell script
There is really no other way to make everything happen
Small products might get away with make or ant or whatever but real products are made from a variety of projects and shell scripts are the only good way to glue them all together.
I've recently started using Tundra, a very light-weight and scalable (CPU core wise) build system, with very fast and reliable dependency analysis written in C, using pthreads/winthreads and taking advantage of SIMD instruction sets. It supports multiple build configurations and variants side-by-side, multiple disjoint build targets/deliverables, and has very DRY LUA-based DSL for project configuration. It doesn't hide CFLAG details, it gets out of the way and lets you precisely control what you're passing to the compiler for each tool chain you want to support. It's released under the GPLv3, was written by a DICE employee and is being used to build Battlefield 3.
I was able to get rid of ~25 CMakeLists.txt that made up thousands of lines of text and replaced it with a single ~350 line tundra build file, in building a 200,000+ LoC multi-platform project and combined with a very stripped down autoconf script for environment detection and config.h generation, was able to support all of my requirements. I had to hack and hack to get CMake to try to support what I needed.
https://github.com/deplinenoise/tundr,a
http://j.mp/emwZ9Z for the PDF presentation slides.
Disclaimer: I have no relations with the author of Tundra or EA/DICE.
Oh please people. Is that really all you care about? Your fave buzzwords? Or a contest to bitch about the most obscure make replacement you can find?
First, that completely misses the point of explaining techniques. This is ironic because IME most people complaining about "classic" make and derivatives --sysV make, bsd's pmake, gnu's NIH gmake, etc.-- completely fail to understand how basic rules work, down to creating massive blobs of rules that really oughtn't work at all and can be replaced with just a few of the most basic of generic rules. That's a failure to attain easily achievable simplicity and thus a clear failure of understanding your own tools. My run-ins with cmake exhibit more of the same, only now through generated rules full of absolute paths and failure to use variables and generic rules seemingly just to artificially raise the importance of the generating tool. Instead of putting effort into creating more convoluted meta tools, spend a couple minutes with the manual and learn how to do it with the existing facilities, please.
Second, it's typical developer short-sightedness, conveniently forgetting that down the road the software needs to be used and thus usable by others, too. To wit: I've seen OSS projects "upgraded" to the very latest and hippest in make-like replacement things just so now the build has colours in it. But that does require that everyone building the system --not just the developer or a few package building specialists either-- now also has to have that thing just to build the software. That's a completely gratuitous added dependency, especially as the project is simple enough that a basic make does just fine and has for years. Replacing it does no good even if it does no harm. So why bother all of your users that build your software themselves with it? It's poor release engineering, is what it is.
Sadly, there's no simple cure for NIH disease.
We're moving back to make. Full disclosure: it was my idea to move to scons in the first place. Dammit.
In the course of every project, it will become necessary to shoot the scientists and begin production.
Software Build Systems, this is really a nice book, I got a lot from this book... now I build a crystal jewelry online store adopt the ways from this book!