Slashdot Mirror
Autotools
Muad writes "John Calcote is a senior software engineer in Novell's Linux business, who after slogging up the steep learning curve the Autotools triad poses to those packaging software according to the portable GNU conventions for the first time, very kindly decided to make the experience easier to newcomers by sharing his years of experience and carefully crafted bag of tricks. His book is a welcome update to a field that has not seen entries now for a full ten years, so long has been the time since GNU Autoconf, Automake, and Libtool by Gary V. Vaughn, Ben Ellison, Tom Tromey, and Ian Lance Taylor hit the shelves. Unfortunately, the publishing industry is driven by the need to turn a profit to fund its endeavors, and specialist items like this book are not obvious candidates for volume selling - which is a credit to No Starch Press' willingness to venture down this path." Keep reading for the rest of Federico's review.
Autotools: A practitioner's guide to GNU Autoconf, Automake, and Libtool
author
John Calcote
pages
360
publisher
No Starch Press
rating
8/10
reviewer
Federico Lucifredi
ISBN
1593272065
summary
Teaches how to master the Autotools build system to maximize your software
The book opens with John's experiences in adopting the Autotools, and quickly offers what is in my view a very important word of caution that is often lacking in the few tutorials I have seen on the Net: the Autotools are not simply a set of tools but foremost the encoded embodiment of a set of practices and expectations in the way software should be packaged the GNU way. While it is acceptable for beginners not to know what these expectations are, the right frame of mind to approach the Autotools is to focus on learning what way the Autotools operate, what they are trying to accomplish, and why. Attempting to use the Autotools without understanding the bigger picture will lead to very high amounts of pain, as it is one of the toolsets most difficult to adapt for use separate from the policies they represent, so strongly are these conventions embedded in their fabric. With this understanding, it becomes possible to generate extensive configurations with a few lines of Autoconf or Automake - without this understanding, it very quickly becomes a battle to force a round peg into a square tool.
John's style is more extensive and takes a longer path to the "technical meat" of the problem than the 10-year old alternative, but in this reader's opinion it flows significantly better as there is an underlying story, a thread that connects the bits of what is otherwise a pretty arid subject. For those masters of shell-fu, this book is a page-turner, while for mere mortals it is a good, approachable, path into a difficult skill.
The book is structured around the packaging of two different projects, the first being a simplified "Hello, World" project to provide a digestible introduction to the processes and technology of the Autotools, while the second representing the full-blown packaging of a complex, real-world project (the FLAIM high-performance database). This is a very good approach, breaking the theory into many practical examples of practice, and providing many ready-made bits that the rest of us can start our own configuration build files from. The result is a first half providing a gentler, streamlined introduction to the subject matter, before the full jump into the gory details of the most complex possibilities the toolset offers. While it must be noted that John attempts to keep away from those most fine details which "may be subject to change" between minor releases of the tooling, which is doubtlessly good for both our scripts' and the book's shelf life, it must be observed that he does not shy away from very dense (and otherwise utterly undocumented) material, such as the use of M4 macros in Autoconf, something a colleague of mine once pointed to me as "the one more reason I'd rather chew broken glass than deal with Autotools".
Assuming you have the requisite knowledge of Make, Shell scripting (particularly Bash), and GCC that are essential to a developer, packager, maintainer or buildmaster of a Linux, BSD or *NIX project, or that you are on your way to achieving those skills, this is a book that belongs in your shelf, right next to the RPM documentation. This is material for experts or experts in the making, but in my opinion you will find no better introduction to this complex subject. I had it on my wish list well before it was ever printed, and its presence on my desk caused several other developers in my office to order their copies pretty much on the spot upon finding out of its existence. Either as a learning tool for a skill you are trying to attain, or as a reference to turn to when faced with the complexities of this unique set of tools, this book is well worth its price tag.
I certainly hope this is not the last publication we see on the Autotools in this decade, but either way, it is a good start indeed - and my hope is that the publisher will refresh the title when an update is warranted, without waiting ten years!
Federico Lucifredi is the maintainer of man (1) and a Product Manager for the SUSE Linux Enterprise and openSUSE distributions.
You can purchase Autotools: A practitioner's guide to GNU Autoconf, Automake, and Libtool from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
John's style is more extensive and takes a longer path to the "technical meat" of the problem than the 10-year old alternative, but in this reader's opinion it flows significantly better as there is an underlying story, a thread that connects the bits of what is otherwise a pretty arid subject. For those masters of shell-fu, this book is a page-turner, while for mere mortals it is a good, approachable, path into a difficult skill.
The book is structured around the packaging of two different projects, the first being a simplified "Hello, World" project to provide a digestible introduction to the processes and technology of the Autotools, while the second representing the full-blown packaging of a complex, real-world project (the FLAIM high-performance database). This is a very good approach, breaking the theory into many practical examples of practice, and providing many ready-made bits that the rest of us can start our own configuration build files from. The result is a first half providing a gentler, streamlined introduction to the subject matter, before the full jump into the gory details of the most complex possibilities the toolset offers. While it must be noted that John attempts to keep away from those most fine details which "may be subject to change" between minor releases of the tooling, which is doubtlessly good for both our scripts' and the book's shelf life, it must be observed that he does not shy away from very dense (and otherwise utterly undocumented) material, such as the use of M4 macros in Autoconf, something a colleague of mine once pointed to me as "the one more reason I'd rather chew broken glass than deal with Autotools".
Assuming you have the requisite knowledge of Make, Shell scripting (particularly Bash), and GCC that are essential to a developer, packager, maintainer or buildmaster of a Linux, BSD or *NIX project, or that you are on your way to achieving those skills, this is a book that belongs in your shelf, right next to the RPM documentation. This is material for experts or experts in the making, but in my opinion you will find no better introduction to this complex subject. I had it on my wish list well before it was ever printed, and its presence on my desk caused several other developers in my office to order their copies pretty much on the spot upon finding out of its existence. Either as a learning tool for a skill you are trying to attain, or as a reference to turn to when faced with the complexities of this unique set of tools, this book is well worth its price tag.
I certainly hope this is not the last publication we see on the Autotools in this decade, but either way, it is a good start indeed - and my hope is that the publisher will refresh the title when an update is warranted, without waiting ten years!
Federico Lucifredi is the maintainer of man (1) and a Product Manager for the SUSE Linux Enterprise and openSUSE distributions.
You can purchase Autotools: A practitioner's guide to GNU Autoconf, Automake, and Libtool from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
... they should be replaced by something else.
Seriously, the whole autotools thing is the worse design EVAR.
I suppose it's nice that someone writes a book like this, since a lot of existing projects use autotools (or more commonly, try to by means of copy/paste and cargo-cult based build scripting).
But autotools should really be phased out. It solves a lot of problems that aren't problems anymore, and makes a helluva lot of new ones in the process. There are a lot of up and coming build systems to challenge it, and then there's CMake which is an OK compromise between those and practicality.
xkcd is not in the sudoers file. This incident will be reported.
Good heavens, I'm all for sentences with body, but this is terrible. I actually stopped reading the article after the second one. You know what this site could use? Editors.
Did I miss any novell/suses?
Do you even lift?
These aren't the 'roids you're looking for.
I understand that adoption/marketing/historical factors may have justified this particular approach to cross-platform builds of C/unix apps, but is this such a big problem that it requires 5-6 languages to solve (counting the syntax of C, sh, configure.ac, Makefile.am, makefile and possibly other intermediate formats)? Sheesh...
-1, Too Many Layers Of Abstraction
That would be home made Kombucha.
decided to make the experience easier to newcomers by sharing his years of experience and carefully crafted bag of tricks.
Even better would be reading that this gentleman had gotten behind efforts to make working with the tools easier. Simply teaching me tricks though welcome, is not good enough. Working with the tool(s) still is difficult.
with a nail than use autotools
Poul-Henning Kamp of the Varnish about autotools: "Did you call them autocrap tools ? Yes, in fact I did, because they are the worst possible non-solution to a self-inflicted problem." Read more at: http://www.varnish-cache.org/docs/2.1/phk/autocrap.html
...that you should never use autotools.
The second rule is that is better to use something else like scons.
Shouldn't that be shallow learning curve? Steep would imply quick effortless progress towards expertise (at least if you put the independent variable on the abscissa and dependent on the ordinate as is customary). A shallow learning curve now would imply slow progress...
Those who don't understand Automake are doomed to repeat the mistakes of build systems that are not designed like Automake.
The one language that actually drives all of Automake is ML. Funny it is the one language you didn't list, but you listed a bunch of the high level macro files that get expanded with ML. If you don't know sh, then you shouldn't be programming on the command line (stick to an IDE that does the compilation for you). C isn't required for any component of Automake. If you write a makefile with automake, you made a big mistake, as Automake writes the makefiles for you.
There are other systems out there which are easier to use, but there's only a handful that does things in a manner that is highly reliable and portable on many platforms. Those that do strive for such goals end up operating like Automake, but often they do so without allowing such easy access to the internal guts of what is really going on.
Yes, I like cmake too, but bashing Automake just because you don't understand it is just the computer equivalent of name-calling.
Let's Rooolllll out!
Unfortunately, the publishing industry is driven by the need to turn a profit to fund its endeavors, and specialist items like this book are not obvious candidates for volume selling - which is a credit to No Starch Press' willingness to venture down this path.
The situation is not as dire as this post seems to suggest. Print on demand is an option for a book such as this. Getting a publisher like No Starch is great since they will provide traditional editing, review and marketing services however the publisher is not necessarily making any great investment since they too can take advantage of a print on demand type of approach. There is no longer a need to print a large number of books up front.
Hmmm... are you sure that you didn't mean the macro language M4? I thought that ML was the pure functional ancestor to languages like Haskell.
Anyway, I played around with M4 a little bit because I thought it looked handy for a few things. It has a deceptively simple specification that only takes a few pages, but it's one of the most extreme examples of "emergent behavior" I've encountered. Even simple tasks rapidly become mind boggling due to the deceptively tricky nature of recursive text substitutions and quoting. It's a real brain teaser of a language. It does seem like it would be a nifty tool if I spent enough time to really figure it out.
Obligatory link to a good autotools alternative: CMake. And my CMake tutorial, Learning CMake.
http://www.mcs.anl.gov/~rgupta/calcote_autotools_guide.pdf
Friends don't let friends use autotools.
For the love of insert concept of deity recognised
Why invent Makefile writing scripts or even programs when make and Makefiles can easily do all that is required for cross-platform (and cross-target) compilation? http://sourceforge.net/projects/mk-configure/
I'll throw this bit of fuel on the flame-fest, in the form of a question:
Does anybody else find that Autotools based projects, while being very cross-platform, are almost impossible to actually cross-compile?
I do embedded systems work, and the embedded universe is moving to Linux as the kernel, and quite frequently a sub-set of the Gnu environment for the runtime. So you get things like BitBake, OpenEmbedded, and Angstrom, which attempt to enable you to build a complete system from sources. However, what all these environments do is run QEMU to emulate the target CPU in order to do the builds.
Now, that's stupid IMHO: I have this VERY FAST multi-core workstation, and I am
1) Throwing away all but one core, and
2) Hobbling that core emulating a completely different architecture (e.g. emulating an ARM to build for the OMAP).
Much of the - I shall use the term "work" although a more bovine-scatological term springs unbidden to mind - involves writing "recipes" for BitBake to work around issues in Autotools.
Much of Autotools seems to me to assume that the machine building the code will be the machine running the code - or at least, a machine of the same type as the machine running the code. So all the Autotools "magic" to deduce structure layouts, word sizes, byte orders, and such all assume that you can
a) compile the code using the host's C compiler,
and
b) Run the resulting probe programs on the host.
Both of which are totally FALSE for cross-compiling.
I used to say that you could design a new CPU that nobody had ever seen, and once you ported Binutils, GCC, and Linux to it, you could build an entire functioning distro for it, just by iterating over the various source packages for and cross-compiling. I know better now: most of the source packages out there DO NOT cross compile worth a damn! They might BUILD NATIVELY on a wide range of architectures, but not cross-compile.
www.eFax.com are spammers
And there's worse -- supporting multiple simultaneous build targets. Most of my stuff I build {optimized, debug, optimized-profiling} x {gfortran/gcc, g95/gcc, sunf95/suncc, ifort/icc} for a set of 12 simultaneous build-targets. Conventional build systems do not support multiple simultaneous build-targets well.
"My opinions are my own, and I've got *lots* of them!"
please note that all current 'replacements' are totally wrong and actually work only as puny build systems, not supporting any of the great portability benefits that autotools give. Scons, cmake, whatever else depend on their working installation on _build_ machine. This is wrong, only working shell+make+gcc should be needed to actually build software.
So...
Is there ANY good "replacement", preferably lightweight, with this great virtue? As far as I know, there's none.
Take time to figure it out.
I've been an occasional, uninformed user of m4, but then necessity and a two hour train journey was enough to let me write a very flexible m4 based provisioning system for our Linksys phones. Moves adds and changes are now trivial :-)
can compensate for the low quality of this garbage, not to mention its poor performance (some builds spend more time running configure than actually compiling, installing, and tarring up the resulting run-time and devel packages!)
These tools have to be redesigned from the ground up by someone who understands that free software has made large inroads into the embedded world where it needs to be, doh, cross-compiled.
For my own projects, I develop a carefully-crafted configure shell script by hand and recommend everyone do the same.
The script has carefully developed and tested support for (1) cross-compiling, (2) configuring/building in a separate directory, and (3) installation into a temporary package directory.
Furthermore, this script should ONLY test the features that are actually needed by the program. (The program should assume a reasonably modern system; don't bother testing for obscure bugs in System V release 3, okay?) The script should make sure that it tests only header files and compiling with the toolchain that it is given. The configure test programs programs should never accidentally include something in the build machine /usr/include, or link something from /usr/lib!
Configure scripts should never run any program that they compile because it may be the wrong architecture, and they should not have an "if cross compiling" check which disables their features when cross-compiling and substitutes dumb defaults! Quite simply, implement the tests so that cross-compiling is not needed.
For instance, instead of making a C program which outputs values with printf, make it so that the values are used as initializers for static data. Then use the cross-compiling toolchain's "nm" utility to extract the values from the compiled object file. You don't have to run a program to know how wide a (void *) is. Just do "static char size_of_void_star[sizeof (void *)]". Compile it, and then interrogate the .o to discover what is the content of the size of the data region named by size_of_void_star!
Look at the stupid configure script for GNU bash. When crossing, it assumes totally pessimistic values for all checks that can't be done when cross-compiling. Unless you explicitly override the ac_* variables yourself, you get a shell which has no job control! The bash build assumes that your kernel has a tty system dating back to the middle 1980's.
Nobody should dig through a configure script to find out what broken tests they have to override by setting the variables manually.
Shell code should never be generated by m4 macros, let alone ones which frequently change.
Look at the stupidity of thisl. Suppose you want to produce a minimal patch for some open source project that uses autoconf. Suppose that as part of your patch you have to enhance the configure script with some new options. To do that, you have to modify configure.in. But projects ship with the generated configure script. So you want to regenerate that, right? Oops, your version of autoconf is different, and so you get a HUGE diff. Worse yet, the generated configure script breaks!
So people end up with ten different versions of autoconf installed, so they can always run the right one which matches what the configure script was produced with that is in the tarball.
Build configuration from scratch is not difficult. It is easy. Autotools do not simplify anything. They monstrously complicate something which is already simple and doesn't require simplification or automation. Using the Autotools is a false economy. You may think you are getting something for free and saving time, but in the end you will spend more time over the life of your project wrestling with this garbage (and also waste the time of countless other people you don't even know) than if you just carefully wrote a small configure script by hand.
quagmire got the concept right, but seems to be pretty dead by now ...
Packt are way too fond of publishing any old crap just to get an early book out on a niche topic. They have been known (by me) to randomly spam people who blog on their books' topics, asking them to review the book. Once they get you interested in the free book they start hounding you for a review on your blog. I wouldn't have minded TOO much if their books had anything worthwhile to write about.
Yeah, that whole GNU Compiler Collection thing is just a waste of time. Can't compile a binary out of anything. Thankfully Microsoft were forced to give away a free version of Visual C++ for some reason, having killed all the commercial competition. Also, Intel have released a compiler for linux, which you can actually run in a shell that just happens to be pre-compiled somehow. So, thankfully there are alternatives to GCC.
that's why 2 styles of pkg-ing
many guys blame relative path for security problems
while many devs need this
I whole-heartedly agree with you - and I'll make another point. Much of what auto* was designed to do was to allow porting the Gnu toolchain over to a target which did not have it - e.g. porting binutils and GCC to some Unix box that did not have them yet, thus they had to assume as little about the target system as possible to allow bootstrapping. However, the idea was that once you HAD binutils, GCC, and libc, you could use them, and have a predictable standard environment.
Now-a-days, that is almost a gimme - any new system will almost certainly have GCC as a part of the bring-up. So why not move on to a pkg-config style system, where there is an executable that can be run for the target platform, that can answer questions like "How big is a pointer? an int? A float? Do you have these library routines? What do I do to dynamically link a program?" pkg-conf does a wonderful job for the programs it knows about, why not extend it to answer all the myriad of standard questions that auto* seems to ask every time it's run?
Sure, you'd have to generate that program for a new target - but guess what? most of the "questions" are ones GCC already knows the answer to, so why not just make GCC be the "pkg-config" for basic CPU arch type questions? Then, even if I am cross-compiling for a Floobydust300 rev B processor, all I have to do is 'floobydust-unknown-linux-gcc --whatis "sizeof(int)"' and I have my answer. (alternatively, dump an XML file, or a key=value file, or header, or any number of other approaches).
www.eFax.com are spammers
I have to know. What the hell is the point of this shit? I mean, if it had a link to something, or a shout-out to GNAA, or just anything, I would understand.
"John Calcote is a senior software engineer in Novell's Linux business" He does not. He works for identity business, not SUSE.
Why does it depend on expat?
is that it's got ugly syntax, effectively no cross-compiling support, and less-than-helpful documentation. And its generated Makefiles sometimes miss changes in header files, forcing you to "make clean".
But yeah, it's still a good alternative to autotools.
Is it harder to go up the cliff than up the hill? The cliff. Which one is steeper? The Cliff. Therefore it's harder to climb a steep learning curve to reach the pinnacle of excellence than a shallow one, but it may take longer anyway to reach the same level of expertise.
To everyone recommending CMake: have you actually viewed a reasonably-sized CMakeLists file? It's disgusting, at least as bad as configure.ac. CMake's only selling point is that it supports Windows better. (I can't comment on if it supports it particularly well.) And from the GNU folks POV, why should they support a non-free OS that more than doubles the maintenance load?
You look beautiful! Incidentally, my favourite artist is Picasso.
You're right, M4. Funny, a recent news article must have left me with ML on my mind.
Mbt Shoes Free Shipping can benefit you from toe to head. The key of mbt shoes for running is its sole, which is the corn of Masai Sensor. The patented sole of whrer to buy mbt shoes creats instability that gives you a great workout. Who makes mbt shoes feature a sporty look with a sleeker-profile. Wear mbt shoes is remarkable shoe. Regarding to its special series, mbt shoe manufacture tone and shape the body,which make you more healthier in summertime.Therefore, mbt shoe official website keeps rising violently and now you see mbt shoes cheap everywhere.
http://www.mbt-shoes.com