Slashdot Mirror
Why Switch a Big Software Project to autoconf?
woggo queries: "I'm a CS grad student working on a large research project (over 1 million lines of code, supported on many platforms). The project has been under development for several years, and the build system is nontrivial for end-users. We'd like to make it easier to build our software, and I'm investigating the feasibility of migrating to GNU autoconf. I need to demonstrate that the benefits of autoconf outweigh the costs of migrating a large system of makefiles with a lot of ad-hoc kludge-enabling logic. Has anyone made a similar case to their advisor/manager? Does anyone have some good 'autoconfiscation' war stories to share? (I've already seen the Berkeley amd story and the obvious links from a google search....)" Depending on the intricacies of the build process, such a conversion might take an awful lot of work. It might be easier to put a nicer face on the "nontrivial build process", although there is something to be said for the ease of "./configure; make; make install"
There's even more to be said for running short test programs which discern the same things that configure does, and then writing the correct .h file to disk. That way, you can write a makefile which ALWAYS works. The trouble that I have with configure is that it creates a makefile which differs from machine to machine, so that debugging it is quite difficult. Plus, if configure gets something wrong, you can make and make and make all day long, and you'll never fix it. So even though configure, in theory, saves you from having to maintain a makefile, in reality it means that everyone who runs configure may have to debug the makefile.
In other words, it's better to have one makefile, and fix the problems in the makefile, than to have to fix problems in configure or your configure specification.
If you want an example of how this can work, look at Dan Bernstein's djbdns, or qmail. On both of these, "make" Just Works(tm).
-russ
Don't piss off The Angry Economist
Users had to be taught about setup.exe (or install.exe or whatever) once. And these days, autorunning CDs are removing the need to do that again. Remember, most things that you claim are easier in Windows are only easier because you've already been taught to do them. Teach someone how to use rpm/dpkg/apt and installing software with them will become just as easy.
As a fairly newbie Linux user it is not doing rpm or make that is the problem. I mean telling someone that rpm is the Linux version of setup is not the problem. The problem is: 1. Dependecies. 2. When the make doesn't work. My suggestion for (1) is to just include all the files you need in the one download. Sure it'll make the rpm much bigger (which is why you include alternative rpms for power users) but I'm telling you, confront a newbie Linux user (even one who isn't totally computer illiterate) with a list of 20 dependencies and you'll lose them. I mean just putting all the rpms in .tar.gz with a warning that you need certain minimum requirements eg. Red Hat 6.2 with a readme of which order to install the rpms would make life much easier. Give them all the files needed and step-by-step instructions (as in they can cut and paste the command line instructions) and they'll manage. I mean Windows install programs don't ask you to download 20 or so .dlls. And yes, sure Debian has a wonderful system around this but hey try to get newbie Linux users to actually work out how to install and run Debian in the first place...
Just do the conversion and see what's it like.
The talking about takes longer than doing it.
Then you'll really know.
I think the important difference you're forgetting is that Windows installers are installing binaries, while configure and make are building source code. A better analog in the Linux world is rpm or apt-get. There are very few programs these days that don't come with prebuilt binaries for several Linux configurations. And, no, I don't think rpm/apt-get (graphical, if you like) is much harder than setup.exe (modulo dependency satisfaction in rpm, but I would hardly consider the fact that most Windows programs are distributed with (possibly out of date) versions of a lot of libraries you already have to be advantageous).
I would argue that one of the functions of configure (aside from detecting hella shit about your build environment) is to "ask a few questions." Just doWe have a monstrously large software project with home grown kludges for building on multiple platforms that is a mess of shell scripts and makesfiles. It's ugly and the time required to keep fixing and extending this system is a heavy burden. Plus, it's only tolerable to use.
I've used autoconf a little on a different smaller project and I know this much about it.
When autoconf works, it works great. As a user, there's great delight in typing those 3 magic commands and having a whole series of feature tests fire off and configure the build process. And, as a developer, every time the system works (and it works more than kludge systems), it saves me from hassles of porting to platform de jour. If you look at some fairly complicated systems such as Octave or teTeX build, it's nothing short of amazing.
However, that said, autoconf is complicated. And, you really have no choice but to learn a fair amount of the complications to deal with the issues that will inevitably come up during cross platform testing. I've mostly learned autoconf, a little of automake and almost none of libtool. The m4 macro language for defining the autoconf macros has syntax that always gives me nausea.
If you implement autoconf for your project, great. But, expect a substantial fraction of your time-savings from it to be dedicated to becoming more of an expert at autoconf, m4, sh, etc.
autoconf is better than imake, IMHO, but it sure is short of being as good as it should be. Large projects can justify the time for a person learning the intracacies of autoconf; small projects cannot, unless you happen to know it already. Learning autoconf is kind of like learning nroff just to write a man page - the time investment is more justifiable if you use your expertise to write more than one of them, or to write a really important man page.
P.S. Is there any chance that the software carpentry project will come out with something soon? Some of their early submissions for improved build tools really sounded intriguing, like Tan, an XML based description.
"Provided by the management for your protection."
i've been using these tools for about 2-1/2 years. autoconf and automake have saved me a lot of hassle, i think. but i cannot recommend that anyone in their right minds uses libtool. libtool starts from a reasonable premise: the creation of shared libraries is done in very different ways on different platforms. it continues on to try to provide some standard method for developers to use when they need to do this. the problem is that for one reason or another, its gone way too far in a direction that with a little bit of objectivity is really quite bizarre. libraries end up being built in hidden directores, and are initially replaced by text files of dependencies. you have to run "libtool -gdb" if you want to debug your program without installing the libraries first. there are many wierd and obscure consequences to this design. and don't get me started on "libltdl", the libtool "portable dlopen() wrapper". in recent versions of this library (which were full of really stupid bugs) there are now hooks to replace malloc/realloc/free, for reasons that nobody on the libtool mailing list understood (probably some windows bullshit to do with calling malloc at some point during runtime linking, but who knows?). libltdl goes as far as "emulating support for run time linking even on platforms that don't support it". they are proud of this bizarre "accomplishment". libtool's basic problem is that it starts out trying to solve a goal that very few people face: trying to build a shared library on multiple platforms. because this is a very complicated task, libtool is necessarily complex. if you are writing a shared library for one or two platforms, particular where they are both fairly similar, using libtool to do this rather than doing it with your own understanding of the process can provide hours of frustration, confusion and despair. i don't see anyway forward for libtool. the sensible way to solve shared library problems is to wrap the actual linker in a new executable that operates in a standard way across platforms. this would hide the complexities and oddities of the process away from autoconf/automake and you.
This is not a no-brainer guaranteed win. It is easy to make autoconf do horrible, silly, things - which make your code less portable. As always, the very first thing you do should be to make your code as portable as possible to begin with; this is much, much, more efficient than any kind of configuration system.
Current versions of configure seem to sometimes produce makefiles which can't be used with classic Berkeley make, and depend on GNU extensions...
My blog: http://www.seebs.net/log/ --- My iPhone/iPad app: http://www.seebs.net/seebsfrac/
libtool compiles every file twice because the options needed for statically linked code aren't the same as the ones needed for dinamically linked objetct files.
.o file that will be used in a shared library is built, you need to pass -fPIC in order to tell gcc to create "position independent code". With this options gcc uses a register to provide this feature. As this register is lost for normal variable optimizations, statically linked code should not be compiled with this option turned on.
When a
This is not true at all. There's nothing in autoconf that mandates sloppy programming.
the argument 'configure ; make ; make install' is easy is stupid, as well. y'know why? 'cause 'make install' is easier.
OK, but what if I want to set a different prefix, turn on optimisation, enable debugging, and tell the build where Qt is installed ? I've seen straight make builds where this information is distributed across several Makefiles, and that's a pain.
i'd offer just a little extra advice: do not require GNU-specific stuff. don't use any of the gmake-specific features or gcc language extentions.
This I agree with.