Managing Projects with GNU Make

sarumont writes "Every Open Source developer uses or has used GNU make at some point or another. Everyone who has ever compiled a piece of Open Source software has used GNU's make. So what exactly is GNU make and how does it work? The 3rd Edition of 'Managing Projects with GNU Make' tells you all about using GNU make and more."

Comments

Quite the assumption

[dave1g]

I'm pretty sure there are plenty of open source developers who have never touched GNU Make.

you might try these guys.

www.virtualdub.org
www.dscaler.org

And many more.

Open source on windows, OMG it does exist!!!!

Step out of your Linux bubble.

Re:Quite the assumption

[Umbral Blot]

As a Sourceforge admin (see my sig) I can testify from personal experience that Sourceforge has a heavy Linux bias. I do not think that this necessarily comes from the site, but who the site affiliates with, and the people who are drawn to it. First off is FreshMeat. FreshMeat is a great way to create interest in your project and attract developers, and yet windows only projects cannot use it. There is no good reason for this except an anti-windows bias. And secondly the majority of Sourceforge users are Linux users, which means that Linux projects attract more developers and users, which means in turn that their projects shoot up in popularity and name recognition much faster than a Windows project. And finally those Linux projects, being popular have an easy time finding someone to port to Windows, but a low key windows project has a hard time finding someone to port it to Linux since no one knows about it. Hence a kind of self-perpetuating process has arisen that downs out windows projects.

Re:Quite the assumption

[Haeleth]

I'm pretty sure there are plenty of open source developers who have never touched GNU Make.
Open source on windows, OMG it does exist!!!!
Step out of your Linux bubble.

What's Linux-centric about GNU make?

I'm primarily a Windows user, and I used to use GNU make all the time. Until I realised I preferred omake.

Re:Quite the assumption

[zerblat]

Well, you do know that Sourceforge was created by VA Linux (now known as VA Software)? And Freshmeat has always been a directory of software for *nix. There are zillions of sites that only list Windows software. There was a need for a site where you could find *nix software, and Freshmeat filled that gap.

And secondly the majority of Sourceforge users are Linux users

I'm not so sure about that. Certainly, SF is a site for open source-software, and a lot of OSS is *nix-centric (since, well, the most popular open source OSs are Unix-like). On the other hand, if you look at the most popular downloads at SF, IFAICS the most downloaded apps are either cross-platform or Windows only. The same is also true for the most active projects (but perhaps to a lesser extent).

And finally those Linux projects, being popular have an easy time finding someone to port to Windows, but a low key windows project has a hard time finding someone to port it to Linux since no one knows about it.

That doesn't explain why popular and well known projects such as Virtualdub and Filezilla haven't been ported. My guess is that it has more to do with the fact that *nix software is usually made to be portable from start (well, to some extent) and rely on free libraries and developer tools which often already have been ported to Windows.

Or anything in Java, or course

[JavaRob]

There's a *lot* of open source in Java, and basically all of them use ant.

I'm sorry, what was this article about?

Re:Java?

[WolfWithoutAClause]

"Make?" - you insensitive clod!

Significant, invisible characters like tab? Just say no. And that's just the start of your problems...

I still have nightmares.

SCons is much better than GNU Make

[Eric Smith]

I used to use GNU Make extensively, and considered myself to be reasonably close to being an expert with it. But a friend introduced me to SCons, and I've found it to be much easier to use.

SCons has automatic dependency checking built in, supporting many kinds of source files, but if it doesn't have what you need it can be easily extended.

SCons remembers the command line used to compile/build a given file, so it automatically figures out that it should rebuild that file if the command line arguments change. With Make it is very difficult to do that, so "make clean" is used much more often than it should be needed.

SCons is written in Python, and the SConstruct files it uses analagously to Makefiles are fundamentally Python scripts, but you don't need to know Python to use SCons. However, if you do know Python you can easily extend SCons.

SCons integrates well with Steven Ellis' 'nc' network compilation tool (though nc works with make also).

Re:SCons is much better than GNU Make

[slamb]

SCons remembers the command line used to compile/build a given file, so it automatically figures out that it should rebuild that file if the command line arguments change. With Make it is very difficult to do that, so "make clean" is used much more often than it should be needed.

There are a couple other reasons "make clean" is used more often than it should:

• Generating proper dependencies from source is a pain. SCons does this automatically for all of the built-in builders (notably including C/C++), and it has infrastructure for making your user-supplied builders do the same.
• Most makefile setups are recursive. They should not be (see Recursive Make Considered Harmful for a good explanation). It's a pain to make non-recursive make, because the tools aren't set up for it. SCons is.

SCons is written in Python, and the SConstruct files it uses analagously to Makefiles are fundamentally Python scripts

...which is wonderful. The thing about make is that you have to be familiar with so many different syntaxes and APIs to do the simplest thing. There's the shell + make + m4 + autoconf's libraries (workarounds for non-portable shell utilities) + automake's libraries. It's a huge pain, because there's not a single place to look things up. You have to find the appropriate shell utility...then check in the autoconf/automake documentation to see if there are portability wrappers. scons is simpler; you can find most stuff with just their manpage, and possibly Python's documentation when you need to do actual programming in the build system. One (very simple) syntax. Two sources of API documentation.

The autoconf/automake system is nice in that it makes no assumptions about the user's system beyond make - everything is just generated to plain shell scripts and makefiles. But it's such a pain for the developer that it's not worth it. SCons assumes the user has a working Python installation, which makes everything more pleasant. Python provides the same level of functionality with the same interface on every platform.

Re:SCons is much better than GNU Make

[eraserewind]

Most makefile setups are recursive. They should not be (see Recursive Make Considered Harmful for a good explanation). It's a pain to make non-recursive make, because the tools aren't set up for it. SCons is.

This is my biggest (but not only) problem with GNU Make. It does not have support for the theoretically correct way to use it. It's not like the paper is new, or even that the maintainer disagrees with it's findings, but there is no movement at all to add support for the features that would make it easy to use. GNU Make is a project that is totally defined by it's implementation. There is no concept of a "correct" way is should be behaving. There is only the way it behaves.

Re:SCons is much better than GNU Make

[eraserewind]

Well, GNU Make is open-source, so you can extend it, if you want to.

Well, you can extend it, but you more than likely won't be able to get your patch into the main version. The project is extremely conservative about any new functionality. If it's not in GNU make, you can't use your new functionality if you hope to distribute to more than yourself.

It's almost the same as the situation with internet explorer. Until the program with the most market share improves, everyone who cares about working with the marjority of people is stuck with a deficient system.

Build tools

[SunFan]

One thing I find quite remarkable is that in a couple of decades, make is still the only mainstream multi-language multi-platform build tool. The alternatives are either not widely used or are language-specific like Ant. With so many people not liking make, it's suprising an alternative tool hasn't really caught on.

Everyone?

[Brandybuck]

Everyone who has ever compiled a piece of Open Source software has used GNU's make.

It is quite possible to compile a piece of Open Source Software without GNU make. It's not easy, to be sure, since there are so many projects out there that require GNU make (automake doesn't help matters much), but it is possible. There is BSD make, Solaris make, Microsoft's strange nmake, and several others. GNU is but one of many, and it's not even the only free make.

The problem is that the make standard is so tepid that to get a decent make you need to extend it. So what we end up in reality is a lack of a make standard. I can write a complete C program that will compile with any standard C compiler. I can write a powerful bourne shell script that runs on any Bourne compliant shell. But to write a Makefile that will work under all makes is quite difficult.

Newer and better alternatives to make

[Anonymous Coward]

There are newer and much better alternatives to make.

1. boost-build v2 is the absolute BEST if you want to build C/C++ projects with multiple compilers & versions & targets--or even on simple projects that require a one-liner to feed into boost-build2 (normally taking 5-20 lines in GNU make).

Upside for boost-build2? Wraps compiler/linker flags in a generic language for many compilers and versions(gcc, msvc, bcc, etc). Also very easy for simple projects but truly shines on huge and complex projects. Jamfiles can inherit properties , requirements, targets, etc. from parent directories. Very very cool.

Downside to boost-build? Documentation truly sucks compared to other tools. Docs getting better but new users should prepare to unexpectedly find features they could've used to avoid hours of effort.

Boost-build v2 uses bjam but there seems to be a plan to add support for Python.

2. Scons is the next best thing to boost-build v2. The underlying language is Python but you don't have to be a Python expert to use it. And the documentation is much better than boost-build v2. However, it takes many more lines to get things done than boost-build v2 (which isn't all that bad considering boost-build v2 can do things in only 5 lines to replace a 40-line gnu makefile).

3. rake is a make-alternative written in Ruby. For all you recent Ruby converts, be sure to check it out. I love Ruby but I gotta admit, I don't see anything out there being better than boost-build v2 today.

GNU make served us well but it is time to move on to better choices that make us more productive. Just like cvs having served us well but svn and others being a better choice today.

Re:Newer and better alternatives to make

[VZ]

All these tools (and also cmake and A-A-P mentioned in the other reply) are great for developers but not ideal for the end user who almost surely doesn't have them installed on his machine.

If you want to use something as easy on developer but which would still require no additional on the users machine, have a look at bakefile. This is a very useful tool, especially for open source programs where users often have or are asked to rebuild the program from source and installing additional tools is just an extra hurdle for them.

Re:Very few original broken tools replaced

[macshit]

I think the main problem is that many people still want their projects to be pretty portable, at least among unix variants. Once you start requiring people to have "super-duper-make" (requiring pyperl version X.Y.Z) installed, they often just give up on your software rather than deal with the hassle of installing whatever your favorite tool is.

Traditional make is a lowest-common-denominator, however much it sucks. GNU make is slowly becoming widespread enough that it may be a viable alternative.

The autotools are widespread for a similar reason, in that they try very hard to produce portable configure scripts.

[I don't have much experience with autoconf alternatives, but those I've seen -- e.g. imake and Larry Wall's Configure stuff -- suck even more than autoconf. It's sort of like democracy: the worst system there is, except for all the others :-]

A-A-P

[Noksagt]

A-A-P, led by Bram Moolenaar (of vim fame) looks promising too.

Re:Ripping DVDs

[mvdw]

I agree with the fact it's cool - I generate static internal web pages using make + sed + cat on cygwin. I'd hate to regenerate all the pages just because one page changed...

The pages are essentially built up from a header file, a sidebar file, a text file (the only bit that changes regularly), and a footer file. The make file scans the directory for all the *.src files, then generates a page for each *.src file, copies it into the right folder as index.html, making the folder if it's not there already. Easy way to generate consistent pages, although it would've been even better if I could have got css to work properly...