Slashdot Mirror


Managing Projects with GNU Make

sarumont writes "Every Open Source developer uses or has used GNU make at some point or another. Everyone who has ever compiled a piece of Open Source software has used GNU's make. So what exactly is GNU make and how does it work? The 3rd Edition of 'Managing Projects with GNU Make' tells you all about using GNU make and more."

49 comments

  1. Quite the assumption by dave1g · · Score: 3, Informative

    I'm pretty sure there are plenty of open source developers who have never touched GNU Make.

    you might try these guys.

    www.virtualdub.org
    www.dscaler.org

    And many more.

    Open source on windows, OMG it does exist!!!!

    Step out of your Linux bubble.

    1. Re:Quite the assumption by Umbral+Blot · · Score: 3, Insightful

      As a Sourceforge admin (see my sig) I can testify from personal experience that Sourceforge has a heavy Linux bias. I do not think that this necessarily comes from the site, but who the site affiliates with, and the people who are drawn to it. First off is FreshMeat. FreshMeat is a great way to create interest in your project and attract developers, and yet windows only projects cannot use it. There is no good reason for this except an anti-windows bias. And secondly the majority of Sourceforge users are Linux users, which means that Linux projects attract more developers and users, which means in turn that their projects shoot up in popularity and name recognition much faster than a Windows project. And finally those Linux projects, being popular have an easy time finding someone to port to Windows, but a low key windows project has a hard time finding someone to port it to Linux since no one knows about it. Hence a kind of self-perpetuating process has arisen that downs out windows projects.

    2. Re:Quite the assumption by dave1g · · Score: 1

      I have no problem with the linux bias on sourceforge (personally have only dabbled in linux, but Im taking a begining linux user course and hope to some day make the switch).

      I just wanted to point out that not all open source development is done on linux.

      The fact that the author thought that all open source developement has something to do with GNU shows he is just another close minded linux fanboy.

      While arguably we can thank GNU for the existence of the free software movement, since then it has branched out all over the place.

      Open Source is merely a way of writing programs.

      Ask linus himself.

      And as others have pointed out, Java developement is another large catagory of opensource that doesn't touch GNU make.

    3. Re:Quite the assumption by Haeleth · · Score: 4, Insightful

      I'm pretty sure there are plenty of open source developers who have never touched GNU Make.
      Open source on windows, OMG it does exist!!!!
      Step out of your Linux bubble.


      What's Linux-centric about GNU make?

      I'm primarily a Windows user, and I used to use GNU make all the time. Until I realised I preferred omake.

    4. Re:Quite the assumption by dave1g · · Score: 1

      Nice :-)

    5. Re:Quite the assumption by zerblat · · Score: 2, Insightful
      Well, you do know that Sourceforge was created by VA Linux (now known as VA Software)? And Freshmeat has always been a directory of software for *nix. There are zillions of sites that only list Windows software. There was a need for a site where you could find *nix software, and Freshmeat filled that gap.
      And secondly the majority of Sourceforge users are Linux users
      I'm not so sure about that. Certainly, SF is a site for open source-software, and a lot of OSS is *nix-centric (since, well, the most popular open source OSs are Unix-like). On the other hand, if you look at the most popular downloads at SF, IFAICS the most downloaded apps are either cross-platform or Windows only. The same is also true for the most active projects (but perhaps to a lesser extent).
      And finally those Linux projects, being popular have an easy time finding someone to port to Windows, but a low key windows project has a hard time finding someone to port it to Linux since no one knows about it.
      That doesn't explain why popular and well known projects such as Virtualdub and Filezilla haven't been ported. My guess is that it has more to do with the fact that *nix software is usually made to be portable from start (well, to some extent) and rely on free libraries and developer tools which often already have been ported to Windows.
      --
      Please alter my pants as fashion dictates.
    6. Re:Quite the assumption by Anonymous Coward · · Score: 0

      What's Linux-centric about GNU make?

      The fact that it works like butt on Windows? You forgot to mention that part. About how it works like butt. There are WAY better options on Windows.

    7. Re:Quite the assumption by FictionPimp · · Score: 1

      The next version of filezilla supports linux. I have downloaded and compiled the alpha, it works great.

  2. Java? by Anonymous Coward · · Score: 0

    You insensitive clod!

    1. Re:Java? by WolfWithoutAClause · · Score: 2, Insightful
      "Make?" - you insensitive clod!

      Significant, invisible characters like tab? Just say no. And that's just the start of your problems...

      I still have nightmares.

      --

      -WolfWithoutAClause

      "Gravity is only a theory, not a fact!"
    2. Re:Java? by Anonymous Coward · · Score: 0

      Visibility of tab characters is a function of your viewer/editor, not the character - emacs has a "make mode" that stops you worrying about tabs.

      Though I actually do hate the tabs myself (I'd prefer a nice clean sexp syntax), editing make files isn't hard unless you're doing it in freaking NOTEPAD.EXE or something.

    3. Re:Java? by Anonymous Coward · · Score: 0

      Visibility of tab characters is NOT a function of your viewer/editor. There is no character for tab. Period. It's an alignment character, not a graphic one. Your editor may highlight a tab for you, for example search on tab in vim and they are all highlighted, but the character is invisible. By design.

      Yes, make sucks for using tabs. And basically needing a file per directory is also retarded. And "Your editor should have make support" is an absurd argument. Basically there's nothing good about make at all...

    4. Re:Java? by Anonymous Coward · · Score: 0
      You mustn't be a programmer - of course visibility of tab characters is a function of the viewer/editor - if the editor is written to show "X" everywhere "a" occurs, for example, it will, well, just do that. Similarly, if it is configured to show
      ". "
      everywhere TAB char is encountered, it will, well, just do that!

      Sure, you may conceive of ASCII TAB as a nonprinting whitespace character - but that's up to the program. You might consider the program "wrong" - but it won't stop existing because of that (unless you're powerful enough to have all copies erased and the histories rewritten or something, in which case - hi, Bill! spare a dime? I'm old enough to remember you actually saying "640K ought to be enough", even though you've apparently managed to muddy the waters of history enough to cause other people to doubt it ever happened...)

    5. Re:Java? by Anonymous Coward · · Score: 0

      Huh? What? ''If an editor is written to show "X" everywhere "a" occurs, for example, it will, well just do that.'' Why the frack would an editor do that? Oh yeah, because in this case "a" is an invisible tab character and "X" is something we can see. So make is still using an invisible character. And make still sucks.

    6. Re:Java? by Anonymous Coward · · Score: 0

      Huh? You just don't get it, do you? ALL characters are invisible until they are made visible. Which characters are made visible is up to the program, and ultimately therefore the programmer (as regards "ultimately the programmer", except in the case of random or evolved programs, obviously).

  3. Or anything in Java, or course by JavaRob · · Score: 2, Informative

    There's a *lot* of open source in Java, and basically all of them use ant.

    I'm sorry, what was this article about?

    1. Re:Or anything in Java, or course by Anonymous Coward · · Score: 0
      I'm sorry, what was this article about?
      It wasn't about Java. Now go crawl back into your hole Javaboy.
    2. Re:Or anything in Java, or course by JavaRob · · Score: 1

      Strangely enough, I'm crawling back into... the PHP code I'm working on. Taco, can we *please* change usernames but keep the UID?

  4. SCons is much better than GNU Make by Eric+Smith · · Score: 4, Informative
    I used to use GNU Make extensively, and considered myself to be reasonably close to being an expert with it. But a friend introduced me to SCons, and I've found it to be much easier to use.

    SCons has automatic dependency checking built in, supporting many kinds of source files, but if it doesn't have what you need it can be easily extended.

    SCons remembers the command line used to compile/build a given file, so it automatically figures out that it should rebuild that file if the command line arguments change. With Make it is very difficult to do that, so "make clean" is used much more often than it should be needed.

    SCons is written in Python, and the SConstruct files it uses analagously to Makefiles are fundamentally Python scripts, but you don't need to know Python to use SCons. However, if you do know Python you can easily extend SCons.

    SCons integrates well with Steven Ellis' 'nc' network compilation tool (though nc works with make also).

    1. Re:SCons is much better than GNU Make by noselasd · · Score: 1

      I'll second this. While scons doesn't solve every damn problem, it's
      much easier to use over make.
      I'm not going to look back.

    2. Re:SCons is much better than GNU Make by slamb · · Score: 4, Interesting

      SCons remembers the command line used to compile/build a given file, so it automatically figures out that it should rebuild that file if the command line arguments change. With Make it is very difficult to do that, so "make clean" is used much more often than it should be needed.

      There are a couple other reasons "make clean" is used more often than it should:

      • Generating proper dependencies from source is a pain. SCons does this automatically for all of the built-in builders (notably including C/C++), and it has infrastructure for making your user-supplied builders do the same.
      • Most makefile setups are recursive. They should not be (see Recursive Make Considered Harmful for a good explanation). It's a pain to make non-recursive make, because the tools aren't set up for it. SCons is.

      SCons is written in Python, and the SConstruct files it uses analagously to Makefiles are fundamentally Python scripts

      ...which is wonderful. The thing about make is that you have to be familiar with so many different syntaxes and APIs to do the simplest thing. There's the shell + make + m4 + autoconf's libraries (workarounds for non-portable shell utilities) + automake's libraries. It's a huge pain, because there's not a single place to look things up. You have to find the appropriate shell utility...then check in the autoconf/automake documentation to see if there are portability wrappers. scons is simpler; you can find most stuff with just their manpage, and possibly Python's documentation when you need to do actual programming in the build system. One (very simple) syntax. Two sources of API documentation.

      The autoconf/automake system is nice in that it makes no assumptions about the user's system beyond make - everything is just generated to plain shell scripts and makefiles. But it's such a pain for the developer that it's not worth it. SCons assumes the user has a working Python installation, which makes everything more pleasant. Python provides the same level of functionality with the same interface on every platform.

    3. Re:SCons is much better than GNU Make by eraserewind · · Score: 2, Interesting
      Most makefile setups are recursive. They should not be (see Recursive Make Considered Harmful for a good explanation). It's a pain to make non-recursive make, because the tools aren't set up for it. SCons is.
      This is my biggest (but not only) problem with GNU Make. It does not have support for the theoretically correct way to use it. It's not like the paper is new, or even that the maintainer disagrees with it's findings, but there is no movement at all to add support for the features that would make it easy to use. GNU Make is a project that is totally defined by it's implementation. There is no concept of a "correct" way is should be behaving. There is only the way it behaves.
    4. Re:SCons is much better than GNU Make by Anonymous Coward · · Score: 0

      Well, GNU Make is open-source, so you can extend it, if you want to. But as pointed out, it's a really broken system from how software was made in days of, by today's standard, hardware limited to the point of being unusable.

      I'm planning to investigate SCons one of these days, to see what the deal is about, if it lives up to the hype. But, to the observer, it seems to have two shortcomings compared to make:
      1. Full-blown Python scripts is an overkill for simple projects. I mean, you can learn the generic Makefile language within five minutes, but learning a new language (especially one as unusual as Python) would take a bit longer (but at least it's easier than Perl).
      2. It seems to be more about hacking around the problems of C/C++ (not completely unlike lint and ccache) than building a better general-purpose build tool, which hence makes it less useful for other languages.

    5. Re:SCons is much better than GNU Make by eraserewind · · Score: 3, Insightful
      Well, GNU Make is open-source, so you can extend it, if you want to.
      Well, you can extend it, but you more than likely won't be able to get your patch into the main version. The project is extremely conservative about any new functionality. If it's not in GNU make, you can't use your new functionality if you hope to distribute to more than yourself.

      It's almost the same as the situation with internet explorer. Until the program with the most market share improves, everyone who cares about working with the marjority of people is stuck with a deficient system.
    6. Re:SCons is much better than GNU Make by Anonymous Coward · · Score: 0

      I agree about Scons being better than make.

      The only thing preventing me from using Scons is boost-build v2 (milestone 10). A better, easier and more powerful make replacement than Scons.

      Boost-build v2 uses bjam for the underlying language but they are working on adding support for Python scripting.

      After trying Scons, I said, "this is so much better than make!"

      After trying boost-build v2, I said, "holy shit this is even better than Scons!"

      You can't go wrong with either choice. Both are much better than make and I highly recommend them.

    7. Re:SCons is much better than GNU Make by Anonymous Coward · · Score: 0

      I totally agree. I've used SCons for a cross-platform project where we were building for Windows, Mac, Linux and FreeBSD. One SConstruct file to rule them all. :)

      It was very easy to get working, but once it was stable, I admittedly did write quite a bit of custom Python code to make it do exactly what we wanted. We had some platform-specific tweaks in a few places (since we were using Microsoft Visual C++ 2003 .NET on Windows, GCC elsewhere), but it's fairly easy to do this.

      The one criticism that I have is that you have to jump through some hoops to manually build the {input source, output object} list if you don't want to use the SCons recommended directory layout (SConstruct files in every source directory, rather than one big SConstruct file at the root) - but that may just be me being a control freak or not quite understanding what I was asking Scons to do..

    8. Re:SCons is much better than GNU Make by Ed+Avis · · Score: 1

      I'm interested to know what you mean. Why does GNU make not have support for the right way to use it (namely, a single Makefile with correct depedencies)? Which features would you like the maintainers to add?

      --
      -- Ed Avis ed@membled.com
    9. Re:SCons is much better than GNU Make by Anonymous Coward · · Score: 0
      1. Full-blown Python scripts is an overkill for simple projects. I mean, you can learn the generic Makefile language within five minutes, but learning a new language (especially one as unusual as Python) would take a bit longer (but at least it's easier than Perl).

      You don't need to use the full programming language to make a simple SConstruct. The SCons API and documentation are set up to be friendly to people who do not know Python.

      2. It seems to be more about hacking around the problems of C/C++ (not completely unlike lint and ccache) than building a better general-purpose build tool, which hence makes it less useful for other languages.

      I'm not sure what you mean there. SCons is definitely a general-purpose build tool. It has built-in support for those languages, but also for many others. And you can build other things easily. (Even without Python knowledge. See the generic Command builder.)

  5. Build tools by SunFan · · Score: 2, Insightful


    One thing I find quite remarkable is that in a couple of decades, make is still the only mainstream multi-language multi-platform build tool. The alternatives are either not widely used or are language-specific like Ant. With so many people not liking make, it's suprising an alternative tool hasn't really caught on.

    --
    -- Microsoft is the most expensive commodity operating system and office suite vendor in the marketplace.
    1. Re:Build tools by Anonymous Coward · · Score: 0

      language-specific like Ant

      What's language specific about ant? You can make any extension to compile / deploy / package any code you want to do. Make should be thrown on the rubbish heap and replaced with ant. Must easier to make a build.xml file and maintain it.

    2. Re:Build tools by SunFan · · Score: 1

      What's language specific about ant?

      It requires J2SE, and it's files are XML. This only makes sense if you are already programming in Java. If a project has no needed dependency on Java, then why force it?

      --
      -- Microsoft is the most expensive commodity operating system and office suite vendor in the marketplace.
  6. GNU Make vs Solaris/Sys V make by ratsg · · Score: 1

    GNU Make is certainly a wonderful tool, and as a Solaris admin of 10 years, it is typically a tool that I install on systems where I need to compile/install open source software but, I have compiled quite a bit of of open source software using

    /usr/ccs/bin/make

    Some applications have problems with the make that ships with Solaris, but most work fine. My statement is is reference to the opening paragraph that states:

    "Everyone who has ever compiled a piece of Open Source software has used GNU's make."

  7. Very few original broken tools replaced by Ars-Fartsica · · Score: 1
    sh (uselss for scripting, completely broken at anything but trivial command line work) and make (terrible, painful and not very featureful) are probably top of the list for me...I'm amazed these were not replaced long ago. The entire autotools chain should be discarded.

    Would love to see a distro that replaces sh with python/perl...replaced make with scons etc.,....at least this distro could claim a differentiating point beyond the installer. I would use it.

    1. Re:Very few original broken tools replaced by Anonymous Coward · · Score: 0
      Would love to see a distro that replaces sh with python/perl...replaced make with scons etc.,....at least this distro could claim a differentiating point beyond the installer. I would use it.
      And how would you compile thousands of packages that today require make and/or sh (anything requiring autoconf)? Changing your distro is one thing, but convince thousands of people while you're at it, and say good-bye to compiling the Linux kernel, etc., for instance.
    2. Re:Very few original broken tools replaced by eraserewind · · Score: 1
      And how would you compile thousands of packages that today require make and/or sh (anything requiring autoconf)?
      I agree with Mister AC here. Make has huge market share currently and anything that aims to usurp it will ultimately need to support the Makefile syntax, and a way to convert between the two. Otherwise intertia wins the day again. That's not to say that you can't make a niche for yourself with a totally new tool, but you just won't convert everyone else to it.
    3. Re:Very few original broken tools replaced by macshit · · Score: 2, Insightful

      I think the main problem is that many people still want their projects to be pretty portable, at least among unix variants. Once you start requiring people to have "super-duper-make" (requiring pyperl version X.Y.Z) installed, they often just give up on your software rather than deal with the hassle of installing whatever your favorite tool is.

      Traditional make is a lowest-common-denominator, however much it sucks. GNU make is slowly becoming widespread enough that it may be a viable alternative.

      The autotools are widespread for a similar reason, in that they try very hard to produce portable configure scripts.

      [I don't have much experience with autoconf alternatives, but those I've seen -- e.g. imake and Larry Wall's Configure stuff -- suck even more than autoconf. It's sort of like democracy: the worst system there is, except for all the others :-]

      --
      We live, as we dream -- alone....
    4. Re:Very few original broken tools replaced by Ars-Fartsica · · Score: 1
      Oh I agree it would be difficult to create such a distro...you would basically have to have a method for yanking the existing toolchain out and in its place putting in the equivalent modern tool syntax.

      As for sh this could be done...in fact many distros are using Pythin in many places where sh was once used.

      In the end it will take one hugely succesful and unavoidable project to push alternatives. For example, if the kernel itself pushed an alernative to make, this alternative would almost certainly spread to other projects.

      In any case its likely a non-issue as long term more people are using prepackaged software, and also using code that is outside the gnu tools realm, like Python and mono code.

  8. Everyone? by Brandybuck · · Score: 2, Insightful

    Everyone who has ever compiled a piece of Open Source software has used GNU's make.

    It is quite possible to compile a piece of Open Source Software without GNU make. It's not easy, to be sure, since there are so many projects out there that require GNU make (automake doesn't help matters much), but it is possible. There is BSD make, Solaris make, Microsoft's strange nmake, and several others. GNU is but one of many, and it's not even the only free make.

    The problem is that the make standard is so tepid that to get a decent make you need to extend it. So what we end up in reality is a lack of a make standard. I can write a complete C program that will compile with any standard C compiler. I can write a powerful bourne shell script that runs on any Bourne compliant shell. But to write a Makefile that will work under all makes is quite difficult.

    --
    Don't blame me, I didn't vote for either of them!
  9. Newer and better alternatives to make by Anonymous Coward · · Score: 5, Interesting
    There are newer and much better alternatives to make.


    1. boost-build v2 is the absolute BEST if you want to build C/C++ projects with multiple compilers & versions & targets--or even on simple projects that require a one-liner to feed into boost-build2 (normally taking 5-20 lines in GNU make).


    Upside for boost-build2? Wraps compiler/linker flags in a generic language for many compilers and versions(gcc, msvc, bcc, etc). Also very easy for simple projects but truly shines on huge and complex projects. Jamfiles can inherit properties , requirements, targets, etc. from parent directories. Very very cool.


    Downside to boost-build? Documentation truly sucks compared to other tools. Docs getting better but new users should prepare to unexpectedly find features they could've used to avoid hours of effort.


    Boost-build v2 uses bjam but there seems to be a plan to add support for Python.


    2. Scons is the next best thing to boost-build v2. The underlying language is Python but you don't have to be a Python expert to use it. And the documentation is much better than boost-build v2. However, it takes many more lines to get things done than boost-build v2 (which isn't all that bad considering boost-build v2 can do things in only 5 lines to replace a 40-line gnu makefile).


    3. rake is a make-alternative written in Ruby. For all you recent Ruby converts, be sure to check it out. I love Ruby but I gotta admit, I don't see anything out there being better than boost-build v2 today.


    GNU make served us well but it is time to move on to better choices that make us more productive. Just like cvs having served us well but svn and others being a better choice today.

    1. Re:Newer and better alternatives to make by VZ · · Score: 3, Informative

      All these tools (and also cmake and A-A-P mentioned in the other reply) are great for developers but not ideal for the end user who almost surely doesn't have them installed on his machine.

      If you want to use something as easy on developer but which would still require no additional on the users machine, have a look at bakefile. This is a very useful tool, especially for open source programs where users often have or are asked to rebuild the program from source and installing additional tools is just an extra hurdle for them.

  10. A-A-P by Noksagt · · Score: 2, Informative

    A-A-P, led by Bram Moolenaar (of vim fame) looks promising too.

  11. There are ways to create portability support by Ars-Fartsica · · Score: 1
    You could either use the stick or the carrot...the "carrot" approach would be to get a hugely pivotal piece of code pushing alternatives. As I said in another post, if the kernerl was to use an alternative to make, this alternative would almost certainly see widespread adoption elsewhere.

    The "stick" approach would be to mandate inclusion in the LSB so that all distros carry the tool. The BSDs and other OSs would certainly follow suit.

    I agree its a difficult legacy problem to solve but it is not intractable.

  12. tmk by systems · · Score: 1

    And there is also tmk at http://tmk.sourceforge.net/

  13. Ripping DVDs by jbltgz · · Score: 1

    GNU Make is so cool I use it to rip DVDs!@#$%^&

    1. Re:Ripping DVDs by mvdw · · Score: 2, Informative
      I agree with the fact it's cool - I generate static internal web pages using make + sed + cat on cygwin. I'd hate to regenerate all the pages just because one page changed...

      The pages are essentially built up from a header file, a sidebar file, a text file (the only bit that changes regularly), and a footer file. The make file scans the directory for all the *.src files, then generates a page for each *.src file, copies it into the right folder as index.html, making the folder if it's not there already. Easy way to generate consistent pages, although it would've been even better if I could have got css to work properly...

  14. Everyone? by Just+Some+Guy · · Score: 1
    --
    Dewey, what part of this looks like authorities should be involved?
  15. Pet Peeve: Make abusers by milgr · · Score: 1

    One of my peeves is people who use make, but don't make use of its basic functionality.

    One place I worked, they used a program that was similar (and equivalent) to make. But, rebuilding after one source file changed would recompile all 100 source files. Other components coded the build files correctly.

    Another place I have worked uses gnu make. They rebuild a minimum of one file from each source directory during every build. This causes quite a bit of visual cruft - and makes it more likely that warning messages won't be noticed.

    make is just not that difficult to use correctly. When makefiles work nicely, the development process runs smoother.

    --
    Where law ends, tyranny begins -- William Pitt
  16. Disappointing portability discussion by Anonymous Coward · · Score: 0

    Since it had been so long since the second edition came out, I was really looking forward to this updating. Although in general it's a fine effort with a lot of good information, I have to say I was extremely disappointed with the chapter discussing portability. In this day and age of multiple platforms and cross-compilation, the author basically just punts and says, "It's really hard, use Cygwin."

    That advice might work for software you're developing yourself, but it's of no use if you're trying to ship software that others must build. You can't assume that everyone has Cygwin installed on their systems, and you can't force them to do so without losing a lot of potential users/customers.

    It's a real shame, because the world could use a good discussion of how to write really portable Makefiles, and who knows how long it'll be until there's a fourth edition?