Slashdot Mirror


Pragmatic Project Automation

twelve71 (Alan Francis) writes "Apologies in advance for overuse of the word 'pragmatic,' but Dave Thomas and Andy Hunt together form a company called The Pragmatic Programmers, and published a book I'm sure many of you have read, titled The Pragmatic Programmer: from Journeyman to Master. The Pragmatic Programmer (or 'PragProg' as it is usually referred to) is a wonderful grab bag of 'good old common sense,' but its main strength (covering a very broad range of subjects) means that the authors have left a few holes around some important details. To plug some of these holes, and provide a good grounding for those just starting out, they have recently published 'The Pragmatic Starter Kit' - a set of books covering in detail some of the basics mentioned in PragProg." Read on for Alan's review of the latest book in the kit. Pragmatic Project Automation author Mike Clark pages http://www.pragmaticprogrammer.com/bookshelf/index.html publisher Pragmatic Bookshelf rating 9 reviewer Alan Francis ISBN 0974514039 summary Learn to use common, freely available tools to automate build, test, and release procedures.

The first 2 (or possibly 3) books are Pragmatic Version Control with CVS and Pragmatic Unit Testing (which is available in Java and .NET flavours).

Pragmatic Project Automation is the latest book in the series and, interestingly, this book wasn't actually written by either Dave Thomas or Andy Hunt, but by Mike Clark (contributor to the 'Bitter EJB' book, editor of the JUnit FAQ, and responsible for the JUnitPerf and JDepend tools). Mike does a great job of ensuring this book fits in with the overall style of the other books in the series.

Up front, in case you're a "cut to the chase kind of guy," this book (and the others in the series) are must-haves and as a consultant with ThoughtWorks I'll have a set ready to distribute whenever I start working with a new client team.

Content The book helps the reader build a Java project slowly over the chapters, starting with a manual build-and-deploy process and automating a new aspect of it, chapter by chapter, until by the end of the book the software compiles, archives, deploys and configures itself multiple times a day. In addition, the build tools notify you of success or failure in a variety of interesting ways including email, SMS messages and different colored lava lamps.

The first chapter provides a good introduction to the different types of automation available to projects. It also introduces the acronym 'CRISP' to help the reader remember the desirable characteristics of an ideal build process: Complete, Repeatable, Informative, Scheduled and Portable.

Chapter 2 gets you Repeatable by using ANT to bring together all the various steps you currently perform on your project into a single, one-click build. Chapter 3 works to turn the one-click build into a no-click build using tools as simple as cron as well as more complicated tools like the ANT scheduling tool CruiseControl. By the end of these two chapters your software can be compiling and testing itself automatically each time changes are checked into the version control system. But this is only the beginning.

Chapters 4 and 5 address the "Complete" and "Portable" portions of the CRISP model discussing how to include packaging, release management and deployment into your scheduled build. The last chapter addresses "Informative." How to monitor the build for success or failure, how to notify members of the project team using email, SMS, RSS or even the red and green Lava Lamps I mentioned above.

Summary This book, and the others in the series, provide a much needed set of manuals for getting a good set of basic practices up and running at the start of a project. Unlike the Unit Testing book, there's not a lot of programming in this one, but it's a worthwhile read for any programmer, regardless of experience level.

Many people are becoming interested in eXtreme Programming and Agile methods for software development. These books help to support some of the key ideas of those methods - extensive unit testing and continuous integration.

The main flaw in the book doesn't affect its usefulness, only its readability. Of all the files used in the sample project, the only one covered in any detail is the build file. The source and manifest files that we're writing the ANT file to build are never discussed or described - we are left to guess at what the sample project might contain (unless we download the code from the website). While this didn't change what I was learning (how to manipulate the project files with ANT), I like to understand all the details and this omission did occasionally leave me a little irritated.

You can purchase Pragmatic Project Automation from bn.com. Slashdot welcomes readers' book reviews. To see your own review here, carefully read the book review guidelines, then visit the submission page. If you are interested in the Pragmatic Programmers, see also this interview linked earlier from Slashdot.

26 of 69 comments (clear)

  1. Seen outside space ship: by Anonymous Coward · · Score: 2, Funny

    Squeegee Robot with sign saying: "Will Automate For Food"

  2. Re:What if your an idealist?? by kfg · · Score: 5, Funny

    What if your an idealist??

    You write emacs.

    KFG

  3. Props to Mike for JDepend... by tcopeland · · Score: 4, Interesting

    ...I've integrated it into our hourly build; it's handy info.

    1. Re:Props to Mike for JDepend... by s88 · · Score: 2, Interesting

      Yea, we use it on our continuous build dashboard to test for package cycles everytime a comit is issued. Here's an example.

  4. The pragmatic bookshelf by Colonel+Panic · · Score: 4, Informative

    The Pragmatic Programmers seem to be doing a great job with their new publishing efforts. They're able to get new titles out much more quickly than some of the more established publishers by following their own advice about automation and applying it to publishing. I recall reading on one of their blogs that they were able to go from idea to actual book in under six months.

    I can't wait for their upcoming Ruby book.

    1. Re:The pragmatic bookshelf by Mandomania · · Score: 2, Informative

      For those that don't already know, the first edition of this book is available free of charge.

      It's a great book and I can't wait for the second edition.

      P.S. Send an email to mando.escamilla@gmail.com if you want some gmail invites.

      --
      Mando

  5. nice book by Anonymous Coward · · Score: 5, Interesting

    This is a nice little book. Don't forget the web site either.

    I'm not a Java programmer (mostly Ruby and Perl) but I found a lot of stuff in this book inspiring. After reading this book I got bitten by the automation bug and did stuff like this:

    * gave my big deployed apps RSS feed logging for errors (i.e., now I can track recent warnings and errors in apps deployed across the country, just in NetNewsWire) .. why didn't I think of this before??

    * wrote a ruby script to automatically run unit tests whenever the files change.. based on a script on the web site. this is really cool! You edit your source file, save it, and pause to glance at the unit test window. again, why on earth didn't I think of this before!

    * wrote an automated test framework using WWW::Mechanize to log into web apps and check for errors or anything else and send out emails

    * use a nightly cron to check latest code out of CVS, run all tests, and run web tests..

    Anyway "project automation" is like unit testing .. once you get a taste you start doing it all the time and your productivity goes up yet another notch.

    So even if you don't do Java you can find a lot of ideas in this book (like the lava lamps showing build status!)

  6. The lava lamp automation software... by tcopeland · · Score: 4, Informative

    ...is right here.

    It uses Java to talk to the serial port... seems like a good application for ruby-serial, too...

  7. Re:What if your an idealist?? by TigerNut · · Score: 3, Insightful

    Not really an offtopic question (at least as far as it relates to general pragmatism vs. an idealistic "every bit must be polished to a mirror-like hue" development approach). However... all software exists to "do stuff". If it's not there in time, then it's just wasted disk space and it inconveniences the hell out of the other folks that were depending on that software to do their thing. Therefore, on-time delivery is not always just a nice-to-have, and being pragmatic about architecture and feature sets can produce better code in the end.

    --

    Less is more.

  8. Re:What if your an idealist?? by TomorrowPlusX · · Score: 4, Funny

    Well then, you'd use an an apostrophe, wouldn't you?

    --

    lorem ipsum, dolor sit amet
  9. Oh, *automation*. by autophile · · Score: 2, Funny
    Oh, I thought it said "Pragmatic Project Automaton", which would be real useful.

    --Rob

    --
    Towards the Singularity.
  10. Buy direct from the Pragmatic Bookshelf by AndyHunt · · Score: 5, Informative
    You can also buy the PDF and paperback direct from the publisher (and we really appreciate that) at www.pragmaticprogrammer.com/sk/auto


    That's also the home page for the book with downloaded code samples and so on.


    thanks!

    /\ndy

    1. Re:Buy direct from the Pragmatic Bookshelf by AndyHunt · · Score: 3, Informative
      We put the paper tree version and the combo pack (paper and PDF) on sale; the PDF by itself is still at the regular price.

      thanks for your support!

      /\ndy

  11. Good Start by Derkec · · Score: 2, Insightful

    This book seems like a good one for all developers to read. I'm honestly only about half way though it, but I've skimmed the whole thing.

    The big thing that jumps out at me is that he promotes how cool CruiseControl is for automating builds, but uses cron for his release builds. Generally, I would want to use the same tool to create my continuous integration, nightly and production release builds. I was under the impression that there were ways to make that happen in CruiseControl.

    disclaimer: I help write a product that does do this, so maybe I'm just projecting biases.

    The section on diagnostics also looks quite interesting, but I'm not that far yet.

    1. Re:Good Start by cubicleman · · Score: 3, Informative

      On my current project, we've used Cruise Control for continuous integration and to run our suite of Junit tests (2500 tests and counting now) for the last 20 months or so..on a largish J2EE app (800+ ejbs, 30 developers, 3 distinct client builds), it makes sense and helps keep in check the stability of the app..

  12. Re:Best companion book to Pragmatic Programmer... by GoofyBoy · · Score: 2, Insightful

    >I found the best recommendation from my shelf to be "Computer Hardware Theory" (1972 University of Illinois) and "Fundamentals of Signal Theory" (1960).

    Two books from 30 years ago on computer hardware and one is on Signal Theory?

    Unless I'm doing assembly, could you explain why these books would be good for me, a programmer?

    --
    The surprise isn't how often we make bad choices; the surprise is how seldom they defeat us.
  13. What if I program in C++ ? by Catamaran · · Score: 2, Interesting
    I am very interested in the ideas presented here (extreme, agile, automated), but my experience is that - even with CppUnit and C++ support for ANT - the fit is not very good. Most of the test and build automation that one hears about is targeted toward Java.

    On the other hand, there is Test Environment Toolkit that noone seems to use. And STAF which requires a huge investment of time just to comprehend.

    So, question: what tools do people find useful for build/test automation with C++ ?

    --
    Test 1 2 3 4
    1. Re:What if I program in C++ ? by stew1 · · Score: 5, Informative

      I did XP in extreme C++ for about a year (by extreme C++, I mean boostified, Alexandrescu'd C++). We used CppUnit for our test framework.

      I'm not especially satisfied with the currently available C++ unit testing frameworks. CppUnit and Boost's both have trade-offs. I suspect they'll both get better, though.

      The Ant-Contrib project's cc task works pretty well, in my limited experience. I was playing around with it just this weekend. I've yet to set up a tinderbox build process, but I don't see why this wouldn't be easy with either Cruise Control or good ol' cron.

      In my mind, the two biggest hurdles with doing XP in C++ are build speed and developer prejudice. You can tackle build speed with a combination of ccache, distcc, good programming principles, and cash. Tackling developer prejudice is harder. A lot of C++ programmers like to write low-level, unsafe, old school C++ code. Modern C++ mostly lets you discard unsafe coding practices without sacrificing efficiency. Whether you can convince an old C++ programmer of this is another matter; it depends on the person. I've had decent success taking Java programmers and teaching them modern C++ via pair programming.

      Using Boost helps, indirectly. You write safer code, which gives you a faster development cycle. All in all, I think the basic tools are there. It might be a little harder to get fancy lava lamp integration going with C++, but there's no reason why you can't have a good build process. It's just that a lot of C++ projects haven't evolved (I think this is due, in part, to the fact that the C++ community is late to the internet; a lot of C++ programmers just don't know what's, out there.

      cheers,

      Jon

    2. Re:What if I program in C++ ? by Tirsen · · Score: 2, Interesting

      Use DamageControl. It is a rewrite of CruiseControl in Ruby and is completely build tool agnostic. It is also written by guys from ThoughtWorks that got fed up with CruiseControl requiring Ant. (Full disclosure: I'm one of them.)
      It's not 1.0 yet but it's been building most of the projects at Codehaus for half a year now.

      http://damagecontrol.codehaus.org

      Another tip: Use something like Ruby/Perl/Python for your automation needs. Not only are they platform-agnostic, they are also waaay more powerful than Ant. (And great fun to use!)

  14. Re:Best companion book to Pragmatic Programmer... by bay43270 · · Score: 2, Insightful

    I thought he was joking. Maybe he was and the moderators didn't get it. No way is "Art of Computer Programming" a book about "applying code to real-world situations". Interesting academic information, yes, but not exactly mainstream stuff.

  15. Re:Best companion book to Pragmatic Programmer... by uncadonna · · Score: 2, Insightful
    I thought he was idly pretending to have a clue.

    The fact that it got modded up is alarming.

    --
    mt
  16. Re:Best companion book to Pragmatic Programmer... by twiddlingbits · · Score: 2

    Hmm..well Hardware still consists of the same things it always has, Logic, ALU, Memory, Cache, etc. It just works faster. Things like how to understand accuracy & precision from the number representation in the Hardware haven't changed. Signal Theory..very hairy Math... Mathematical Algorithms have been known for high/low/band/gap and other types of filters for umpteen years. All the theories underpinning the mathematics hasn't changed in some cases 100+ years. I don't recall the basic theories of Calculus, Algebra, Statistics and Linear Algebra changing. I can recall picking up a math book from the 1950's at a book sale and seeing a lot of things I recall from my college math in the 1980s. The language has changed a bit, some terminolgy is different but the equations are the same.

  17. Hooray for PragProg! by mopslik · · Score: 2, Funny

    (or 'PragProg' as it is usually referred to)

    I'm a huge fan of PragProg. Oh Yes, we Rushed though development of our last application. Only once did we lose Focus, but you Can easily avoid these issues. Since its Genesis, we've got nearly a Marillion hits a day. I can't wait Tull I pick up this book.

  18. Re:Best companion book to Pragmatic Programmer... by SlowMovingTarget · · Score: 3, Insightful

    Comparing Pragmatic Programmer to The Art of Computer Programming is like saying that raisins are just like prunes because they both make you go to the bathroom if you eat enough of them.

    Don't get me wrong, I absolutely love Pragmatic Programmer and would consider it a classic, but TAOCP clearly has more, ummm, fiber.

  19. Interesting by Anonymous Coward · · Score: 2, Informative

    I like this book and the review is good, but take the review with a grain of salt. The reviewer is a Thought Works consultant. The book spends some solid time on CruiseControl which is a Thought Works donation to the open source community. That he likes it isn't too much of a surprise - particularly since the book, in fact, is quite good.

  20. Re:Best companion book to Pragmatic Programmer... by aled · · Score: 2, Interesting

    the more you know to develop your own syntax within your C is the better; to build your own Abstraction Layer. ...Unix was not supposted to be an Operating System but an actual behavioral API for how code may interact with foreign code upon differing architectures.

    Use Java Language (cleaner syntax than C), with the Java Virtual Machine (abstraction layer) and the Java libraries (API that works on different architectures) and problem solved.

    --

    "I think this line is mostly filler"