Slashdot Mirror


Pragmatic Project Automation

twelve71 (Alan Francis) writes "Apologies in advance for overuse of the word 'pragmatic,' but Dave Thomas and Andy Hunt together form a company called The Pragmatic Programmers, and published a book I'm sure many of you have read, titled The Pragmatic Programmer: from Journeyman to Master. The Pragmatic Programmer (or 'PragProg' as it is usually referred to) is a wonderful grab bag of 'good old common sense,' but its main strength (covering a very broad range of subjects) means that the authors have left a few holes around some important details. To plug some of these holes, and provide a good grounding for those just starting out, they have recently published 'The Pragmatic Starter Kit' - a set of books covering in detail some of the basics mentioned in PragProg." Read on for Alan's review of the latest book in the kit. Pragmatic Project Automation author Mike Clark pages http://www.pragmaticprogrammer.com/bookshelf/index.html publisher Pragmatic Bookshelf rating 9 reviewer Alan Francis ISBN 0974514039 summary Learn to use common, freely available tools to automate build, test, and release procedures.

The first 2 (or possibly 3) books are Pragmatic Version Control with CVS and Pragmatic Unit Testing (which is available in Java and .NET flavours).

Pragmatic Project Automation is the latest book in the series and, interestingly, this book wasn't actually written by either Dave Thomas or Andy Hunt, but by Mike Clark (contributor to the 'Bitter EJB' book, editor of the JUnit FAQ, and responsible for the JUnitPerf and JDepend tools). Mike does a great job of ensuring this book fits in with the overall style of the other books in the series.

Up front, in case you're a "cut to the chase kind of guy," this book (and the others in the series) are must-haves and as a consultant with ThoughtWorks I'll have a set ready to distribute whenever I start working with a new client team.

Content The book helps the reader build a Java project slowly over the chapters, starting with a manual build-and-deploy process and automating a new aspect of it, chapter by chapter, until by the end of the book the software compiles, archives, deploys and configures itself multiple times a day. In addition, the build tools notify you of success or failure in a variety of interesting ways including email, SMS messages and different colored lava lamps.

The first chapter provides a good introduction to the different types of automation available to projects. It also introduces the acronym 'CRISP' to help the reader remember the desirable characteristics of an ideal build process: Complete, Repeatable, Informative, Scheduled and Portable.

Chapter 2 gets you Repeatable by using ANT to bring together all the various steps you currently perform on your project into a single, one-click build. Chapter 3 works to turn the one-click build into a no-click build using tools as simple as cron as well as more complicated tools like the ANT scheduling tool CruiseControl. By the end of these two chapters your software can be compiling and testing itself automatically each time changes are checked into the version control system. But this is only the beginning.

Chapters 4 and 5 address the "Complete" and "Portable" portions of the CRISP model discussing how to include packaging, release management and deployment into your scheduled build. The last chapter addresses "Informative." How to monitor the build for success or failure, how to notify members of the project team using email, SMS, RSS or even the red and green Lava Lamps I mentioned above.

Summary This book, and the others in the series, provide a much needed set of manuals for getting a good set of basic practices up and running at the start of a project. Unlike the Unit Testing book, there's not a lot of programming in this one, but it's a worthwhile read for any programmer, regardless of experience level.

Many people are becoming interested in eXtreme Programming and Agile methods for software development. These books help to support some of the key ideas of those methods - extensive unit testing and continuous integration.

The main flaw in the book doesn't affect its usefulness, only its readability. Of all the files used in the sample project, the only one covered in any detail is the build file. The source and manifest files that we're writing the ANT file to build are never discussed or described - we are left to guess at what the sample project might contain (unless we download the code from the website). While this didn't change what I was learning (how to manipulate the project files with ANT), I like to understand all the details and this omission did occasionally leave me a little irritated.

You can purchase Pragmatic Project Automation from bn.com. Slashdot welcomes readers' book reviews. To see your own review here, carefully read the book review guidelines, then visit the submission page. If you are interested in the Pragmatic Programmers, see also this interview linked earlier from Slashdot.

69 comments

  1. What if your an idealist?? by CrashPanic · · Score: 1, Funny

    What if your an idealist??

    --
    "There's no set architecture in Linux. All roads lead to madness" -Microsoft
    1. Re:What if your an idealist?? by kfg · · Score: 5, Funny

      What if your an idealist??

      You write emacs.

      KFG

    2. Re:What if your an idealist?? by TigerNut · · Score: 3, Insightful

      Not really an offtopic question (at least as far as it relates to general pragmatism vs. an idealistic "every bit must be polished to a mirror-like hue" development approach). However... all software exists to "do stuff". If it's not there in time, then it's just wasted disk space and it inconveniences the hell out of the other folks that were depending on that software to do their thing. Therefore, on-time delivery is not always just a nice-to-have, and being pragmatic about architecture and feature sets can produce better code in the end.

      --

      Less is more.

    3. Re:What if your an idealist?? by TomorrowPlusX · · Score: 4, Funny

      Well then, you'd use an an apostrophe, wouldn't you?

      --

      lorem ipsum, dolor sit amet
    4. Re:What if your an idealist?? by ExMember · · Score: 1

      What if your an idealist??

      You write emacs.

      or worse, HURD.

    5. Re:What if your an idealist?? by Anonymous Coward · · Score: 0

      What if your an idealist??

      "you're".

  2. Seen outside space ship: by Anonymous Coward · · Score: 2, Funny

    Squeegee Robot with sign saying: "Will Automate For Food"

  3. Props to Mike for JDepend... by tcopeland · · Score: 4, Interesting

    ...I've integrated it into our hourly build; it's handy info.

    1. Re:Props to Mike for JDepend... by s88 · · Score: 2, Interesting

      Yea, we use it on our continuous build dashboard to test for package cycles everytime a comit is issued. Here's an example.

  4. The pragmatic bookshelf by Colonel+Panic · · Score: 4, Informative

    The Pragmatic Programmers seem to be doing a great job with their new publishing efforts. They're able to get new titles out much more quickly than some of the more established publishers by following their own advice about automation and applying it to publishing. I recall reading on one of their blogs that they were able to go from idea to actual book in under six months.

    I can't wait for their upcoming Ruby book.

    1. Re:The pragmatic bookshelf by bay43270 · · Score: 1

      I assume you're aware that this will be a second edition. The first was good, but I could do without the printed reference material. I think if they cut that out they could bring it up to the standards of their other books fairly easily.

    2. Re:The pragmatic bookshelf by Mandomania · · Score: 2, Informative

      For those that don't already know, the first edition of this book is available free of charge.

      It's a great book and I can't wait for the second edition.

      P.S. Send an email to mando.escamilla@gmail.com if you want some gmail invites.

      --
      Mando

  5. nice book by Anonymous Coward · · Score: 5, Interesting

    This is a nice little book. Don't forget the web site either.

    I'm not a Java programmer (mostly Ruby and Perl) but I found a lot of stuff in this book inspiring. After reading this book I got bitten by the automation bug and did stuff like this:

    * gave my big deployed apps RSS feed logging for errors (i.e., now I can track recent warnings and errors in apps deployed across the country, just in NetNewsWire) .. why didn't I think of this before??

    * wrote a ruby script to automatically run unit tests whenever the files change.. based on a script on the web site. this is really cool! You edit your source file, save it, and pause to glance at the unit test window. again, why on earth didn't I think of this before!

    * wrote an automated test framework using WWW::Mechanize to log into web apps and check for errors or anything else and send out emails

    * use a nightly cron to check latest code out of CVS, run all tests, and run web tests..

    Anyway "project automation" is like unit testing .. once you get a taste you start doing it all the time and your productivity goes up yet another notch.

    So even if you don't do Java you can find a lot of ideas in this book (like the lava lamps showing build status!)

    1. Re:nice book by Anonymous Coward · · Score: 1, Insightful

      What other wheels have you re-invented?
      Simple tools for nearly everything you mentioned already exist.

    2. Re:nice book by Anonymous Coward · · Score: 0

      It might be helpful if you named them.

  6. Best companion book to Pragmatic Programmer... by SlashdotTroll · · Score: 1, Troll

    I found that Donald Knuth's "Art of Computer Programming" was the most visionary book in applying code to real-world situations. Although this slashdot story is a status of the first anouncment on the Pragmatic Programmer being composed a while ago, I'm thankful its status is made known. I bought it from Amazon.com and am only 33% into the book. It is verry enlightening, equaly so to "Art of Computer Programming." I think it doesn't bring enough introductory information to warrant purchase by a Novice. To prepare for this book, I found the best recommendation from my shelf to be "Computer Hardware Theory" (1972 University of Illinois) and "Fundamentals of Signal Theory" (1960). Being a programmer isn't supposed to be all-software; you need to know the basics of the hardware so you can prepare your code to be cross-platform.

    --

    I am the nightmare of nightmares.

    1. Re:Best companion book to Pragmatic Programmer... by johnnyb · · Score: 0, Offtopic

      My Book is focused on kind of the same area, but focuses on the interface between the application and the operating system and hardware. While Knuth essentially assumes the complete lack of an operating system, my book teaches how it interacts with your program through system calls, linking, memory management, etc. It talks about calling conventions, addressing modes, and all of the other gritty details that programmers need to know that usually just comes in bits and pieces from other sources.

    2. Re:Best companion book to Pragmatic Programmer... by GoofyBoy · · Score: 2, Insightful

      >I found the best recommendation from my shelf to be "Computer Hardware Theory" (1972 University of Illinois) and "Fundamentals of Signal Theory" (1960).

      Two books from 30 years ago on computer hardware and one is on Signal Theory?

      Unless I'm doing assembly, could you explain why these books would be good for me, a programmer?

      --
      The surprise isn't how often we make bad choices; the surprise is how seldom they defeat us.
    3. Re:Best companion book to Pragmatic Programmer... by bay43270 · · Score: 2, Insightful

      I thought he was joking. Maybe he was and the moderators didn't get it. No way is "Art of Computer Programming" a book about "applying code to real-world situations". Interesting academic information, yes, but not exactly mainstream stuff.

    4. Re:Best companion book to Pragmatic Programmer... by uncadonna · · Score: 2, Insightful
      I thought he was idly pretending to have a clue.

      The fact that it got modded up is alarming.

      --
      mt
    5. Re:Best companion book to Pragmatic Programmer... by twiddlingbits · · Score: 2

      Hmm..well Hardware still consists of the same things it always has, Logic, ALU, Memory, Cache, etc. It just works faster. Things like how to understand accuracy & precision from the number representation in the Hardware haven't changed. Signal Theory..very hairy Math... Mathematical Algorithms have been known for high/low/band/gap and other types of filters for umpteen years. All the theories underpinning the mathematics hasn't changed in some cases 100+ years. I don't recall the basic theories of Calculus, Algebra, Statistics and Linear Algebra changing. I can recall picking up a math book from the 1950's at a book sale and seeing a lot of things I recall from my college math in the 1980s. The language has changed a bit, some terminolgy is different but the equations are the same.

    6. Re:Best companion book to Pragmatic Programmer... by SlashdotTroll · · Score: 1

      quoth { "Unless I'm doing assembly, could you explain why these books would be good for me, a programmer?" } quoth Application and System layer programming was once united under one title "Programmer", but it is now separated. The books I suggested only contribute to the over-all understanding to establish pre-ponderance in your source-code that it may be of good portage and maintenance should another be granted its access. Today, Application Programmers are separated unto further hostile classification based upon hardware architecture and software platforms. If you want your application to behave nearly identical on vague hardware as a Agenda VR3 PDA, Ipaq, Win32 X86, and Linux/Sparc, then the more you know to develop your own syntax within your C is the better; to build your own Abstraction Layer. Can't you imagine that Hardware and Software were once one and the same, but now that are splintered off. Dennis Ritchie was quoted as saying somthing similar at the last convention he attended; OSCON, Iirc. Verry intelligent man, but even he admitted that Plan9 is re-inventing what other *NIX inventors already solved and lost; port Unix to other platforms and thus losing compatibility. Many people believed Dennis Ritchie to suggest that Unix was not supposted to be an Operating System but an actual behavioral API for how code may interact with foreign code upon differing architectures. Sun datalinks and LUMP nodes don't actualy have hardware with the same signal specification even though they both follow the same RFC. Just my point of view.

      --

      I am the nightmare of nightmares.

    7. Re:Best companion book to Pragmatic Programmer... by SlowMovingTarget · · Score: 3, Insightful

      Comparing Pragmatic Programmer to The Art of Computer Programming is like saying that raisins are just like prunes because they both make you go to the bathroom if you eat enough of them.

      Don't get me wrong, I absolutely love Pragmatic Programmer and would consider it a classic, but TAOCP clearly has more, ummm, fiber.

    8. Re:Best companion book to Pragmatic Programmer... by Anonymous Coward · · Score: 1, Insightful

      Do you really think that your book is "focused on kind of the same area" as _The Art of Computer Programming_? I have read both books, and really don't understand how you could consider this to be the case. I do consider your book to be one of the two best in the area it covers though, and reccomend it highly.

    9. Re:Best companion book to Pragmatic Programmer... by johnnyb · · Score: 1

      No, actually, as the introduction says, it's more geared as a "prequel" to TAOCP.

    10. Re:Best companion book to Pragmatic Programmer... by aled · · Score: 2, Interesting

      the more you know to develop your own syntax within your C is the better; to build your own Abstraction Layer. ...Unix was not supposted to be an Operating System but an actual behavioral API for how code may interact with foreign code upon differing architectures.

      Use Java Language (cleaner syntax than C), with the Java Virtual Machine (abstraction layer) and the Java libraries (API that works on different architectures) and problem solved.

      --

      "I think this line is mostly filler"
  7. The lava lamp automation software... by tcopeland · · Score: 4, Informative

    ...is right here.

    It uses Java to talk to the serial port... seems like a good application for ruby-serial, too...

  8. Wrong Link by Anonymous Coward · · Score: 0
    1. Re:Wrong Link by darth_MALL · · Score: 0, Offtopic

      Actually it's Here.

    2. Re:Wrong Link by Anonymous Coward · · Score: 0

      Take off, hoser.

  9. Programming Ruby by haskellcompiler · · Score: 0

    Programming Ruby - The Pragmatic Programmers' Guide is an excellent guide to learn Ruby, full of examples and great explanations.

  10. Oh, *automation*. by autophile · · Score: 2, Funny
    Oh, I thought it said "Pragmatic Project Automaton", which would be real useful.

    --Rob

    --
    Towards the Singularity.
  11. Helpful links by Anonymous Coward · · Score: 1, Informative

    Probably most of us already know what these are, but for those who don't:

    Extreme Programming.

    Agile software development.

  12. Buy direct from the Pragmatic Bookshelf by AndyHunt · · Score: 5, Informative
    You can also buy the PDF and paperback direct from the publisher (and we really appreciate that) at www.pragmaticprogrammer.com/sk/auto


    That's also the home page for the book with downloaded code samples and so on.


    thanks!

    /\ndy

    1. Re:Buy direct from the Pragmatic Bookshelf by Gailin · · Score: 1

      Just curious.

      Why are the PDF version and the dead tree version the same price (excluding $2.82 for shipping)?

      I expected there to be a cost savings for buying the electronic version.

      No matter really though, I'm going to buy it just the same.

      Gailin

      --
      I wish there was a fscking blue pill
    2. Re:Buy direct from the Pragmatic Bookshelf by AndyHunt · · Score: 3, Informative
      We put the paper tree version and the combo pack (paper and PDF) on sale; the PDF by itself is still at the regular price.

      thanks for your support!

      /\ndy

  13. Good Start by Derkec · · Score: 2, Insightful

    This book seems like a good one for all developers to read. I'm honestly only about half way though it, but I've skimmed the whole thing.

    The big thing that jumps out at me is that he promotes how cool CruiseControl is for automating builds, but uses cron for his release builds. Generally, I would want to use the same tool to create my continuous integration, nightly and production release builds. I was under the impression that there were ways to make that happen in CruiseControl.

    disclaimer: I help write a product that does do this, so maybe I'm just projecting biases.

    The section on diagnostics also looks quite interesting, but I'm not that far yet.

    1. Re:Good Start by cubicleman · · Score: 3, Informative

      On my current project, we've used Cruise Control for continuous integration and to run our suite of Junit tests (2500 tests and counting now) for the last 20 months or so..on a largish J2EE app (800+ ejbs, 30 developers, 3 distinct client builds), it makes sense and helps keep in check the stability of the app..

    2. Re:Good Start by Derkec · · Score: 1

      Cool, do you have CruiseControl manage the release builds as well or something else?

    3. Re:Good Start by cubicleman · · Score: 1

      No..that's still a human process. A release manager decides which builds are released to QA for testing on a schedule (and then branches those releases that QA passes off into builds to be deployed to the clients for their testing).

  14. In other news... by Discotechnica · · Score: 1, Funny

    Following his success in writing a book on Project Automation Dave Thomas decided he'd share his knowledge of Pragmatic Patty Production with the world.

    In this tell all exposee of beefly manufacturing processes, he teaches the enthralled reader how to use common, freely available tools to automate build, test, and release highly beeftacular burgers at a fraction of the current cost!

    Way to go, Dave Thomas!

    1. Re:In other news... by Anonymous Coward · · Score: 0

      That Dave Thomas (the found of Wendy's) is dead and has been for a couple of years. So unless he ghost wrote it for someone else...

  15. What if I program in C++ ? by Catamaran · · Score: 2, Interesting
    I am very interested in the ideas presented here (extreme, agile, automated), but my experience is that - even with CppUnit and C++ support for ANT - the fit is not very good. Most of the test and build automation that one hears about is targeted toward Java.

    On the other hand, there is Test Environment Toolkit that noone seems to use. And STAF which requires a huge investment of time just to comprehend.

    So, question: what tools do people find useful for build/test automation with C++ ?

    --
    Test 1 2 3 4
    1. Re:What if I program in C++ ? by Derkec · · Score: 1

      Most of the setups I've seen are running CppUnit or similar with Make for builds and tests. Another option is OpenMake

      Making that happen repeatedly on a controled server is the domain of Anthill, CruiseControl and a handful of for money tools.

    2. Re:What if I program in C++ ? by stew1 · · Score: 5, Informative

      I did XP in extreme C++ for about a year (by extreme C++, I mean boostified, Alexandrescu'd C++). We used CppUnit for our test framework.

      I'm not especially satisfied with the currently available C++ unit testing frameworks. CppUnit and Boost's both have trade-offs. I suspect they'll both get better, though.

      The Ant-Contrib project's cc task works pretty well, in my limited experience. I was playing around with it just this weekend. I've yet to set up a tinderbox build process, but I don't see why this wouldn't be easy with either Cruise Control or good ol' cron.

      In my mind, the two biggest hurdles with doing XP in C++ are build speed and developer prejudice. You can tackle build speed with a combination of ccache, distcc, good programming principles, and cash. Tackling developer prejudice is harder. A lot of C++ programmers like to write low-level, unsafe, old school C++ code. Modern C++ mostly lets you discard unsafe coding practices without sacrificing efficiency. Whether you can convince an old C++ programmer of this is another matter; it depends on the person. I've had decent success taking Java programmers and teaching them modern C++ via pair programming.

      Using Boost helps, indirectly. You write safer code, which gives you a faster development cycle. All in all, I think the basic tools are there. It might be a little harder to get fancy lava lamp integration going with C++, but there's no reason why you can't have a good build process. It's just that a lot of C++ projects haven't evolved (I think this is due, in part, to the fact that the C++ community is late to the internet; a lot of C++ programmers just don't know what's, out there.

      cheers,

      Jon

    3. Re:What if I program in C++ ? by MimsyBoro · · Score: 1

      You should try cxxtest.
      It's a great unit testing framework that's VERY flexible.

      --
      God made the natural numbers; all else is the work of man - Kronecker
    4. Re:What if I program in C++ ? by Tirsen · · Score: 2, Interesting

      Use DamageControl. It is a rewrite of CruiseControl in Ruby and is completely build tool agnostic. It is also written by guys from ThoughtWorks that got fed up with CruiseControl requiring Ant. (Full disclosure: I'm one of them.)
      It's not 1.0 yet but it's been building most of the projects at Codehaus for half a year now.

      http://damagecontrol.codehaus.org

      Another tip: Use something like Ruby/Perl/Python for your automation needs. Not only are they platform-agnostic, they are also waaay more powerful than Ant. (And great fun to use!)

  16. Grab Bag by MikeMacK · · Score: 1
    The Pragmatic Programmer (or 'PragProg' as it is usually referred to) is a wonderful grab bag of 'good old common sense'

    So, the PragProg is a grab bag. Who knew.

  17. Hooray for PragProg! by mopslik · · Score: 2, Funny

    (or 'PragProg' as it is usually referred to)

    I'm a huge fan of PragProg. Oh Yes, we Rushed though development of our last application. Only once did we lose Focus, but you Can easily avoid these issues. Since its Genesis, we've got nearly a Marillion hits a day. I can't wait Tull I pick up this book.

    1. Re:Hooray for PragProg! by Limburgher · · Score: 1
      Lunch. . .coming. . .up. . .(gwaaaaf)

      Sorry. Good joke, just. . .a bit rich. :)

      --

      You are not the customer.

    2. Re:Hooray for PragProg! by chochos · · Score: 1

      Banter, burble, bicker, bicker bicker... man, all this elephant talk and you forgot to include King Crimson!

  18. Cross platform, ant independant dashboard by s88 · · Score: 1

    If you are looking for a more generic build/test runner dashboard than cruise control, take a look at DART. Its free, open source, and makes very few restrictions on those projects which wish to use it.

    Here are some example dartboards.

    Here is my dart board for the Mobius Project I work on.

  19. I'm not sure what 'Fark' is, but what about ... by cubicleman · · Score: 1

    'Fahrt Nacht Bumplitz' ? I saw that headline whilst reading a German newspaper whilst riding an Italian train a few months ago..

  20. Thankyou! by Anonymous Coward · · Score: 0

    johnnyb,

    Thankyou for uncovering your book's title to me. I'ld login as "SlashdotTroll" and post, but a number of users have prejudiced my name and reduced my karma from Positive to Bad. (let it be known, a good deed on slashdot will never go unpunished) Thanks for that link and I hope to order it soon. Do you sell direct, through an eBay or Yahoo Auctions store, or is it on Amazon or my local book market? Thanks again,

    -SlashdotTroll

  21. Interesting by Anonymous Coward · · Score: 2, Informative

    I like this book and the review is good, but take the review with a grain of salt. The reviewer is a Thought Works consultant. The book spends some solid time on CruiseControl which is a Thought Works donation to the open source community. That he likes it isn't too much of a surprise - particularly since the book, in fact, is quite good.

    1. Re:Interesting by twelve71 · · Score: 1

      This is a fascinating comment.

      It starts by suggesting the good review is supect because I have a vested interest in CruiseControl, and then goes on to say the review is good because the book is good.

      For the record, yes I work for TW. I don't however have a vested interest in Cruise. I joined TW well after Cruise became an open source project soI don't view it any differently than ANT - it's an OSS tool I use, rather than a TW tool.

      Again, for the record, I'd never really set a cruise intance up before and so found the book genuinely helpful.

      --
      -- Twelve|71
  22. I hate to fart in church by Anonymous Coward · · Score: 0

    But am I the only one who thought "The Pragmatic Programmer" wasn't worth the paper it was printed on. While I've enjoyed other classics* and found them stimulating or invaluable, I found that book insufferable.

    *some theoryish: SICP, PAIP, and compiling with continuations; some whimsical--Programming Pearls and Gabriel's memoir-ish musings about patterns; some journeymen--O'Reilly's lex'n'yacc and sed'n'awk books, Kernighan's Practice of Programming, and even Stroustrup's definitive tome.

  23. Re:WINDOWS XP SP2 COMPATIBLE KEYGEN 4 IN 1 by Anonymous Coward · · Score: 0

    Worst. Troll. Ever.