Ant - The Definitive Guide

pankaj_kumar writes "Apache Ant, the Java replacement for make , belongs to the rare breed of category killer software for automating Java software development tasks. It is an Apache open source project, has won numerous awards, boasts comprehensive online documentation and is used by most Java developers. So, what could a book say that is already not available online?" Read on for Kumar's review of Ant: The Definitive Guide. Ant The Definitive Guide author Steve Holzner pages 316 publisher O'Reilly rating 8 reviewer Pankaj Kumar ISBN 0596006098 summary Complete Build Management for Java with Ant

As a long time Ant user, I have written many Ant build scripts, automating my builds and speeding up the overall development cycle, mostly relying on its excellent online documentation. As a Java developer, I have admired its simple and intuitive interface and the modular design. So on getting Ant: The Definitive Guide in my hands I wasn't expecting a whole lot new to learn, and thought of using it only as a reference book.

After having the book on my desk for more than a month, though, and occasionally flipping through its pages whenever I would otherwise have consulted the online documentation, I must say that I had been missing out on some very important things: tasks like ftp and war deployment that I was simply not aware of and had never felt the need to look up, but could very well use. The other interesting thing I noticed was that my build scripts became smaller, more modular and easier to read.

Like most books in the The Definitive Guide series, Ant The Definitive Guide assumes a certain level of familiarity with underlying technologies such as Java and XML and focuses on providing complete, reference like details of Ant features and tasks. These description are generously supplemented with examples and code fragments.

But so is the the online documentation for Ant! Will someone gain additional insight in using Ant, or be able to work faster, or make better use of Ant capabilities, by consulting this book, instead of the online documentation for a particular Ant task? To find the answer, I randomly picked two topics -- filesets, an important and oft-used Ant datatype, and javac, a core Ant task -- and compared their online description with the one in the book. Here is what I found.

Besides the datatype definition, explanation of various attributes, sub-elements, and the examples, the book also covers how to specify conditional inclusion or exclusion of certain filename patterns when a property is set (or unset). Though this can be inferred from online documentation by a determined user, this particular use is far from obvious. The coverage in the book also talks about the relationship of the fileset datatype with the javac task, pointing out that the fileset attribute dir is equivalent of javac attribute srcdir, as attribute dir will be confusing in javac: is it referring to source directory or destination directory. This is the kind of insight that really helps a user.

The treatment of the javac task in the book is not much different from the one in the online documentation. Both have almost the same material, though the information in the book is better organized for new users. On the other hand, I found the online documentation to be more complete, especially with respect to the compiler specific options and behavior idiosyncrasies.

Here is a rundown on what the book covers: Chapter 1, Getting Started is a quick primer on Ant, with sufficient details for a new user to start using Ant for very simple build tasks. Chapter 2, Using Properties and Types introduces the building block tasks and datatypes, such as property, condition, fileset, path like structures, selectors and so on, used in other Ant tasks. Chapter 3, Building Java Code covers the tasks and activities around compiling Java source files (ie; javac), organizing the build steps in various targets within a single build scripts and/or across multiple scripts, generating documentation using javadoc and creating distribution jars and zip files. The rest of the chapters are devoted to tasks for specific purposes, such as launching external programs (Chapter 7, Executing External Programs), copying files and manipulating directories either on the same machine or over the network (Chapter 4, Deploying Builds), running JUnit tests (Chapter 5, Testing Builds with JUnit) and so on. There are also separate chapters covering interaction of Ant with XML and XDoclet (Chapter 9, XML and XDoclet) and with Eclipse (Chapter 11, Integrating Ant with Eclipse). The last chapter, Chapter 12, Extending Ant, talks about extending Ant by doing things like adding your own tasks, creating custom filters, writing your own build listeners and loggers etc. This chapter also has a small section on how to embed a script written in one of the supported scripting languages within an Ant script.

As you can see from this outline, the book covers almost everything that is to know about Ant and other related software.

So, what is not so good about this book? Well, I didn't find anything wrong with the topics which are actually covered by the book. Of course, there are additional things that I would have liked to see in the book: (a) A good sample Ant script which could be used as the starting point for most small to medium-sized projects; (b) A more thorough explanation of how dependencies among targets determine the execution sequence and how this fits in with explicit invocation of targets; and (c) pictures to illustrate some of the concepts such as life cycle of an Ant task, selection of files in a fileset and the dependency tree of targets.

Overall, I found the book to be comprehensive, well organized, easy to read and good value for money.

You can purchase Ant: The Definitive Guide from bn.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.

Comments

Make file

[alex323]

I like how the make file are XML based. With other makefiles, you get stuck if your tab has a space in it :/ Good review though!

Re:Make file

[Cryptnotic]

I hated that about it. You can't make a file using just a text editor unless you want to break your fingers on the shift, < and > keys.

Re:Make file

[truckaxle]

Whoever decided to use a tab as a delimiter should not be allowed to die a natural death. This "feature" has cause countless hours of loss productivity. At one time the make processor gave some obscure warning on the missing tab and left you scratching your head for hours.

Re:Make file

[jarich]

I've fuond enough samples in the bundled set of Ant docs that I only have to customize Ant scripts, not create them.

Re:Make file

[arcanumas]

Still, i think it would be fair to say that if your tab does not produce a .. tab, then it is not a... tab. It's something else. :)
Writing Makefiles in editors where the tab is a tab should produce no problems.

Re:Make file

[dglo]

I like how the make file are XML based. With other makefiles, you get stuck if your tab has a space in it

<sarcasm>
Cuz Ant is so forgiving if you leave out a '<' or '>' character...
</sarcasm>

Re:Make file

[Baki]

people should use a decent editor such as emacs. its make mode highlights the tabs so you can readily see what's up.

Re:Make file

[jarich]

Yeah, but I can ~see~ the missing angle brackets! And with proper indentation, matching in XML elements is trivial...

Of course, most people I know who hate XML write their XML on a VERY long lines, with lots of elements jammed in together, and then they complain that it's hard to read.

Re:Make file

[quasi_steller]

Yeah, since xml is so great and we are not creative enough to develop a better format than xml or makefiles.

Re:Make file

[Anonymous Coward]

Yeah but any worthwhile xml editor will show you exactly what's missing.

Re:Make file

[codegen]

Just like JCL(IBM's job control lanaguage). Read the New Hackers Dictionary description of JCL. An excerpt:

"Most programmers confronted with JCL simply copy a working file (or card deck), changing the file names. Someone who actually understands and generates unique JCL is regarded with the mixed respect one gives to someone who memorizes the phone book"

About sums up my opinion of ANT.

Re:Make file

[jarich]

Then I respectfully submit that you have very little experience JCL or Ant (or both).

Re:Make file

[jarich]

Perhaps that was a bit hasty...

Let me instead ask you... do you work with Java code? If so, how do you build your projects?

Do you let your IDE do all the work? Or does your company IT department handle all compiles? Or...?

Re:Make file

[Anonymous Coward]

XML is what makes Apache Tomcat, along with all the other Apache-related Java stuff, CRAPPY. Look at a typical config file for Tomcat. Is that intuitive?

I think the java technology is cool, but who in their right mind would even dream of such a hiddeous thing as config files in XML with tags like <Listener className="org.apache.catalina.mbeans.GlobalResour cesLifecycleListener" />? If Debian didn't provide semi-sensible defaults, I'd have no idea how to set up the damn thing.

XML is a BAD IDEA for config files. In fact, it's a BAD IDEA for most types of information, as you can store things MUCH more efficiently and more readably other ways.

Re:Make file

[Anonymous Coward]

Any worthwhile text editor will show you the tabs when opening a make file.

Not that this excuses the tab-issue in make files, but it's not a problem unless you're using Notepad or something silly like that.

Re:Make file

[caseydk]

Yep and just about every XML editor/parser out there - including Ant itself - will scream if it's not a well-formed document.

mwuhahahaha

[sargosis]

So, what could a book say that is already not available online?"

Nothing, you have to read it.


Sorry, i just had to...

Re:mwuhahahaha

[BlogPope]

So, what could a book say that is already not available online?"

Nothing, you have to read it.

You can read the book on the "pot" a lot easier than the online version (though I'm sure some Slashdotters have a net erminal there too)

Re:mwuhahahaha

[Anonymous Coward]

freakin a

it was a joke you lame arse nuck fut

Ant life cycle...

[op12]

So, what is not so good about this book?...Of course, there are additional things that I would have liked to see in the book: (c) pictures to illustrate some of the concepts such as life cycle of an Ant

Here ya go:
Life Cycle

GUI

[hattan]

Is there a graphical application to help generate ant build scripts? Maybe it will help those starting out in Java get familiar with ANT. A lot of new programmers I talk to are afraid to use ANT because it looks complicated.

Re:GUI

[malraid]

ANT is a tool for *programmers*. A GUI ?? I don't think that's a good idea. It's just like VB monkeys making GUIs that can't do shit, because they haven't written a single line of code !!! Compared to writing any half decent java program, I find ANT to be easy and simple to make simple tasks. The documentation has LOTS of examples. Am I being elitist? Maybe...I don't like people who think they can program with only a mouse. Sorry, that's my opinion.

Re:GUI

[madth3]

A graphical aproach I've seen to handle ant scripts is done by NetBeans. Starting with version 4 all the project management is ant-based and you can add targets and parameters from the IDE.
Last time I used it the paths were added as absolute paths so I continued editing my scripts by hand, though.

Re:GUI

[Reverend528]

Well, contrary to popular belief, XML wasn't really meant to be directly edited by humans. Large nested structures become exceptionally difficult to read and there's not really a clear way to break them down into smaller abstract parts. Not to mention, it's a rather verbose language (compared to something like lisp which has a nearly identical structure with half as much syntax).

If there isn't a GUI for editing ant files, there really should be.

Re:GUI

[jarich]

Look at docs that come with Ant. I think it's the docs/manual/index.html

There are examples of how to use every task. It's really pretty easy once you get started.

Re:GUI

[it_flix]

Doesnt eclipse use Ant for its builds?
Just figure out how they generate their ant scripts...

Re:GUI

[Anonymous Coward]

FinalBuilder uses XML for the build scripts but gives you a nice drag&drop UI, so you never have to write another tag again.

http://www.finalbuilder.com/index.html

Re:GUI

[glesga_kiss]

Is there a graphical application to help generate ant build scripts?

Not quite a help wizard, but Eclipse is really good with ant files, especially the most recent version. Will auto-complete tags and variable names, plus some other useful tools. I can't imagine doing them in a text editor!

Re:GUI

[owlstead]

Eclipse, one of the more heavily used Java IDE's, does have a "sort of" Ant editor. It's still plain text, but with completion support and error checking etc to make it more pallable. And it does provide a GUI for the launch configurations. Other IDE's do also have Ant support, although I am not familliar with the extend of support for most IDE's.

Ant files are pretty easy to read, and they are normally not computer generated. I've never had much problems reading them, and they are infinitely more easy to read than most make files.

Re:GUI

[Anonymous Coward]

Why the parent is flamebait is beyond me. Java "programmers" are becoming the bottom of the food chain in my opinion due to the fact that java syntax is so easy to learn that most anyone with minimal skills can get something working, consider themselves experts after a few classes, and soon thereafter J2EE experts. I've been a software engineer for several years, with primarly java development recently...the only good thing is that I'm glad that my competition in the job market is frighteningly bad. mod me flamebait too if you want...

Re:GUI

[zarr]

Doesnt eclipse use Ant for its builds?

No, it doesn't (Thank God!). You're thinking of netbeans.

Re:GUI

[Decaff]

[Doesnt eclipse use Ant for its builds? ]

No, it doesn't (Thank God!). You're thinking of netbeans.

Why 'thank God'? Why should internal use of Ant be a problem? On the contrary, it has major advantages: It means you can build the project outside of the IDE, and it means that the IDE can make use of just about any Java tool, as almost all of them provide functionality as Ant tasks. Compare this to eclipse, where you need to install plug-ins.

Re:GUI

[gedhrel]

... or just use Eclipse's embedded Ant.

We use ant build scripts alongside all our (eclipse-hosted) projects. We make sure the ant script is compatible with eclipse's notion of dynamic compilation, so that the two will work seamlessly together. However, the big advantage (as you point out) is that this means we don't need to run eclipse in headless mode to build and deploy a project: just use ant.

Price-wares.

[Anonymous Coward]

"You can purchase Ant: The Definitive Guide from bn.com."

Or you can find a copy at your local used book store for $2.00

Only two words necessary

[greg_barton]

Use Maven. :)

Re:Only two words necessary

[puppetman]

I have a web service application that resides as a seperate module in CVS. I needed the ability to build a JAR, and move it either to local qa application servers, or a remote server.

I thought I'd try Maven. After spending 4 hours looking for and reading what passes for documentation on the Apache site, I gave up, and wrote a batch script to do it in 15 minutes.

The Apache docs suck for many of their projects (Tomcat, Maven, Axis). Do the developers do it on purpose so that they can make money writing books explaining the tools? I have no idea.

To be fair, some Apache documentation is good (in the commons projects, like CLI and HttpClient).

Re:Only two words necessary

[Anonymous Coward]

Maven has serious issues as eloquently explained by Hani. Be sure to check out his newest article on mergere, the company that attempts to support maven.

Re:Only two words necessary

[jbrownc1]

I wish someone would write a book about Maven that was as helpful as this Ant book sounds like it might be. I am also put off by the not-quite-helpful documentation on the Maven site.

Re:Only two words necessary

[MemoryDragon]

Lots of vital information for various apache projects can be found on the wikis. It all usually comes down to finding people who write the docs. But I must admit, the current doc development cycle of using forrest and sending in patches is not very beginners friendly, while writing docs is the best thing for getting started for beginners to get into a project. The apache projects are in a chicken egg situation, documentationwise, I think many of them should point out their respective WIKIs more clearly and frequently ask users in the mailing list to add vital info which they post their to their wiki.

Re:similarly

[jen0r]

haha, agreed!

Basically

[Safety Cap]

Make is for kiddiz,

Ant is for Toddlers,

Maven is for Adults

Re:Basically

[Karma Farmer]

Make is for C programmers,

Ant is for Java programmers,

Maven is for idiots.

Save Money

[Anonymous Coward]

Save more than FOUR BUCKS by buying the book here: Ant - The Definitive Guide

Bookpool has it cheaper than bn.com

[COBOL/MVS]

This book is cheaper at bookpool.com ($22.50) than at bn.com ($27.96 bn price/$25.16 member price). Get it there and save a few $$.

Re:Bookpool has it cheaper than bn.com

[poopdeville]

Or wait for the torrent and get it free.

Kumar?

[Anonymous Coward]

What kind of name is Kumar? What is that like 5 o's or 2 u's?

Re:Kumar?

[Anonymous Coward]

Harold. .

Re:Kumar?

[frodo from middle ea]

Well in Hindi , or any Indian language, it means, "the young one", which can mean either young, or more exotically a virgin (male).

Some how I think most /. would better associate with the later meaning of the word.

Necissity if you use Ant

[Anonymous Coward]

I have worked on a couple products which have used ant as the build system. These were released a few years ago and I think the version was ant 1.3. The online doc at the time ~2002 was mediocre. I think it has gotten better now, but the book still covers all the basic features you will ever need and decreased our old make full build time from 30-40 minutes to about 10 with ant on the same hardware, as well as making it much easier to make changes and add new steps. When making minor changes ant will recompile and determine dependencies in just a few seconds where our make system would still take 5-10 minutes just for incremental change. Granted our make system could have been optimized, ant was a good excuse to start over. I have a paper copy, but do also use safari to search through the text. This gives you the best of being online and having the book.

Ant IS complicated

[mekkab]

Inheriting fully-formed Ant scripts for a big build and then having to add in your new units is not only daunting, but confusing as well.

But its just like makefiles... stop being scared, figure it out, and then welcome to the world of easy-builds.

(or you could just add your object files to the jars by hand...)

Not as good as MakeXS ...

[Anonymous Coward]

Seriously though, Ant's biggest problem is Java. GNU Make is so lightweight.

Re:Not as good as MakeXS ...

[ebuck]

Contrary to popular belief, GNU make is no lightweight. Sure, the make executable is lighter than and + a JVM, but the first thing the make executable calls will be a shell, and that will be calling a program, and you'll use many many programs to do anything that isn't trivial.

Worse yet, make runs each task in a new shell, so you'll have many many shells being created and destroyed. If your project grows to any considerable extent, you'll soon see ant's solution of one shell (the JVM) as a lightweight solution, even if the shell itself is bigger.

Frankly, I don't care about building Java.

[david.given]

I want to build generic programs. Java, C, Pascal, Occam, COBOL, shell scripts. This is because the applications I need to build tend to be a mixture of various different paradigms: little tools that are built to produce data massagers that convert offline data into compilable data; metacompilers like yacc and bison; XML preprocessors like xslt; traditional C or C++; documentation builders like Javadoc or WEB... I can't find anything that will handle all this cleanly.

Is there anything out there that is (a) easily deployable (nothing turns off a prospective user than being told to download and install a complicated build system that depends on $LANGUAGE_OF_THE_MOMENT), (b) suitably flexible that I can customise it to work with all my little build tools, and (c) sufficiently elegant that I won't want to vomit looking at my build scripts six months later?

So far, the only thing I've found that works at all is traditional make. Which, I'm afraid, sucks. Makefiles scale very badly (recursive make. Eeeaah), don't handle disparate rulesets well (I want to build these C files with this rule and these with this other rule... oh, wait, I can't), and the dependency handling is practically nonexistent (you can fake to a certain extent with .d files, but that all falls apart as soon as you need to depend on dynamically made files).

A case in point: I maintain the ACK, a portable compiler toolchain that's about 20 years old. The build system is an intricate network is shell scripts and recursive makefiles. It works, but it's largely incomprehensible, very slow, doesn't handle incremental rebuilds, and is going to be a maintenance nightmare should we ever need to do any major revamps. I'd love to replace it; I've gone out actively looking for something better --- and I've failed.

Any suggestions?

This has been a public service rant by a stressed build technician.

Re:Frankly, I don't care about building Java.

[Anonymous Coward]

Yeah, neither does this guy:

http://www.jroller.com/page/fate/

Re:Frankly, I don't care about building Java.

[Anonymous Coward]

have a look at maven or scons. maven supports a number of languages. you can write plugins to support more. if you can, spend a bit of time with it. most people who hate it expect it to offer instant gratification. as with most ultimately good things, that's never the case.

Re:Frankly, I don't care about building Java.

[iabervon]

Actually, make scales pretty well if you have a single make process read a bunch of files with "include", possibly recursively. It handles having different rulesets if you use static pattern rules. The only problem I haven't been able to solve effectively in my use of make is getting it to recheck dependancies after recomputing them (and I haven't figured out how to pull together the linking requirements of libraries and get each one listed once in the necessary order).

The main thing is that you really want to have a Makefile with all your rules, and have a file of variable definitions for each directory, so that the build system rules don't have to be duplicated everywhere.

Re:Frankly, I don't care about building Java.

[Abcd1234]

"I want to build generic programs. Java, C, Pascal, Occam, COBOL, shell scripts."

Well, nice if you to post your complaints in a forum about Ant and Java, then.

Re:Frankly, I don't care about building Java.

[Anonymous Coward]

I use FinalBuilder and never looked back.
I does everything you want and more :) :

http://www.finalbuilder.com/index.html

Supported compilers (you can also add your own easily) :

Visual Basic Project
VC6 Project
C++ Builder Project
Chrome
Delphi Win32
Borland Resource Script
Delphi .Net Project (D8 and D2005)
Java Compiler
MadExcept
VS.Net Solution
AssemblyInfo Updater
MS C# Compiler
MS C# Project Compiler
MS VB.Net Compiler
MS VB.Net Project Compiler
MS J# Compiler
MS J# Project Compiler
Borland C# Compiler

Re:Frankly, I don't care about building Java.

[argent]

Any suggestions?

Prolog.

No, I'm not kidding. Solving complex dependency trees is what Prolog's *for*.

Re:Frankly, I don't care about building Java.

[jag164]

Oh you sick sick twisted individual.... You do get a gold star though. Kudos.

Re:Frankly, I don't care about building Java.

[sinserve]

win32 fag

Re:Frankly, I don't care about building Java.

[Anonymous Coward]

You can build ant tasks to do anything. An Ant build file simply calls these ant tasks. So you could have a task that calls your C++ compiler, fiddles with your XSLT, etc...

Re:Frankly, I don't care about building Java.

[gargleblast]

The build system is an intricate network is shell scripts and recursive makefiles. ... I'd love to replace it; I've gone out actively looking for something better --- and I've failed. Any suggestions?

1. Keep using make
2. Use included makefiles instead of recursive makes. See Recursive Make Considered Harmful
3. You can of course specify the build commands individually for each target. However you can also make per-target modifications to implicit rules using e.g. '+=' syntax (you are using GNU make, aren't you?)
4. You'll need to run the .d files through some heavy scripting, but you can get them to behave in the presence of generated files

Re:Frankly, I don't care about building Java.

[Anonymous Coward]

You can use Scons. A python based build system, has excellent dependency handling, flexible, has many built in rules for existing tools, handles the disparate rulesets elegantly, and solves a whole lot of problems.

I like it.

Re:Frankly, I don't care about building Java.

[kaffiene]

Ant supports other languages - c / C++ is pretty standard. Any new language can be handled via plugins

Ant does the job...

[Anonymous Coward]

But I prefer to use make and a Makefile to compile my Java stuff because make is available on every Unix by default.

Windows? There I compile my java stuff too using make in cygwin.

Make is available on more platforms than Java or Ant together.

The best thing about using make and a Makefile instead of Ant is that I don't need to read books like this.

Oh... And I can use make and a Makefile to compile C code and the Java code of the same project.

Re:Ant does the job...

[Decaff]

Oh... And I can use make and a Makefile to compile C code and the Java code of the same project.

If you understood Ant, you would know that it can do this as well.

Re:Ant does the job...

[Lothsahn]

The best thing about using make and a Makefile instead of Ant is that I don't need to read books like this.

You don't need to learn make from books because you already know it, and if you knew ant, you wouldn't need to read books like this either.
Even if you didn't know ant, the online documentation is sufficient. That is how I learned ant.

Re:Ant does the job...

[p2sam]

You can. But you should not. You're falling into the Turing Tar Pit. The C and C++ building facilities are too immature for now. And the Ant developers certainly are not putting priority in making it work well with C and C++ building anyhow.

FYI: I use Ant to build/test/deploy my Java code.

Re:Ant does the job...

[publius_ovidius]

make is awful. Different operating systems have different command line syntax, different path separators, limits to command length, etc. And which make are you using? nmake, dmake, bsd make, etc? There are tons of different make programs, many of which are subtly incompatible with one another and sometimes incompatible with different versions of themselves.

So if you can tightly control the operating systems you are distributing to and tightly control the versions of make you are using, make is a fine solution. For those of us who distribute open-source code, make is an absolute nightmare. How many of us want to repond to someone claiming they can't install our software or VMS? Admittedly, some don't care, but I don't want a reputation as someone who distributes non-portable software. (Note, this isn't intended as a flame. Your needs are likely completely different from my own and make may suit them perfectly.)

Re:Ant does the job...

[Anonymous Coward]

make is awful.

That is girltalk.

Re:Ant does the job...

[HuguesT]

There is only one true make, and that is GNU make. Works everywhere, including on VMS.

Your complaint belongs in the late 80s or so.

Re:Ant does the job...

[Shayde]

But I prefer to use make and a Makefile to compile my Java stuff because make is available on every Unix by default.

This is such a ridiculous assertion. Lets take this apart for a moment.

First of all, Makei s not available on every Unix by default. You have to install it from whatever distribution methodolog you happen to use.

Once it's installed, you STILL have to install the JDK for ANY application you're building, no matter what. You're a java developer, ergo, you need the JDK. With me so far?

If you have the JDK installed, you have a Java VM. Ant uses the Java VM (it's distributed in binary form as a .JAR file. You have to have Java to run it.

So, by your own definition here, every platform you'd be developing or running apps on will run Ant. So much for that excuse.

Moving along. Installation of Ant is TRIVIAL. Unpack the distribution into /usr/ant (or wherever). Do 'export ANT_HOME=/usr/ant; export PATH=$PATH:$ANT_HOME/bin' - or make symlinks as appropriate, and voila, it's installed.

Remember, Ant will run on every system that Java runs on, which is every platform YOU'D be working on.

If you want to use Make because you're used to it, more power to ya, but don't use false statements to justify your own preferences.

Re:Ant does the job...

[Decaff]

You can. But you should not. You're falling into the Turing Tar Pit. The C and C++ building facilities are too immature for now. And the Ant developers certainly are not putting priority in making it work well with C and C++ building anyhow.

This is a reasonable point. However, the original post mentioned C or C++ as part of a general Java project, so any support should be adequate. For better or worse, ant is Java-focused, so C and C++ building is not going to be a priority.

Re:Ant does the job...

[publius_ovidius]

Regardless of whether or not that is true (I wouldn't swear by it), the point is not whether or not everyone can use the particular version of make I want them to use. It's whether or not the version of make they actually do use is compatible with my Makefile. That's why Java has Ant. That's why Ruby has Rake. That's why Perl has Module::Build. That's why more and more open-source software projects are realizing that make doesn't cut it. Just because you think that GNU make is superior to Solaris make (which behaves quite differently in many cases) doesn't mean that that a Solaris shop will easily convince their admin to install your version of make just because you have a personal preference.

You should read up about the limitations of make and compatability issues of different versions.

Your complaint belongs in the late 80s or so.

Your defense belongs in the late 90s or so :)

Re:Ant does the job...

[ebuck]

Ant-contrib offers a C / C++ build task, and I've used it with a lot of success for building the .so libraries that my JNI calls will use.

The documentation isn't great, actually (when I used it) it sucked, or more to the point, I couldn't find it. Finally I found an open source project that used the task successfully, and used thier build.xml file as a template for the compiling I had to do.

Mabye I'm being too harsh with respect to the documentation, since a quick look over at http://ant-contrib.sourceforge.net/cc.html seems much more comprehensible than I remember it a few years back.

Re:Ant does the job...

[MemoryDragon]

Make yes, but make is nothing more than an extended batch processor with a lousy syntax, make relies heavily on third party tools, which you might not get at various platforms make itself runs on. One of the big advantages of newer build systems like ant is, that you do not need those third party programs anymore.

Re:Ant does the job...

[p2sam]

Let's put it this way. I would have no problems telling my boss that we should use ant to build Java projects. But I would not risk it if we were doing C/C++ projects.

Ant and C/C++ is not ready for professional use. That's why the facilities are still in the "contrib" tree, and not the offical one.

Re:fp

[Anonymous Coward]

STFU you idiot! Don't give them any ideas! You stupid noob.

Re:I know the moderators will tag me, but...

[Decaff]

I know the moderators will hit me for this, but from the sys admins point of view - ant is evil!

Ant isn't for sys admins.

It is simply another build program to learn. The C and C++ stuff for the kernel is "make" - now I have to learn "ant" for Java programs.

As Java isn't used (yet!) for kernel stuff, what is the problem?

I am not against learning new things, but where some people see XML as a good thing - I think it sucks! I despise any time I see a config file that is XML. XML is for the "parsing impaired" - use a plain text file like everything in /etc !!!

Sorry, wrong way around. XML was designed specifically from the start to be parser-friendly. There are very simple APIs which handle any XML file such as SAX. /etc is a very good example of what is wrong with plain text files - no single program or API could ever handle the range of formats of files there! Incidentally - XML IS plain text.

SCons

[Chmarr]

If folk want something better than make, but don't want something so Java-centric. Have a look at SCons.

The software, as well as the confuguration files, are actually Python. But, you won't notice until your build requirements get quite complex.

Scons keeps track of dependencies using MD5sums on the tail nodes. This takes up a bit more processing time, but more than makes up for it with highly-parallelizable builds (SCons + distcc totally rocks), guaranteed correct builds (never do 'make clean' again!).

We've just converted a project from Make to SCons, and it's cut our build time by about 40%. I might even be able to convince our java guys to try it out, too :) (Yes, SCons handles Java as well as C or C++).

Re:SCons

[vivek7006]

In your experience how stable is SCons? Their website gave me the impression that it is a beta software.

Re:SCons

[stefanb]

Scons keeps track of dependencies using MD5sums on the tail nodes.

I'm not familiar with SCons, but this doesn't compute. The problem with make isn't with recognising changed files, it's identifying dependencies in source files (hence 'make depend').

'make clean' is an artifact of changes to the build environment that are not properly represented in the dependency graph. Looking at SCons, it appears the authors have a few neat ideas, but I fail to see how a different management tool will help in working out the dependencies.

ant is nice if you want to work around the inefficencies of starting up multiple VMs from the command line (the inefficencies should be addressed on the OS/VM level), and if you want a cross-platform build environment if you can't establish a POSIX shell environment. Not particularly bad, but to really brilliant either...

Re:SCons

[Chmarr]

At this time, pretty damn solid. Yes, it's 'officially' beta, but they're just polishing the edges now.

The only problems I've run into so far relate to SCons being a little too clever for it's own good, and not 'neatly' letting me do things that make could do. I worked around them, once I understood SCons better

For example, SCons lets you generate static and shareable objects using 'StaticObject' and 'SharedObject', and will 'do what's necessary' to make that work on your platform. Eg, under *nix, it will add '-fpic' to SharedObject. It will also refuse to let you add a StaticObject into a SharedLibrary. Which in most cases is just what you'd want to stop happening, but... if you KNOW that in a particular case it will work, then SCons is getting in the way. Eg, we wanted a .a full of .so files that were compiled with -fpic, and then add that whole .a into a .so. Perfectly valid, but odd, and SCon's structure didn't let us do that easily.

But... there's workarounds and they all 'make sense', but require a little more SCons knowledge. The mailing list is quite active and helpful, though.

If your project is straightforward (even if it's HUGE), then there's no problems at all.

Re:SCons

[Chmarr]

I glossed over some of the features, and reasons for them, so let me explain.

The 'md5sum' thing ensures that you're not relying on timestamps. Especially important if you're dealing with several machines that might not have their clocks synchronised perfectly. Furthermore, the 'signature' of any of the build nodes is the concatenation of the MD5s of all it's dependencies, PLUS the command lines to generate that node. Change ANY one of those, including changing any compiler flag, and the signature doesn't match, and the node gets rebuilt.

SCons does its own inherent dependency checking, and caches that, too, plus recording the signatures of all the files needed to generate that information. If any of THAT changes, it will re-get dependencies, and then use the result to determine if a node needs to be rebuilt. This, plus the previous part, makes 'make depend' unnecessary. SCons always gets it right, which cannot be said for Make, or (I think) even Ant... (We currently DO have problems with Ant after a 'cvs update'.). I'm pretty sure that you can make your '.depend' files depend on the files that were used to generate it, but the solution for that for Make (unsure for Ant), is UGLY. SCons handles it all automatically.

SCons also handles cross-platform builds. It's actively tested under *nix and Windows.

Really, if any product deserves to be called a 'category killer' for this category, it isn't Ant, it's SCons.

Re:SCons

[Anonymous Coward]

I absolutely, positively do not want to install python on my system. Especially just to build non-python programs. I'm quite happy with make.

Re:SCons

[jshowlett]

Why? If a tool does a job exceedingly well, who cares if it requires ${INFRASTRUCTURE_I_HAVE_A_PET_PEEVE_ABOUT} ?

Re:SCons

[bruthasj]

If you are using scons to do what it claims out of the box, then you might be fine. But, if you try to extend it like it should be able to, you'll run into problems.

Scons is also confusing if you want to do anything more than make c files out of one directory.

Also, the community surrounding it is insufficient to provide full support. Go in with the knowledge you're going to have to open the covers to see what's inside. (ie. you have to be good at Python, even though they claim otherwise.)

Re:SCons

[Chmarr]

I feel the need to disagree with you on all those points.

I've created several extensions, custom Builders in SCons parlance, with fairly little difficulty.

SCons not only handled traversing multiple directories with ease, it did a MUCH better job doing so than Make provides. in Make, if you have dependencies deep within two distinct directory branches, you have to make one directory depend on the other, making a parallel build bottleneck quite badly. SCons figured out the EXACT dependency itself with no configuration.

The community is pretty active, much more so than some other communities I'm on. No guarantee that they'll be able to solve any particular person's problems, of course.

Agreed that knowing basic python, like how to create an IF structure to create a variant within the SCons configuration files, of example, helps. You certainly don't need to know how to do more advanced Python. However, I've NOT had to look at the SCons source code yet, and the Make structure I converted from was quite complex.

I WILL admit that if you dive in expecting to create a complex build structure as your first project you're going to flounder a little. This is also true of Make and Ant (from what our Ant-using developers tell me). So... just like anything, start simple, or learn to hit the ground running.

Re:SCons

[bruthasj]

If you disagree, please look at the mailing list archives with posts from "Jeff Pitman" and help me with my issues.

thanks,
jeff

Re:SCons

[Chmarr]

Fine! Answered! :)

Re:I know the moderators will tag me, but...

[snookerdoodle]

'Not sure why you woulda gotten modded down had you just put in the first two sentences because you have a valid point. OTOH, it might be more relevant to a discussion of Ant vs Make than a discussion of a book review on Ant.

I.e.: I see your point (see below), but what does that have to do with this book?

I've been "bilingual" for awhile and that fact landed me in a mixed C++/Java project last year. Some consultants had built some of the java stuff using Ant and it just made things confusing and unnecessarily complex to have builds using both Make and Ant. Eventually all of the build.xml files were converted to makefiles and we were homogeneous.

Mark

Perhaps a lot...

[Anonymous Coward]

So, what could a book say that is already not available online?

There is so much online information on Linux that it requires expertise just to find it.

As a novice Linux user, I'll take a good Linux book over the Web every time.

Who cares what you think

[poopdeville]

You fail it.

Re:I know the moderators will tag me, but...

[Baki]

besides, what's stopping you to use make should you create your own java sysadmin tools?

i don't assume that, as a sysadmin, you should have to deal with (java) software that others write, except maybe to install it.

JUnit?

[FranTaylor]

One of the nicer features of ant, one that doesn't seem to be mentioned, is its JUnit integration. There is a very JUnit reporting mechanism built into ant. It wasn't exactly what I wanted, but with a fairly small modification (superclassing, no actual code mods) I was able to get it to work just as I wanted it to.

Hey, make fiends, try this: add new functionality to make, extending its syntax. PITA, you say. It's very simple with ant.

Re:JUnit?

[mark_lybarger]

ant made a mistake wrt junit. they include the ant task as part of the ant distribution (in the ant/lib folder), but they don't include the junit.jar. thus, a _standard_ install is not able to do junit testing.

you don't have access to always go around putting extra jars into the ant lib folder or even the ~/.ant_lib or whatever private folder ant also looks at for its main classloader. you can nearly assume you can do the following:

cvs co projectname; cd projectname; ant

that's about it.

a good goodbye to make forever

[Anonymous Coward]

Given that I've spent about 1 year working on legacy makefiles at different companies, I am glad to see make phased out.

I'm much more productive using something other than make for building multi-project software packages.

Additional Ant Goodies

[Robotron2084]

This book is also available online at O'Reilly's Safari site.

Also, although Ant is used mainly to build Java, it is NOT java-centric. It can be used to compile any language.

Re:Additional Ant Goodies

[steve_l]

It is biased towards java though. All the stuff related to testing and executing and packaging is built around ant. And as most of the volunteer developers work on java projects, it is the java centric tools that get maintained the most. Same goes for the SCM tasks; the SVN and CVS tasks get much more effort invested than, say, the SourceSafe ones.

steve

(ant developer)

Forgive my ignorance...

[Kr3m3Puff]

Can someone explain to me the difference between Ant and Maven? I have noticed almost everyone supports Ant, but there are people who also do Maven. Are there any advantages, disadvantages, are the different but overlap greatly, and why does Apache support what appears to me to be two competitive technologies...

Very confusing to someone who just has a high-level understanding of Apache world.

Re:Forgive my ignorance...

[Bj�rn]

Here are two URLs that explain Maven better than I can:

• http://www.onjava.com/pub/a/onjava/2004/08/04/mave ntips.html
• http://www.theserverside.com/articles/article.tss? l=MavenMagic

Basically Maven is an extra layer on top of Ant. Or another way of putting it is that Maven is way to program Ant in an XML based language called Jelly. Maven has the concept of repositories and versions, so that you can say that a particular jar file requires specific versions of other jar files in the repository. If you find yourself writing a lot of similar Ant code, Maven is definitely worth a look. It takes a little while to get into, but once you do it is fairly straight forward.

Re:Forgive my ignorance...

[BigTimOBrien]

Maven sits at a higher level than Ant, where Ant is really just a system for defining targets and dependencies between targets, Maven abstracts build logic across all project by way of plugins. So, instead of telling your 10 separate web application projects how to compile, test, and generate a WAR file, you simply describe each project in a common Project Object Model (POM), you then invoke something like the WAR plug-in, which takes your project's model and generates a WAR file. Long story short, in Ant you need to give each project explicit instructions for every task in your build.xml file. In Maven, you rely on common logic shared by all projects.

Confusing? Well it's not something you should rush into. IF you would like more information about Maven, take a look at Maven: A Developer's Notebook, Vincent Massol and I just finished this bok in March of this year and it was released last month. It is a good introduction to Maven.

And, if you were wondering, the developers are doing a better job at project documentation for Maven 2.0. It is shaping up well.

Does it apply to NAnt too?

[Space_Soldier]

There is a port of Ant to .NET available from here. It had a great following until MS announced MSBuild. Unfortunetly, MS put RIP on it when they created MSBuild, which comes with .NET. Visual Studio project files are MSBuild files.

Re:I know the moderators will tag me, but...

[Imposter_of_myself]

Unfortunately, sys admins run across ant when they work with tomcat. If you don't want to pull down byte code off the web - you want to "build" tomcat to get the latest version - then you have to encounter ant. XML, in my book, just makes reading config files more complicated. Now, instead of looking at the actual "data", I have to sort through a bunch of noise - the tags.

My gripes about ant

[demilurker]

1. The pulling teeth part! You need to use the 3rd party ant-contrib package to even get basic if-else and for loop functionality. And even then, since it's XML, you end up with crap like this:

<if>
<equals arg1="${foo}" arg2="bar"/>
<then>
<property name="bat" value="barf"/>
</then>
</if>

... give me a break.

What I'm always hoping for is a build system that is just a perl / python library, so I can write the build script in a real language and call the build system when I want to.

2. Ant's simplistic rebuild-on-file-mod way of doing things misses things. Say class A.java calls class B.java. You build and all is well. Then you change B.java's interface, but not where it is called in A.java. By right your build should break now, but ant only rebuilds B.java, because it's file mod time is updated; it does no analysis of what depends on it and should be rebuilt. So you run the program and if that line of code gets called you get a NoSuchMethodError, examine that stack trace, and manually delete the .class file or somesuch. What ant needs is something like what gcc -M does for make.

... hey, I use it, but it's the year 2005 for crying out loud.

Re:My gripes about ant

[cratermoon]

If you knew what you were talking about, you'd know it was javac that does that, not Ant. In fact, Ant has the task that helps address the shortcomings of javac.

Re:My gripes about ant

[abdulla]

Seems like you should check out SCons. As far as I know, it takes care of all the gripes you have with Ant. The build scripts are in python, so you code in python right there. And SCons uses md5sums and does all the dependency checking for you.

Re:My gripes about ant

[metamatic]

http://make.rubyforge.org/

Re:My gripes about ant

[adtifyj]

make provides you the real language that you seek. You may have used it before, its called /bin/sh.

Re:I know the moderators will tag me, but...

[Procyon101]

XML is for the "Parser impared" in that the notation is designed to make it simple to write parsers, rather than make it simple to use the syntax. Parsers are not difficult to write, and once 1 person has written one, everyone can use it... on the other hand, the programmer gets stuck with the syntax of the data forever. XML syntax is mostly noise with a bit of data thrown in and delivers on none of it's promises.

I completely agree

[HelloKitty]

Has anyone used Ant with other languages besides Java?

If it's Java only, then I just don't care... scons is pretty neat for c/c++, haven't used it on other languages though.

Re:I completely agree

[Anonymous Coward]

I haven't directly invoked gcc with ant but I have used ant to run build scripts for a c++ project (the scripts run autoconf/automake).

It's a useful thing to do in a mixed j2ee/C++ development environ to provide automated building, unit testing, and documentation for all projects using a uniform set of ant targets (i.e then can run it all remotely with cruisecontrol or similar app)

To run bash shell scripts or do c++ compilation there is an ant extension on sourceforge http://sourceforge.net/projects/ant-contrib/

Re:I completely agree

[DrEasy]

Well, I don't know if that answers your question, but I use Ant to package my Firefox extension. It has commands for creating zip files and Jars, as well as creating and moving folders, which is all I need.

Ant is extensible, so I wouldn't be surprised if a lot of tasks have been made available for programming languages other than Java.

Re:I completely agree

[ebuck]

Yes.

I've used it with C and with C++ to build shared libraries on a Linux platform that were later bound to JAVA via a JNI interface.

Take a look at ant-contrib, they have a c/c++ compiler ant task. I'm not sure about other languages, but here's some other things I've done with Ant that aren't really JAVA centric.

I've used Ant to transform XML to other various things, including DocBook build systems to various HTML, PDF, TXT, etc. files.

I've used Ant to verify that documents conform to their appropriate DTDs / XSDs. Java developers take note! Next time you have an XML file driving some sort of configuration, validate it. It will save you a lot of time.

I've used Ant to wrap source code control systems for those who can't seem to remember various tagging and branching policies. I've also used it to automate the running of your unit tests pre-check-in, diallowing the check-in on failure.

And you can write your own tasks for Ant. I've only had a limited amount of experience with this, but I wrote a (rather simple) task to drive an (obsolete) in-house develpment workflow system (post check-in). We were going to write some extensions to have it install our database tables (via JDBC) but I left that group and the idea dissolved.

Finally, you can write custom loggers for Ant which become very useful when integrating Ant into other pre-existing systems or when performing custom reporting of the current build status. One such logger I wrote reported the current build state (running / completed), the last successful build, the output from the last 10 runs, where the build died (if it failed), and who to blame (from the last check-in) via dropping html files on a web server.

So, Ant is the build system of choice for JAVA, but you can certainly do other things with it. Just don't expect it to work like Make, because it's strengths and weaknesses are in differnt "places" than Make's strenghts and weaknesses.

Re:I completely agree

[jag164]

Oh goodness yes. We use ant and cpptasks in a modularized product of about 600 files, consisting of various shared/static libs, unit tests, and execs. With some work, it plugs into eclipse's CDT too. Simply automates doxygen builds, and rpm builds. Oh yeah, a few modules have to be cross platform so the same build.xml works on XP and MSVC7 with those targets. All this in about a five hundred line build.xml file that took about 3 hours to initially setup and 5 minutes here and there to maintain.

Good stuff...

Re:I completely agree

[mtg101]

We use ANT at work to build the J2EE server, the J2SE server comms module, the J2ME Java phone client, the C++ M$ Smartphone and Windoze Mobile clients, and the C++ Symbian Series 60 and UIQ phone clients, the C++ win32 test system, and some other C++ clients for propriety phone OSs. It's really rather versitile.

Re:I completely agree

[david.given]

Alas, ant requires Java, which immediately disqualifies it for my purposes. (Maven, too. scons uses Python so it's not as bad, but it's still a fairly weighty extra requirement that I'd rather avoid if I can.)

However, I would be interested to know how to use Ant with the CDT for other purposes --- it's got to be better than Make, particularly if Eclipse will automate it. Do you have any references?

Re:I completely agree

[gedhrel]

"got to be better than make". I'm not so sure. There appear to be a lot of people at loggerheads over ant vs make in this thread. Perhaps they fail to see the distinction, which I'm sure you're familiar with: compiling C (and C++) is harder than compiling java, by a long chalk.

With ant, you use the javac task and let it get on with it. Generally, no more than 10% (upper limit) of my ant scripts are actually devoted to compiling java. That's because platform dependencies, defines, conditional includes and library definitions are not something that tend to be a problem in the java world. Java itself is not a difficult language to compile, compared (say) to C++. Conditional includes, etc, _have_ been a problem with every major C / C++ project I've worked on.

We use ant where possible to produce a one-command "get this project running" target. The ant scripts can be quite complex, but they're predictable and (since we work with remote project partners across europe) such a setup makes troubleshooting far simpler: we require developers to provide as little external environment as possible, and have ant recreate it. (It turns out that when I did this for the first time I was basically reinventing what maven does; however, we still find maven opaque in comparison.)

The situation is much more complex with our C++ stuff, which consequently tends to live in-house. It's possible to get a makefile working that'll cooperate with a posix-compatible environment in order to work. If you can guarantee that environment will exist, you might well be better off sticking with make, or look at jam.

Re:I completely agree

[MemoryDragon]

You do not see the full picutre. ANT and other newer build systems like maven try to be as platform independend as possible (which make clearly isnt). platform independency of a build system is not that it runs on any platform, but that it behaves exactly the same on any platform it can run no matter which programs are installed. With make, you need external ftp, tar etc... whatever, because make is a glorified batch system. You will run into troubles once you make on a platform you do not have those external tools or if the external tools behave differently.

Newer build systems solve those problems to a big degree by delivering 99% of those tasks themselve and even if they do not, normally they force you to program extensions in a manner that they are platform independend.

Those things are the biggest advantages of build systems like maven or ant, compared to make. All you need is a VM and you are set. With make, you need the same make (there are vendor differences), you need the same set of tools, and if you are unlucky even a rather similar filestructure.

as a sysadmin, dealing with XML and java

[wsanders]

Well, as a sysadmin in a big J2EE shop, I spend about half my time fixing other applications' and other peoples' broken XML, so dealing with ant is no different. Most people around here see ant as a make-equivalent.

I was under the impression that XML was designed to be written and used by machines with little intervention from people. Some people insist it's a programming language. I hope those people burn in heck.

Re:as a sysadmin, dealing with XML and java

[ebuck]

Well, your impression is wrong, you desires are right, and you're ignoring a great tool to fix your problems.

XML was always meant to be human readable and human editable, that's why it isn't in a more compact binary format, and the tag names are typically full words describing the surrounding contents. Machine handling is very important too, and that's where the strict syntax comes into play.

XML isn't meant to be a programming language. Actually, in ant, XML is the configuration file, not a scripting language, although the configuration is one to define a target and it's associated actions, so I can see how the difference can be overlooked. Those that do use it as a programming language should properly be disposed of in a manner similar to your description.

Ant is the solution to your problems, assuming that you can solve your problems in house. Have part of the build verify the XML configuration files by verifying the XML files against their respective DTDs or XSDs. There's already an ant task for it, XmlValidate. It shouldn't be much of an issue to set up and use, unless you knowledge of XML does not extend into validation. But even then, you only have to set it up once to use it a thousand times.

"the rare breed of category killer software"

[Anonymous Coward]

Does this sound like hypespeak to anybody else? Sheesh, I'm not sure what that even means. "category killer?" Is the category dead and buried, no longer exists, or did it just get a knock on its phunny b0ne? :)

Ant is simply not a Make replacement

[Zenin]

I know Make. Make is a friend of mine. Ant is no Make.

Ant can automate Java builds slightly better then a .bat or shell script...but that's sadly about where it ends.

The major problem is that Ant has no built in dependancy system. Dependant tasks don't count as they have no targets and thus nothing to check for dependancies. All real dependancy checking in Ant is embedded in the Ant task code...which varies task to task and worse yet has little to no documentation (not even documented to be taking place or not!). Examples are javac, zip, copy tasks.

Ant + Javamake gets closer to a Make replacement, but not completely.

But really, to be accurate Ant is a portable .bat/shellscript replacement...and that's it. Claiming it is a legit replacement for Make is to completely misunderstand what Make actually is and does. That doesn't mean Ant isn't useful, it is, but it is no Make.

Re:Ant is simply not a Make replacement

[Bj�rn]

Ant has no built in dependancy system

No dependency system? Whatever do you mean? When writing Ant files you configure dependencies between Ant targets. A target in turn has a number of tasks that do the actual work. The if or unless attributes in a target can be used to decide whether to execute the tasks within a target or not. So if you sometimes don't wish to execute a javac, zip or copy task you should place that task in a target with either the if or unless attribute.

Re:Ant is simply not a Make replacement

[Anonymous Coward]

As the parent post stated: "Dependant tasks don't count as they have no targets and thus nothing to check for dependancies." Make has implicit dependency chaining. Ant can't even touch that.

Re:Ant is simply not a Make replacement

[ebuck]

Ant does have a dependset task and an uptodate task for explicitly defining dependencies, in addition it has the condition task to bind several dependencies together in a logical manner.

But still, ant is no make. It's not ant's fault, but more a fault of the underlying reason that make is both powerful and problematic. Make leverages the power of it's shell and all the utilities that have been written explicitly to make shell scripting possible. Make (by itself) doesn't do a lot except call a bunch of programs using a reverse-chaining algorthim. It's the value of all those shell programs that give make it's lifting power.

Ant can call a platform's shell programs too, but then you run into difficulties in communicating rich information between the programs, and ant becomes just a funny, XML configured, JAVA version of make. That's why ant's logging capabilites far outstrip those of make, because it's logging facility can differentiate between errors, warnings, and informational messages, routing them to text files, xml files, databases, and web pages.

Re:Ant is simply not a Make replacement

[adtifyj]

And when Ant finally introduces proper dependency support, the wheel will have turned once more. Im looking forward to the complaints about Ant's dreaded XML problem. Maybe 0x09 will be fashionable again in my lifetime.

Re:Ant is simply not a Make replacement

[MemoryDragon]

Dependency is not such a big deal in a pure java environment. Usually you have a set of jars you program against and you compile a root dir. ANT has a tad more in this regard, you can enable change set checks to keep compile time down. But in most java projects you do not have myriads of inter dependencies which you have to resolve manually by various compile steps. The reason for this is, that java compilers are so fast, that even half a million lines of code and a few thousand classes is not a big deal to compile.

Normally the development way is like that, that you program in eclipse or something similar, where an incremental compiler does an automated check on your sources and keeps them in track, and from time to time, you start ant (which integrates due to the xml syntax like a normal project into almost any java ide) and you hammer it to generate the artefacts and do a final check, that the program builds normally, also sometimes it is used for deployment via ssh/ftp etc...

But the key is, normal dev cycles most of the times do not happen in ANT, it is more a supportive, generation and deployment tool in its current usage. Thus the big need of a C++ like dep check is not really there in java.

Re:I know the moderators will tag me, but...

[Anonymous Coward]

Right - noise. The noise is what tells both the human and the client program what the data is for. When you look at /etc/passwd, is there any clue what all the fields are for? No. It's a convention that allows you to know that the first field is the username, etc.

With XML, all data is *tagged* according to what it means. The tags are predefined, by the programmer, or by a group agreement, and the meanings are, or should be unambiguous.

I don't think XML should be used everywhere - there is a lot of redundancy. Within a program, it's a waste of memory, processor and bandwidth, but between systems that are written by different programmers, it's one of the better ways to move data and have it be understood.

For configuration files, many people will be looking at them, so you want them to be understood, right? Without a lot of backstudying? Good for that, although I really prefer the Apache config file layout as an example - has the relative unambiguity of XML, but it's a lot more concise, with very few paired tags.

Additional Edit-Compile-Debug Goodies

[Anonymous Coward]

"Also, although Ant is used mainly to build Java, it is NOT java-centric. It can be used to compile any language."

For those languages forever tied to the edit-compile-debug cycle.

Where's the GUI?

[ubergeek65536]

Another great tool that I'll never use.

Re:Where's the GUI?

[ebuck]

Asking for every tool to have a gui is a silly as asking for every real-world tool to have a computer.

Machining tools benefit from computer interfaces, screwdrivers do not. Yet I have never seen anyone open a can of paint with a machining tool, but the screwdriver is flexible enough in design to do this task, even though it was not specifically designed to do it.

GUIs are great in many aspects, but they can be just as limiting in others. Pick the right tool for the job. Your build system is probably not even well defined by yourself, so don't take any risks in having a GUI define what you must do, it could be totally wrong.

Re:Where's the GUI?

[Anonymous Coward]

Both Eclipse and NetBeans provide editing tools to ease the task.

Rake

[Anonymous Coward]

If you're looking for a very easy make/ant alternative, look at rake, it's written in ruby, and it's damn easy to use. No fiddling with trailing whitespace or xml.

http://rake.rubyforge.org/files/README.html .adam

Re:I know the moderators will tag me, but...

[Anonymous Coward]

What were those promises, then? Ease of data exchange? Check. Headers alerting the receiving system as to character encoding? Check. Where to find out data format information? Check. Clear separation of formatting chars and data chars? Check. Clearly defining for the rest of the world the meaning of your data? Check.

Then you look at (nearly) any *nix config file, and find that it's a mixture of line noise and printable chars - yeah, that makes sense.

Just buy FinalBuilder - it's worth it !

[Anonymous Coward]

FinalBuilder uses XML for the build scripts but gives you a nice drag&drop UI, so you never have to write another tag again.

http://www.finalbuilder.com/index.html

Ant is a Misapplication of Technology

[aldheorte]

It has been a source of constant dismay to me that something has not arisen to replace Ant in for Java builds. The Ant project has created a terrific mess where people essentially *program in XML*. XML is a set of data structure markup semantics, not a programming language. No potential of repurposing ant build files exists, another knock against using XML. Therefore, from a software architecture perspective, Ant is one of the biggest misapplications of technology I have seen to date and despite that, staggeringly popular. I rest my case and shake my head.

Re:Ant is a Misapplication of Technology

[argent]

Somewhere in the Java code base are the seeds of the first computer virus to ever cross from the silicon to the meat brain. It's the only conclusion I can come up with to explain all the terrible design in just about every significant Java project I've seen so far. Ant is merely the tip of the iceberg.

Re:Ant is a Misapplication of Technology

[holloway]

Dude, you're wrong. As far as appropriate uses of XML goes who do you think would be the authority on that? The W3C who created XML? James Clark? Dave Pawson? People from SGML?

The answer is that all of these groups and people created a XSLT - a functional XML programming language. That's right, the people who produce the XML spec also produced an XML programming language, just like Ant.

XML is a meta-markup language, a series of rules, and you can represent data files as much as you can represent programming languages because they're all text files. You may as well say MAKE scripts are also a misuse of technology, being ASCII, if you're going to claim that ASCII should only be used for configuration files.

With constructs that can be compiled who's to say XML can't be a programming language? The W3C say that depending on the domain it can be, and they disagree with you.

Re:Ant is a Misapplication of Technology

[BigTimOBrien]

Actually there is something that is the natural progrssion from Ant - Maven. Maven is useful because it abstracts common build process across multiple projects by using plug-ins. Maven 1.0.2 is somewhat klunky, and work is progressing on Maven 2, but if you are looking for more information, take a look at Maven Project and buy Maven: A Developer's Notebook.

And, to be fair to Ant, recent versions of Ant 1.6 have added mechanisms to reuse build logic. Really, go check for yourself on the Ant project page.

Re:Ant is a Misapplication of Technology

[Anonymous Coward]

I can't believe that people essentially *program in text files*. Text files are a way of storing unstructured character streams, not a programming language.

A declarative language is really an elaborate config file anyway.

Anyway a cross-platform build tool(even windows!), to which I can generate the XML files using other XML tools, sounds pretty good to me.

Java and XML create a platform agnostic environment. And XML is as close to the informational superfluid as you're likely to find.

Re:Ant is a Misapplication of Technology

[aldheorte]

The authority on it is what constitutes the simplest solution to the problem (of builds in this instance). Introducing the markup overhead of XML for no practical benefit and essentially requiring programming in what is supposed to be a *configuration* file is certainly not that simplest solution. In this manner, XSLT, as well, falls down since much simpler and more powerful functional programming languages exist.

The W3C is not an authority on anything. They are more of an idea reviewing and specification factory and, to the extent that what they propose is adopted, those things that are adopted en masses gain a certain sense of 'authority' by common usage. However, this does not directly lead to the conclusion that those things are the best solutions, nor that their ideas are very good.

Using Ant with PHP

[mgkimsal2]

We're using Ant with PHP. It's working out just fine. I'm preparing an article on an interesting technique we're using to manage multiple projects with a bit of sanity, and Ant is the cornerstone. If you're interested in a draft, email me at mgkimsal@gmail.com or check http://webdevradio.com/ in the next few weeks.

Java is the real problem

[Anonymous Coward]

The problem isn't Ant - it's Java.

Other than Java's appalling use of resources, why are so many Java programs so crummy? They're supposed to be portable across OS's, but I've never seen a non-trivial one that was. Every one seems to require JVM version 1.2.3.4.5_06_04b built at 1243 on the 14th, or it pukes all over the screen.

Ugh.

- Not a Java fan

Re:Java is the real problem

[mparaz]

If you don't mind forking to run external tools, instead of running in the same VM, you could use the native build of ant using gcj as sponsored by Red Hat.

Re:I know the moderators will tag me, but...

[Decaff]

I have to sort through a bunch of noise - the tags

As against what? Some arbitrary and undocumented text format?

Re:I know the moderators will tag me, but...

[Decaff]

XML syntax is mostly noise with a bit of data thrown in and delivers on none of it's promises.

What utter nonsense! Of course XML has delivered. An example is the Open Office document format, that is XML-based, easy to transform to other formats (such as PDF), and because it is XML (open and documented) is also now accepted by KDE office applications.

Re:I know the moderators will tag me, but...

[Decaff]

besides, what's stopping you to use make should you create your own java sysadmin tools?

Nothing, except that Ant has a huge amount of support for Java and associated tools. Almost every tool and product developed to assist Java developers provides an Ant library to add functionality. To work with 'make' would be a huge step backwards.

Tab Delimiting Vs Space Aligning

[buckhead_buddy]

Being in the Mac OS X world, there's a big chasm between the practices of tab-delimiting and space-aligning data.

Unix originated the idea of the tab-delimited file as a simple way to delimit fields and records (with a character that was unusable in the field data). But it quickly fell from favor because text editors were inconsistent when 1-byte of data did not equal 1 space on-screen. The tools failed them so they abandoned tab-delimited file formats.

Oddly enough, the ancient Mac OS (pre-OS X) users had big problems with columnar-aligned, fixed length formats for file formats, mailing lists, and database dumps because the users didn't realize that the proportional fonts weren't properly aligning the data. The tools failed them so they adopted tab delimited file formats.

Now that Mac OS and Unix have sort of come together, I find the quality of tools on both sides could use some improvement. I've found a wealth of data editiing tools that make make editing columnar, tab-delimited, or XML data a pleasure to work with.

The real answer is to spend time to improve the tools if you're not the one in control of the formats.

Re:I know the moderators will tag me, but...

[Anonymous Coward]

There are very simple APIs which handle any XML file such as SAX.

It's a more complicated API than the one for pulling lines out of a text file. (ok, depending on the language you're working with.) /etc is a very good example of what is wrong with plain text files - no single program or API could ever handle the range of formats of files there!

Whereas with XML, you can read it all in one program, but you can't really understand what's there. What good does it do you?

I suppose you could put it into some GUI with a tree and such to make it human readable, but with plain text files you don't have to.

Incidentally - XML IS plain text.

XML is no more plain text than HTML or RTF. It's formatted text. Usually poorly formatted text.

So MAKE it useful for system admins.

[argent]

no single program or API could ever handle the range of formats of files [in /etc]!

And converting them to XML won't change that, because unless you're dealing with files that are very much more complex than the ones in /etc, it's harder to write a description for them so that you can parse them using a general purpose XML library than it is to parse them using the traditional UNIX tools... and having parsed them you haven't actually started to do anything useful with them.

#!/bin/awk

BEGIN { FS=":"; }

$3>1000 { print $6; }

If you want people to treat XML seriously, give them the tools to do something useful with them. Before you produce another "ant", give us xmlgrep, xmlsort, xmlawk, xmlsql.

#!/bin/xmlsql

create table passwd (
name varchar primary key,
passwd varchar,
uid integer, gid integer,
gcos_fullname varchar,
gcos_phone varchar,
...
);
select bind(passwd,'/etc/passwd.xml');

select home from passwd where uid >= 1000;

This would actually be useful, but it's more complex than the "harder to parse" version. So what do you do? Well, you could create a whole new set of files to manage, and do this:

#!/bin/xmlsql

\i /libdata/passwd.sql

select home from passwd where uid >= 1000;

Or maybe:

#!/bin/xmlawk
#include "/libdata/passwd.awk"

$uid >= 100 { print $home }

OK, after a while, you end up with a bunch of files in /etc that are no harder to deal with and a bit better documented, but it'll take a fair amount of work and noobody's really going to take you seriously until you do it. But... there's the other problem. Once you do that, where does ant come in? What useful information can you extract from an ant file? It doesn't even contain a description of the dependencies, which might be useful to know! It's just a bunch of code fragments and calls to build routines only partially described.

Re:So MAKE it useful for system admins.

[Jay Carlson]

Can somebody mod Peter up? Thanks.

But anyway, the problem with

awk 'BEGIN {FS=":"} $3>1000 {print $6;}' &lt;/etc/passwd

is that it assumes that there are no fields that contain a : character. The moment you get a : in gecos or more radically a group name, grep et al fall apart. If we were dealing with CSV files you could see something like

foo,7,*,"bar,baz"

and awk would just cough and die, because there's no good way to explain the escaping conventions to it.

Re:So MAKE it useful for system admins.

[argent]

it assumes that there are no fields that contain a : character.

That's because there are no fields that contain a ":" character. If there could be fields containing a ":" character, then ":" wouldn't have been chosen as a separator. If you put a ":" character in there, then you've created an invalid password file. The standard utilities won't create a file like that, vipw will refuse to write it, and so on.

If you were dealing with CSV files, you'd have the same problem with a malformed CSV file.

Same with XML.

The best you could hope for would be a better error message, but you couldn't even depend on that. Even with full-on relational databases and SQL queries, you still get applications coughing and dying over invalid tables, and you still have to go in with forensic tools and look for the corruption.

God knows I've done enough of THAT.

Re:So MAKE it useful for system admins.

[ebuck]

Well, if your only talking about one file with a static format that won't change over time, then feel free to define the right format for the file and write your parser once.

But many programs are much more dynamic than UNIX's /etc/passwd file. For files that are likely to be needed in a new format at some time, XML coupled with XSL provides a flexible way to move you data programatically into the new format.

You could achieve the same effect with a combination of sed, awk, and sh scripts, but then the burdeon of verification that the scripts worked properly would lie on a human's shoulders, and odds would be high that eventually a mistake would be made.

Also XML provides a facility to pre-verify that the file is in correct format before it is consumed by its intended program. Certainly such things can be done without XML, but there are no gurantees that all programmers build this feature into their executables, and not all programs fail safely when partially configured correctly.

Now, using any kind of formatting you could desire, consider pulling the price of cheese out of document when you have no control over the department that's creating it. With XML you can, because you'll request the DTD of the document, and parse for the tag describing cheese, and it's interior tag describing the price. Sure you could do the same thing with a character delimited field, but when that new guy misformats the document shifting most fields to the right by one index, XML will die on an improperly defined document, while most other methods will read the quantity as the price.

Re:So MAKE it useful for system admins.

[Jay Carlson]

Actually, you *won't* have that problem with an XML file, since XML is closed over Unicode. Assuming you have a valid Unicode string, you can stick it in an XML attribute or an XML PCDATA section, and your output encoder will happily stick in <s and other ampersand-quoting-nonsense as appropriate. Hell, if it's smart enough, it may switch over to CDATA sections if it looks more efficient.

Oh, you're just blatting out XML with printf? You deserve to lose, then.

That's the difference here. You're depending on vipw and pals to carry around this rule that you can't have colons in gecos, but it's not really written down anywhere. I'm pretty sure I've seen systems that have borrowed other passwd fields and shoved stuff like password expiration goo into them; now every utility that operates on /etc/passwd has to know about the sub-field delimiters like ;.

Or not. The way most of this stuff works is that most of the time, the scripts work, and if you push it too far, everything breaks and somebody points at the "Worse Is Better" bumper-sticker affixed to the 3B2.

I dunno. I mean, I have great respect at a paradigm that works well on my 68000 boxes. But it's looking a little tired when crap like creating a file named "--help" or "-rf" in your home directory leads to massive lossage. Let alone "--help -rf". When's the last time you wrote a shell script that fully dashed its actual intended arguments to programs? Oh wait, is "--" as end of options actually POSIX? I forget.

Don't get me started on SQL. I'm still ready to kill MythTV for its attitude of "let me automatically upgrade your schema for you! on every program start!".

Re:So MAKE it useful for system admins.

[Decaff]

it's harder to write a description for them so that you can parse them using a general purpose XML library than it is to parse them using the traditional UNIX tools...

No it isn't. Just stick to the simple XML guidelines and any XML parser can parse them.

If you want people to treat XML seriously

Almost everyone already is treating XML seriously.

, give them the tools to do something useful with them. Before you produce another "ant", give us xmlgrep, xmlsort, xmlawk, xmlsql.

All this is already there. There is a standard XML dialect called XSL that includes searching, sorting, and extracting, and there are other XML dialects for interfacting with SQL. These features have been around for years!

Re:So MAKE it useful for system admins.

[argent]

But many programs are much more dynamic than UNIX's /etc/passwd file. For files that are likely to be needed in a new format at some time, XML coupled with XSL provides a flexible way to move you data programatically into the new format.

I don't think you're disagreeing with unless you're dealing with files that are very much more complex than the ones in /etc, it's harder to write a description for them so that you can parse them using a general purpose XML library than it is to parse them using the traditional UNIX tools. For files where parsing the content is inherently difficult (document markup) or where you need self-documenting content (general databases) XML has a definite place... but the grandparent article was specifically talking about /etc/passwd and its kin.

As for denmark:havarti:3.99:104... if you have humans editing your files directly then XML produces different failure modes, but my recent experience with ant and Tomcat is that it can be a LOT harder to track a problem down. If the files are edited by programs, you're better off in either case... and it's really more critical in XML than in other kinds of files.

And in practice, the price of cheese in denmark would be maintained by someone filling a field somewhere that lead to a statement like "UPDATE inventory SET price='3.99' WHERE country='denmark' AND cheese='havarti';". XML lives in the demimonde between scripts and databases, it's useful for sparse markup in documents, it's useful for data interchange, but in Java it somehow has become the format of choice even where it's wildly inappropriate and where its actual strengths are not even used.

Re:So MAKE it useful for system admins.

[argent]

Just stick to the simple XML guidelines and any XML parser can parse them.

I showed you an example of parsing /etc/passwd in two lines of awk. Your DTD is going to be longer than that.

All this is already there. There is a standard XML dialect called XSL...

Show me. Show me something that's already hooked into a conventional scripting language (sh, awk, etc) that can be used without major dependencies (so that it can actually be used during boot) and that doesn't involve writing XML to parse XML.

Solutions involving Perl or CPAN, or Java, or that are only usable with a webserver installed, will not be considered responsive. When you're sitting at a single-user prompt and the only software available is what fits on your emergency boot floppy solutions like those tend to elicit violent reactions.

Re:So MAKE it useful for system admins.

[argent]

When's the last time you wrote a shell script that fully dashed its actual intended arguments to programs?

Never. I write scripts that actually work on real systems. The solution to the "file beginning with -" problem is "start every pattern with ./". Also, "fully quote all variables" to take care of the "file with a space or shell metacharacters in its name" problem.

This problem has nothing to do with XML, by the way. It's a language issue, and it's shared by every language that includes an "eval" operation, and it's never found in any other languages. The problem is (a) recognising when you have an eval and (b) knowing how to deal with it. Writing your Makefiles in XML doesn't change the fact that eventually you're going to pass the script you've generated to an eval to call an external program.

Don't get me started on SQL

SQL has a very simple set of quoting rules. It's one of the least painful script interfaces I've ever had to deal with.

Oh, you're just blatting out XML with printf? You deserve to lose, then.

Which brings us back to the evil that is ant. It's XML that in the ordinary course of using it is edited by humans in ordinary text editors.

That's a horribly inappropriate use of XML. Pure XML is not the worst format to write code in that I've ever used... after all, Make has its own significant problems, but it's close to the worst one I've run into that wasn't deliberately designed to be hard to understand.

Re:So MAKE it useful for system admins.

[Decaff]

I showed you an example of parsing /etc/passwd in two lines of awk. Your DTD is going to be longer than that.

You don't need a DTD.

You showed how to parse /etc/passwd. What about all the other formats? Show me a general awk script that will retrieve, say, the value of the second parameter in any file under /etc.

I can show you a Java or C or PERL program that can retrieve the value of the second element in any XML file.

Show me. Show me something that's already hooked into a conventional scripting language (sh, awk, etc) that can be used without major dependencies (so that it can actually be used during boot) and that doesn't involve writing XML to parse XML.

Why should I be limited to 30-year-old scripting languages? What you are asking is to show something already present in decades-old applications for the handling of XML, which was invented in the 90s!

If you want examples of XML being used in very small environments, take a look at the Fusion Real Time Embedded XML SAX Parser, or MiniXML. The latter has no dependences, and allows the loading and searching of XML with just a few lines of C. You don't need to write XML to parse XML - you can use the very simple DOM or SAX apis. The latter would certainly meet your boot time/floppy disc requirement.

Solutions involving Perl or CPAN, or Java, or that are only usable with a webserver installed, will not be considered responsive.

Just because you don't consider them responsive, doesn't mean they aren't. Java and XML are used in real-time device control for example. That is about as responsive as it is possible to get.

Re:So MAKE it useful for system admins.

[jrumney]

#!/bin/xmlsql

select bind(passwd,'/etc/passwd.xml'); select home from passwd where uid >= 1000;

How about: /etc/passwd[@uid >= 1000]/@home

XSL already exists, and is much better suited to XML's heirachical structure than trying to force it into a relational model so you can use SQL on it.

Re:So MAKE it useful for system admins.

[argent]

show me a general awk script that will retrieve, say, the value of the second parameter in any file under /etc.

I'm not sure I understand how that's meaningful.

What you're saying is "with XML, you don't need to write a different two-line parser for each file".

What I'm saying is, "with XML, you still have to do something useful with the data". And "doing something useful" involves enough work that the potential savings from the file format are negligable. And "doing something useful" is actually harder with XML because the tools that do said useful stuff don't seem to exist for XML.

What I'm also saying is that there are things that XML is very good at. I would certainly use XML over the dots and backslashes of nroff, for example. The way Apache uses XML to provide metadata markup in its configuration files is great.

Why should I be limited to 30-year-old scripting languages?

I haven't asked you to be "limited to 30 year old scripting languages". I apologise for misleading you, or giving you a convenient hook to use to dismiss my comment, whichever applies.

I gave you a couple of examples of scripting languages, and asked you to show me something like that that allows me to script operations using XML as easily as I can script operations using traditional UNIX table-like files. It doesn't need to be awk, it could be Icon, Tcl, ECMAscript, SQL, or even standalone replacements for grep/sed/whatever that could be integrated into shell scripts.

Just show me something that does as good a job as what I have. I'm not even asking for something better, just a tool that makes integrating XML into /etc a viable option. Surely that's not so hard?

Re:So MAKE it useful for system admins.

[argent]

/etc/passwd[@uid >= 1000]/@home

How do I use that in a script?

XSL already exists, and is much better suited to XML's heirachical structure than trying to force it into a relational model so you can use SQL on it. /etc/passwd is a relation. Whether it's stored in XML, in a UNIX-style flat file, or a table in a relational database, it's still a relation. If treating an XML file as a relation is "forcing it into a relational model" then parhaps XML is not the best tool to use.

Re:So MAKE it useful for system admins.

[Decaff]

What you're saying is "with XML, you don't need to write a different two-line parser for each file".

Exactly.

What I'm saying is, "with XML, you still have to do something useful with the data". And "doing something useful" involves enough work that the potential savings from the file format are negligable. And "doing something useful" is actually harder with XML because the tools that do said useful stuff don't seem to exist for XML.

They certainly do. However, for better or worse, the tools are written in XML (or, rather, XSLT). There is no need to write a variety of tools when there is a single language that allows all this processing and transforming to be done. The tools to do this are the programs such as xsltproc.

Here is an example like that you gave for awk: You would put something like this into an XSL file:

<xsl:template match="@id > 100">
<xsl:value-of select="@home"/>
</xsl:template>

Then you give this XSL file as a parameter to xsltproc or some equivalent program.

Re:So MAKE it useful for system admins.

[argent]

However, for better or worse, the tools are written in XML (or, rather, XSLT). There is no need to write a variety of tools when there is a single language that allows all this processing and transforming to be done. The tools to do this are the programs such as xsltproc.

OK, show me the tools.

[XML I don't want to re-quote to feed back to /.]

That's about half again as much code to just do the extraction. It's not significantly shorter than my SQL example. It's significantly harder to read than the original awk version, and that used fixed constants. And you're not actually doing any processing on it, you're just getting the home directory. Awk contains a compact C-styled scripting language that's eval-safe and turing complete. What's the scripting language in XSL?

[pause for googlage]

Let's see... on the one hand I use awk and write "foo(arg1, arg2);" and on the other hand I write something like...

<xsl:call-template name="foo">
<xsl:with-param name="arg1".../>
<xsl:with-param name="arg2".../>
</xsl:call-template>

No thanks. A real scripting language is not an optional feature.

Re:So MAKE it useful for system admins.

[Decaff]

OK, show me the tools.

xsltproc. Tcl XML handling libraries. PERL handling libraries. Java has this built-in.

Awk contains a compact C-styled scripting language that's eval-safe and turing complete. What's the scripting language in XSL?

Almost all XSLT processors allow embedded JavaScript.

If you don't like that, use a DOM API directly in something like TCL or PERL, or even C (or Java).

No thanks. A real scripting language is not an optional feature.

Then use JavaScript rather than the built-in XSTL calling methods etc.

I think we are drifting away from the topic. You are focussing on what tools you are using to process /etc files - I have already shown that XML can be used in small programs and in limited memory and space (such as embedded and boot situations).

I have sort of lost track of what we are debating!

Re:So MAKE it useful for system admins.

[jrumney]

How do I use that in a script?

How do you use ANYTHING in a script? With a program that understands it.

/etc/passwd is a relation.

To people who understand SQL and are unwilling to learn anything else, everything is a relation. The rest of us will use the appropriate tool for the job. XPath for XML, SQL for relational databases.

Re:So MAKE it useful for system admins.

[argent]

You are focussing on what tools you are using to process /etc files

No shit, sherlock. That is the topic.

I have sort of lost track of what we are debating!

I don't know what you're debating. I'm asking what the advantage of going to XML for the files in /etc would be. So far it looks like the answer is one of: (a) larger and harder to read and manage bootscripts, (b) having every script load a large XML handling library before doing any work, or (c) redesigning the boot process around a Java VM, to avoid having to reload the VM or *ML handling libraries every time.

XML has areas where it's a massive improvement over traditional practice. This isn't one of them.

Re:So MAKE it useful for system admins.

[argent]

How do you use ANYTHING in a script? With a program that understands it.

Good boy. Pretty soon you'll catch on that I have been asking for nothing else but an example of a scripting language that understands that (or something comparable to it) all day.

The rest of us will use the appropriate tool for the job. XPath for XML, SQL for relational databases.

create table passwd (
login varchar primary key,
uid integer,
gid integer,
gcos varchar,
home varchar,
shell varchar
);

That is a complete description of the traditional password file. More recent versions have added more fields, like turning gcos into a two to four fields or adding password expiry information... but it's always still a relation. In no system, anywhere, is this a hierarchical structure.

There are files in /etc that have arbitrary key-value pair associated with object, or an inherently nested structure, but this is not one of them: /etc/passwd is a relation. The only reason XML would be an appropriate tool for the job is if you were already using it for everything else.

Re:So MAKE it useful for system admins.

[Jay Carlson]

Of course you deal with filename correctness, because you're smart and have been around long enough to avoid an Apple-style disaster. Unfortunately, ensuring correctness in sh programming is often seen as some kind of fetish. "Gosh, Mister Wizard, what does "$@" mean?". Oh, and I hope the users of your scripts know to put a ./ in front of script args, too...

This problem has nothing to do with XML, by the way. It's a language issue, and it's shared by every language that includes an "eval" operation, and it's never found in any other languages.

That's not quite true. You will run into this problem even in a hypothetical Tcl-without-eval. Witness the newbie mistake of

set new_list "{$a $b $c}"

The horror is that this will work in casual testing.

The real problem is in string tokenization. It's true that eval will do tokenization, so yeah, you'll immediately run into it there.

I agree that Ant is way up there in the <comic-book-guy>WORST. CONCRETE SYNTAX. EVAR.</> wars, no argument. But a lot of the alleged power of one line awk scripts comes from ignoring correctness, especially in the face of quoting. Your typical PHP code monkey is going to commit two deadly range/domain errors per source file, but at least XML tools have the hope of getting this right.

I've been avoiding the issue for so long that I forget whether you can write correct scripts that use find if find doesn't have a -print0. Oh well, I wasn't using that Solaris box anyway.

Re:So MAKE it useful for system admins.

[Decaff]

[You are focussing on what tools you are using to process /etc files]

No shit, sherlock. That is the topic.

No. The topic is "Ant - the Definitive Guide.". Ant is a build tool. A substitute for 'make'. You seemed to want it to be a replacement for 'awk' and 'sort' etc.

I'm asking what the advantage of going to XML for the files in /etc would be. So far it looks like the answer is one of: (a) larger and harder to read and manage bootscripts, (b) having every script load a large XML handling library before doing any work, or (c) redesigning the boot process around a Java VM, to avoid having to reload the VM or *ML handling libraries every time.

You haven't read with I have posted. XML files can be small, and there is no need to load any large XML library (there are libraries that run in a few tens of kilobytes), and you don't need Java - there are tiny XML libraries for C.

What you get are easier to read files (no mess of arbitrary formats) that can have sensible structures and can have their formats extended without breaking existing use.

For a very small system it would have the advantage that you only need one program and one set of libraries to read all configurations.

Re:So MAKE it useful for system admins.

[argent]

I hope the users of your scripts know to put a ./ in front of script args, too...

I'm not sure what you're getting at. If they're providing an argument that I'm going to interpret as a glob pattern, then I'm responsible for making it work right. If they're globbing it, the fact that my program is a script is irrelevant.

And the whole sh-fetish (speaking of which, that's '${1+"$@"}' if you want to be portable :->) issue is irrelevant if you use an eval-safe scripting language like awk rather than Tcl or sh (though Tcl definitely gives you the best tools to deal with eval of any string-based language I know... and that's no accident). Awk doesn't eval strings, if you quote something it stays quoted: talking about quoting problems in awk is seriously missing the point, and the fact that Larry Wall brought them back in Perl (which is basically awk on steroids and Guinness) is one reason I don't want to use Perl any more than I have to.

It's a lot more complex, building an eval-safe language, than you think. Tcl-without-eval (or more properly, Tcl-without-implicit-eval... and I've had thoughts on this area myself) wouldn't look like Tcl, it'd be more like Smalltalk or Lisp. Taking out eval means taking out subst and uplevel and a whole lot of other stuff as well, so your "{$a $b $c}" mistake wouldn't happen... you wouldn't be able to write that. The XSL example someone posted that used [...] to interpolate queries into strings? That's eval, too. The XSLT examples, with ? More eval. Javascript embedded in XML? More eval.

There's other eval-safe scripting languages. Javascript (the language itself), Icon, I've already named a few. I chose awk because it's one that's already being used, but I really do like javascript a lot... as a language in its own right. It's a pity that it hasn't been given a chance to really sing.

Re:So MAKE it useful for system admins.

[argent]

You seemed to want it to be a replacement for 'awk' and 'sort' etc.

Go back and re-read the thread back to my first comment. Re-read the message I'm responding to. I'm not trying to make XML a substitute for make, etcetera, the person who was proposing to replace all the files in /etc with XML is. THAT is the topic of THIS subthread.

DO that, then try again. I'm not going to try and explain what I'm getting at with someone who's having a completely different discussion. For example, the fact that there's an XML library for C is irrelevant if you don't have a scripting language that's aware of it, but I can't explain why if you're thinking about "ant" while I'm talking about it.

Re:So MAKE it useful for system admins.

[Decaff]

For example, the fact that there's an XML library for C is irrelevant if you don't have a scripting language that's aware of it.

But there are scripting languages that are aware of XML, and there have been for years. TCL, PERL, Python, Ruby etc. There are C programs like xsltproc that can be used from any of these scripting languages in the way that any Unix command can.

If the files in /etc were replaced with XML files, there would be plenty of scripting languages that could pick up and extract information from these files with just a few DOM API calls. (have a look at TclXML or TclDOM, for example).

Re:So MAKE it useful for system admins.

[argent]

I'm talking about nothing more than making XML as easy to use as flat database-style files. I'm not even asking you to do a better job, just to provide the capabilities including the ability to write forensic scripts that will work in single-user mode on a minimal system. You keep giving me solutions that impose significant costs in terms of the minimal requirements to actually get my work done.

But there are scripting languages that are aware of XML, and there have been for years.

There are scripting languages with fairly sizable XML libraries that have to be parsed and interpreted every time the script is run. This is unacceptable: too high a startup cost, too many files have to exist and be uncorrupted.

External programs don't help, because parsing and dequoting the output of the external program, and making sure you're not subject to eval- or quoting- exploits in the process of doing so, is actually harder than just reading a flat file safely.

Show me a native interface to a compiled parser (this C-language XML parser you're talking about) that doesn't itself require significant overhead in time, space, or required files.

Re:So MAKE it useful for system admins.

[jrumney]

That is a complete description of the traditional password file. More recent versions have added more fields, like turning gcos into a two to four fields or adding password expiry information... but it's always still a relation. In no system, anywhere, is this a hierarchical structure.

Because its been designed that way. What if you were to merge /etc/passwd and /etc/group into a single file?

Re:So MAKE it useful for system admins.

[argent]

Because its been designed that way.

And there's good reasons for designing it that way.

What if you were to merge /etc/passwd and /etc/group into a single file?

Well, if you were to do that you'd have one file containing two completely different types of records... one for users, and one for groups. You'd have to make sure you queried for the "user" or the "group" record, so instead of asking for "name=='fred'" you'd have to ask for "name=='fred' && type=='user'", to make sure you didn't get the wrong one. It sure sounds like you'd make it more complex to get information out of the file, regardless of the format. No, I think the reasons for having separate files for passwd and group are entirely sound.

Re:So MAKE it useful for system admins.

[Decaff]

There are scripting languages with fairly sizable XML libraries that have to be parsed and interpreted every time the script is run. This is unacceptable: too high a startup cost, too many files have to exist and be uncorrupted.

This simply isn't true. For example, tDOM is TCL wrapper around expat, an XML parser written in C. No startup cost, no interpreting.

Install tDOM, and you get exactly what you are implying doesn't exist: the ability to parse XML that is simple, script-based, and fast.

just to provide the capabilities including the ability to write forensic scripts that will work in single-user mode on a minimal system

Use Tcl with tDOM, for example.

External programs don't help, because parsing and dequoting the output of the external program, and making sure you're not subject to eval- or quoting- exploits in the process of doing so, is actually harder than just reading a flat file safely.

awk, sed sort etc. are external programs that need the same checking.

XML is far safer, as it can be quickly checked for validity, unlike a series of arbitrary flat file formats.

Re:So MAKE it useful for system admins.

[argent]

awk, sed sort etc. are external programs that need the same checking.

I guess you missed the bit where I was talking about writing the code in awk itself, specifically because awk is eval-safe.

For example, tDOM is TCL wrapper around expat, an XML parser written in C.

You know, I have been asking for a pointer to specific tools like this from the very first message in this thread. Why yes, that was my first request, scripting tools for XML. Why has it taken you this long to provide a link (hmmm... actually, you didn't provide a link, but I suppose google will get me what I need)?

Re:So MAKE it useful for system admins.

[Decaff]

I guess you missed the bit where I was talking about writing the code in awk itself, specifically because awk is eval-safe.

Much of what you were specifying you wanted to do could be done by writing in XML itself (e.g. XSLT) with a single processing tool (such as xsltproc). But, apparently, that was not allowable. Whereas using scripts like awk is.

You know, I have been asking for a pointer to specific tools like this from the very first message in this thread. Why yes, that was my first request, scripting tools for XML. Why has it taken you this long to provide a link (hmmm... actually, you didn't provide a link, but I suppose google will get me what I need)?

Because I have already mentioned so many alternatives! Of course google will get all this! I mentioned that there are very small resource-using C libraries for XML processing. Any one of these can be trivially included in TCL or other scripting languages. I have finally just picked one where someone has done the work for you.

Also, you were using your own selective definition of 'scripting language'. This even excluded PERL. It was hard to figure out what your requirements were.

Re:So MAKE it useful for system admins.

[argent]

Much of what you were specifying you wanted to do could be done by writing in XML itself (e.g. XSLT) with a single processing tool (such as xsltproc).

I don't know if it was you who said disparaging things about using printf to output XML, but I heartily agree. The same applies to hand-editing XML. XML is not designed to be edited by humans - it's even less suited to it than SGML was.

Because I have already mentioned so many alternatives!

You have posted a total of one message that mentioned a tool that was both native code and may be usable in a native fashion from scripts.

I have finally just picked one where someone has done the work for you.

You're the one who want to change the way people manage their configuration files. It's your job to do the work, not mine... I'm not convinced that the work is worthwhile even now.

Also, you were using your own selective definition of 'scripting language'.

No, I was applying additional restrictions on the kind of scripting language I want. And I didn't actually exclude Perl, just CPAN. I haven't tried putting together a stripped down Perl suitable for a boot floppy, so I won't rule it out, but CPAN is definitely too big. I haven't actually tried doing that with Tcl lately, it might have grown too much since the days when it was purely embedded and you could fit the interpreter in a "small model" program on Xenix-x86.

Re:So MAKE it useful for system admins.

[Decaff]

XML is not designed to be edited by humans

It certainly is. One of the main design goals was that it should be easily readable and editable by humans. One of the main points of good XML is that meaningful tags should be chosen so that an XML file can be understood without software.

You have posted a total of one message that mentioned a tool that was both native code and may be usable in a native fashion from scripts.

No. I mentioned MiniXML way back. That can be used in just the same way. It is easy to wrap TCL or other scripting languages around such libraries.

No, I was applying additional restrictions on the kind of scripting language I want.

Exactly. However, I may have misunderstood your post as excluding PERL.

Regarding TCL, I think it is still reasonably Small!

and you could fit the interpreter in a "small model" program on Xenix-x86.

That takes me back! I was a Xenix user in the 80s.

Re:So MAKE it useful for system admins.

[argent]

One of the main design goals was that it should be easily readable and editable by humans.

That was the goal of SGML, and while it didn't succeed as well as it could it did try. XML is what you get when you turn away from that goal in the name of making it negligably easier to parse.

I mentioned MiniXML way back. That can be used in just the same way.

Can be is not will be. Is it, or is this something else you think people who aren't convinced that this is even a good idea whould have to do?

Regarding TCL, I think it is still reasonably Small! /usr/local/lib/tcl/tcl8.3 is 1.7MB. True, that's no CPAN, and it's mostly character encoding information, but still... there's a lot of trimming to do to match awk.

Re:So MAKE it useful for system admins.

[Decaff]

That was the goal of SGML, and while it didn't succeed as well as it could it did try. XML is what you get when you turn away from that goal in the name of making it negligably easier to parse.

Not according to the initial work on XML of Jon Bosak in the mid 90s, and the text of the proposals to the W3C, and the practical experience of many of us who have used XML since then.

Can be is not will be. Is it, or is this something else you think people who aren't convinced that this is even a good idea whould have to do?

I seem to have difficulty figuring out the point you are trying to make. I'm not saying that anyone will use small libraries such as MiniXML or TinYXML to parse small config files in boot-type situations. I'm not even saying that anyone should use such libraries. What I'm saying is that is that XML could be used in that situation. I'm saying that this is a practical possibility. /usr/local/lib/tcl/tcl8.3 is 1.7MB. True, that's no CPAN, and it's mostly character encoding information, but still... there's a lot of trimming to do to match awk.

Of course it is - it is far more than awk - it is a general purpose and flexible language powerful enough for full applications.

If you are really wanting examples of really small boot situations, why even bother with a system big enough to run awk? There are embedded Java systems that run XML parsers to read configuration files that fit within a few 100k of memory!

a JVM + the Java BeanShell script interpreter (143k) combined with nanoXML (30k) are far smaller than a standard Unix kernel + shell + awk.

Re:So MAKE it useful for system admins.

[argent]

If you are really wanting examples of really small boot situations, why even bother with a system big enough to run awk?

Because the whole point of a scripting language is so when you're sitting in front of it trying to make it tick, you can write scripts.

Great Quotes from Computer History

[ebuck]

The best thing about using FORTRAN it that I don't need to read books about C.
The best thing about C is that I don't need to read books about C++.
The best thing about C++ is that I don't need to read books about JAVA.
The best thing about Lisp is that I don't need to read books about Carly Fiona.
The best thing about AWK is that I don't need to read books about Beowulf Clusters.
The best thing about bash is that I don't need to read books about hot grits.

Your statement is truthful, but lacks substance.

toolkit vs framework

[mparaz]

Ant: toolkit for building, declaratively (e.g. say "please compile/javac these stuff, and how" instead of specifying "javac -classpath ... -d ... File.java"

Maven: a framework or skeleton your build will follow, declaratively. "This is the source tree..."

Steven King and XML.

[Anonymous Coward]

You do realize that he was scared by an XML file growing up, and hasn't been the same since?

Good, up to a point

[jshowlett]

As long as you can comfortably fit into a declarative, purely dependency-oriented structure, Ant is great. But as soon as you have to cross the "scripting divide", which has happened with every nontrivial system I have dealt with, Ant becomes really clumsy. To me, the SCons approach of providing a build and dependency management framework that you can easily invoke from a general-purpose scripting language is much more elegant.

Key point: Ant is cross-platform

[thisisauniqueid]

Other replies have mentioned that you can build anything, do anything with Ant.

The thing I want to add is that Ant is cross-platform. Not just cross-platform like the gnu toolchain, but cross-platform in that e.g. a copy command is implemented using an XML element and system variables for paths... that way you can conceptually have exactly the same build process on a wide range of systems, within reason. (You don't need to know if the copy command is "cp" or "copy" on the target system.)

Prolog for Make

[argent]

No, really, I'm not kidding.

Defining your dependencies in Prolog or some similar language makes all kinds of sense.

Re:Prolog for Make

[tengwar]

Can you point us at some example code? For most of us our Prolog skills are ... a little rusty?

Re:Prolog for Make

[argent]

Here's a rather low-level one: Ciao Make.

Re:I know the moderators will tag me, but...

[jgrahn]

XML was designed specifically from the start to be parser-friendly. There are very simple APIs which handle any XML file such as SAX. /etc is a very good example of what is wrong with plain text files - no single program or API could ever handle the range of formats of files there!

And why on earth should there be one? You have to have domain knowledge to parse a file format and do something meaningful with it. Sure, if the file format is a specialization of XML, it helps you find some broken syntax on the XML/DTD level, but the penalty is this horrible tag soup which is never parseable with standard tools like grep, cat, awk etc.

I cannot see why XML people find it worth all that trouble. One language for one purpose is fine with me. This applies to make too -- the language fits the purpose of the tool well, and the TAB issue has never gotten in my way during the maybe twelve years I've used it.

Re:I know the moderators will tag me, but...

[Decaff]

And why on earth should there be one? You have to have domain knowledge to parse a file format and do something meaningful with it.

No. You don't have to have have domain knowledge to do this. For example, with XML, you can do things like 'extract the contents of the second element' with any XML file.

I cannot see why XML people find it worth all that trouble. One language for one purpose is fine with me. This applies to make too -- the language fits the purpose of the tool well, and the TAB issue has never gotten in my way during the maybe twelve years I've used it.

How about the reverse view? Why invent a new format for each purpose, requiring specific domain knowledge even to begin to make sense of the file contents?

penalty is this horrible tag soup which is never parseable with standard tools like grep, cat, awk etc.

Having a range of files with tab, or comma, or colon, or other arbitrary delimiters isn't horrible? This is a kind of delimiter-soup which can't be parsed with standard tools unless you already have detailed knowledge of what the delimiter for a file is.

Ant is very Java-centric and has other problems

[Shirotae]

I have been using Ant for a while now and my experience is that although it is possible to do things other than build Java, I am fighting against a lot of strongly built in assumptions to get it to do anything else well.

Given that Ant uses XML and has an XSLT task, you might think that it would be good for doing XSLT processing, but if you need to use a more advanced XSLT processor (e.g. Saxon) you immediately run into problems. It would not be a big issue if I was working on my own - I could just dump all the extra jars into my Ant lib directory, but that is an administrative nightmare if you are one of a team. You can't even give a colleague a copy of Ant with the extra jars in place - if they have Ant installed and ANT_HOME set, it takes priority and chaos follows.

The review also refers to "excellent online documentation" but I have to disagree with that assessment. After using the online documentation most days for several months I have learned how to find what information there is on the things that are not used much in the endlessly repeated trivial examples, and that what is there can be rather misleading.

Having just looked at what you can get at the O'Reilley Safari site without subscribing, I am not going to get this book. The chapter outlines and summaries cover the same old stuff, heavily Java focused, and not straying far from the conventional view of what people might want to do.

Ant is better than nothing, for Java development it is better than make, and for things I am doing on my own I can bend it to be useful beyond those narrow limits. It falls far short of the hype put about by its advocates.

What he world needs is . . .

[wsanders]

a good "do what I mean" XML to XSD translator.

"...verify the XML configuration files by verifying the XML files against their respective DTDs or XSDs"

I really don't have anything against ant and have a small amount of experience with it. The prejudice against XML might even come from not actually having access to the XML DTD or schema itself in a lot of cases. One fears what one doesn't understand. For example, I have to mass-administer SunONE web server, which stores its configration parameters in a dog's breakfast of several XML and flat files. I can randomly hack the files with perl or sed, and then test each hack to see what it breaks, or if I knew the DTD I could actually make some intelligent decisions about what to do.

Sysadmins should learn more about this stuff, because a lot of programmers are trying to do sysadminish stuff in XML, like the Bourne shell commands embedded in XML, mentioned in a parent. Ick.

Re:I know the moderators will tag me, but...

[Procyon101]

The format is full of syntactic noise. A much simpler format could have been devised if A) They weren't so interested in maintaining a "text" format regardless of data type and B) They had described the data more completely in the tag, including data size to reduce the need for a closing tag. This would also have eliminated the need to escape data. The format would have then been more terse, less redundant, and more easily editable and parseable. The requirement to stick to a text style formats was unneccissary as since the parsers were being supplied, the editors would have been trival to write.

Add to this redundant constructs (elements and attributes are semantically identical, but syntactically different), the need for external validation due to the fact that conforming editors were not supplied, and the need to walk the entire file to simply read an arbitrary node because although all data to offset directly to an index is calculatable at construction it is not supplied, archaic document definition standards, no thought put into stylesheet application for rendering, and the idea that it would make a good language (template OR procedural) in addition to data storage, and you have a poorly thought out and implemented standard.

It was adopted anyway because standardization trumps all of the above... but that doesn't mean it's good.