I hope the users of your scripts know to put a./ in front of script args, too...
I'm not sure what you're getting at. If they're providing an argument that I'm going to interpret as a glob pattern, then I'm responsible for making it work right. If they're globbing it, the fact that my program is a script is irrelevant.
And the whole sh-fetish (speaking of which, that's '${1+"$@"}' if you want to be portable:->) issue is irrelevant if you use an eval-safe scripting language like awk rather than Tcl or sh (though Tcl definitely gives you the best tools to deal with eval of any string-based language I know... and that's no accident). Awk doesn't eval strings, if you quote something it stays quoted: talking about quoting problems in awk is seriously missing the point, and the fact that Larry Wall brought them back in Perl (which is basically awk on steroids and Guinness) is one reason I don't want to use Perl any more than I have to.
It's a lot more complex, building an eval-safe language, than you think. Tcl-without-eval (or more properly, Tcl-without-implicit-eval... and I've had thoughts on this area myself) wouldn't look like Tcl, it'd be more like Smalltalk or Lisp. Taking out eval means taking out subst and uplevel and a whole lot of other stuff as well, so your "{$a $b $c}" mistake wouldn't happen... you wouldn't be able to write that. The XSL example someone posted that used [...] to interpolate queries into strings? That's eval, too. The XSLT examples, with ? More eval. Javascript embedded in XML? More eval.
There's other eval-safe scripting languages. Javascript (the language itself), Icon, I've already named a few. I chose awk because it's one that's already being used, but I really do like javascript a lot... as a language in its own right. It's a pity that it hasn't been given a chance to really sing.
How do you use ANYTHING in a script? With a program that understands it.
Good boy. Pretty soon you'll catch on that I have been asking for nothing else but an example of a scripting language that understands that (or something comparable to it) all day.
The rest of us will use the appropriate tool for the job. XPath for XML, SQL for relational databases.
That is a complete description of the traditional password file. More recent versions have added more fields, like turning gcos into a two to four fields or adding password expiry information... but it's always still a relation. In no system, anywhere, is this a hierarchical structure.
There are files in/etc that have arbitrary key-value pair associated with object, or an inherently nested structure, but this is not one of them:/etc/passwd is a relation. The only reason XML would be an appropriate tool for the job is if you were already using it for everything else.
You are focussing on what tools you are using to process/etc files
No shit, sherlock. That is the topic.
I have sort of lost track of what we are debating!
I don't know what you're debating. I'm asking what the advantage of going to XML for the files in/etc would be. So far it looks like the answer is one of: (a) larger and harder to read and manage bootscripts, (b) having every script load a large XML handling library before doing any work, or (c) redesigning the boot process around a Java VM, to avoid having to reload the VM or *ML handling libraries every time.
XML has areas where it's a massive improvement over traditional practice. This isn't one of them.
However, for better or worse, the tools are written in XML (or, rather, XSLT). There is no need to write a variety of tools when there is a single language that allows all this processing and transforming to be done. The tools to do this are the programs such as xsltproc.
OK, show me the tools.
[XML I don't want to re-quote to feed back to/.]
That's about half again as much code to just do the extraction. It's not significantly shorter than my SQL example. It's significantly harder to read than the original awk version, and that used fixed constants. And you're not actually doing any processing on it, you're just getting the home directory. Awk contains a compact C-styled scripting language that's eval-safe and turing complete. What's the scripting language in XSL?
[pause for googlage]
Let's see... on the one hand I use awk and write "foo(arg1, arg2);" and on the other hand I write something like...
XSL already exists, and is much better suited to XML's heirachical structure than trying to force it into a relational model so you can use SQL on it./etc/passwd is a relation. Whether it's stored in XML, in a UNIX-style flat file, or a table in a relational database, it's still a relation. If treating an XML file as a relation is "forcing it into a relational model" then parhaps XML is not the best tool to use.
show me a general awk script that will retrieve, say, the value of the second parameter in any file under/etc.
I'm not sure I understand how that's meaningful.
What you're saying is "with XML, you don't need to write a different two-line parser for each file".
What I'm saying is, "with XML, you still have to do something useful with the data". And "doing something useful" involves enough work that the potential savings from the file format are negligable. And "doing something useful" is actually harder with XML because the tools that do said useful stuff don't seem to exist for XML.
What I'm also saying is that there are things that XML is very good at. I would certainly use XML over the dots and backslashes of nroff, for example. The way Apache uses XML to provide metadata markup in its configuration files is great.
Why should I be limited to 30-year-old scripting languages?
I haven't asked you to be "limited to 30 year old scripting languages". I apologise for misleading you, or giving you a convenient hook to use to dismiss my comment, whichever applies.
I gave you a couple of examples of scripting languages, and asked you to show me something like that that allows me to script operations using XML as easily as I can script operations using traditional UNIX table-like files. It doesn't need to be awk, it could be Icon, Tcl, ECMAscript, SQL, or even standalone replacements for grep/sed/whatever that could be integrated into shell scripts.
Just show me something that does as good a job as what I have. I'm not even asking for something better, just a tool that makes integrating XML into/etc a viable option. Surely that's not so hard?
When's the last time you wrote a shell script that fully dashed its actual intended arguments to programs?
Never. I write scripts that actually work on real systems. The solution to the "file beginning with -" problem is "start every pattern with./". Also, "fully quote all variables" to take care of the "file with a space or shell metacharacters in its name" problem.
This problem has nothing to do with XML, by the way. It's a language issue, and it's shared by every language that includes an "eval" operation, and it's never found in any other languages. The problem is (a) recognising when you have an eval and (b) knowing how to deal with it. Writing your Makefiles in XML doesn't change the fact that eventually you're going to pass the script you've generated to an eval to call an external program.
Don't get me started on SQL
SQL has a very simple set of quoting rules. It's one of the least painful script interfaces I've ever had to deal with.
Oh, you're just blatting out XML with printf? You deserve to lose, then.
Which brings us back to the evil that is ant. It's XML that in the ordinary course of using it is edited by humans in ordinary text editors.
That's a horribly inappropriate use of XML. Pure XML is not the worst format to write code in that I've ever used... after all, Make has its own significant problems, but it's close to the worst one I've run into that wasn't deliberately designed to be hard to understand.
Just stick to the simple XML guidelines and any XML parser can parse them.
I showed you an example of parsing/etc/passwd in two lines of awk. Your DTD is going to be longer than that.
All this is already there. There is a standard XML dialect called XSL...
Show me. Show me something that's already hooked into a conventional scripting language (sh, awk, etc) that can be used without major dependencies (so that it can actually be used during boot) and that doesn't involve writing XML to parse XML.
Solutions involving Perl or CPAN, or Java, or that are only usable with a webserver installed, will not be considered responsive. When you're sitting at a single-user prompt and the only software available is what fits on your emergency boot floppy solutions like those tend to elicit violent reactions.
But many programs are much more dynamic than UNIX's/etc/passwd file. For files that are likely to be needed in a new format at some time, XML coupled with XSL provides a flexible way to move you data programatically into the new format.
I don't think you're disagreeing with unless you're dealing with files that are very much more complex than the ones in/etc, it's harder to write a description for them so that you can parse them using a general purpose XML library than it is to parse them using the traditional UNIX tools. For files where parsing the content is inherently difficult (document markup) or where you need self-documenting content (general databases) XML has a definite place... but the grandparent article was specifically talking about/etc/passwd and its kin.
As for denmark:havarti:3.99:104... if you have humans editing your files directly then XML produces different failure modes, but my recent experience with ant and Tomcat is that it can be a LOT harder to track a problem down. If the files are edited by programs, you're better off in either case... and it's really more critical in XML than in other kinds of files.
And in practice, the price of cheese in denmark would be maintained by someone filling a field somewhere that lead to a statement like "UPDATE inventory SET price='3.99' WHERE country='denmark' AND cheese='havarti';". XML lives in the demimonde between scripts and databases, it's useful for sparse markup in documents, it's useful for data interchange, but in Java it somehow has become the format of choice even where it's wildly inappropriate and where its actual strengths are not even used.
it assumes that there are no fields that contain a : character.
That's because there are no fields that contain a ":" character. If there could be fields containing a ":" character, then ":" wouldn't have been chosen as a separator. If you put a ":" character in there, then you've created an invalid password file. The standard utilities won't create a file like that, vipw will refuse to write it, and so on.
If you were dealing with CSV files, you'd have the same problem with a malformed CSV file.
Same with XML.
The best you could hope for would be a better error message, but you couldn't even depend on that. Even with full-on relational databases and SQL queries, you still get applications coughing and dying over invalid tables, and you still have to go in with forensic tools and look for the corruption.
no single program or API could ever handle the range of formats of files [in/etc]!
And converting them to XML won't change that, because unless you're dealing with files that are very much more complex than the ones in/etc, it's harder to write a description for them so that you can parse them using a general purpose XML library than it is to parse them using the traditional UNIX tools... and having parsed them you haven't actually started to do anything useful with them.
#!/bin/awk
BEGIN { FS=":"; }
$3>1000 { print $6; }
If you want people to treat XML seriously, give them the tools to do something useful with them. Before you produce another "ant", give us xmlgrep, xmlsort, xmlawk, xmlsql.
This would actually be useful, but it's more complex than the "harder to parse" version. So what do you do? Well, you could create a whole new set of files to manage, and do this:
#!/bin/xmlsql
\i/libdata/passwd.sql
select home from passwd where uid >= 1000;
Or maybe:
#!/bin/xmlawk #include "/libdata/passwd.awk"
$uid >= 100 { print $home }
OK, after a while, you end up with a bunch of files in/etc that are no harder to deal with and a bit better documented, but it'll take a fair amount of work and noobody's really going to take you seriously until you do it. But... there's the other problem. Once you do that, where does ant come in? What useful information can you extract from an ant file? It doesn't even contain a description of the dependencies, which might be useful to know! It's just a bunch of code fragments and calls to build routines only partially described.
Somewhere in the Java code base are the seeds of the first computer virus to ever cross from the silicon to the meat brain. It's the only conclusion I can come up with to explain all the terrible design in just about every significant Java project I've seen so far. Ant is merely the tip of the iceberg.
I may upgrade to Tiger on my own desktop, a friend of mine keeps telling me how cool Spotlight is, but so far Panther has handled everything I've thrown at at, and I've only upgraded to Panther to run some Apple software that requires it... everything else runs on Jaguar.
You can buy Panther from resellers for $50 now. So if you (as I did) installed Jaguar in early 2003, your upgrade cost would be $50... which as it happens is about the same as the antivirus software I have to buy regularly to keep up my A/V updates.
OS X Jaguar... $130 (Retail) OS X Panther upgrade... $50 (macsales.com) Total... $180
Actually, I've found cheaper copies of Panther since.
Anyway the time I need to upgrade to Tiger, it'll be $50. Of course, by then I'll probably have had to upgrade to XP (doesn't that look like a "poison" emoticon?).
but there is just too much useful software that works in GNU/Linux w/o a compatibility layer
I don't understand what you're talking about here, you don't need to run UNIX apps in any Linux compatibility mode. They run natively on OS X, because it is UNIX under the hood. I've never had to load up a "compatibility layer" to run UNIX software on my Mac.
All those are a couple of keystrokes away after you hit control-escape or control-alt-delete. It seems a waste... the Command/Apple key on the Mac is actually used to provide functionality throughout the system in every application and lets me do things like keyboard cut-and-paste in CLI windows.
It's a shame, since Windows otherwise has so much better keyboardability than the Mac OS... but everything Microsoft's done to Windows since 3.11 has made it less effective from the keyboard.
I'm not talking about updates that happen without your knowledge. I'm talking about notification of updated packages that YOU choose to install.
Oh, like the Rad Hat spam-me-harder service? Boy, was that a waste of money.
and keep a ready inventory of all the installed programs (including those installed by dependancy)
If you don't use a package mangler, you don't have any programs installed by a dependency unless you do it yourself.
i've got better things to do with my time than spend it constantly checking for seucrity issues.
That's why I limit the number of packages I install. And why I use an operating system that actually HAS a core operating system, rather than being a bunch of packages flying in close formation... so when I set up a typical internet-facing server it's only got... let's see... less than 20 packages total, and half of them are things like "libtool" that are never executed except during the installation of other packages.
Slashdot Mirror
I hope the users of your scripts know to put a ./ in front of script args, too...
:->) issue is irrelevant if you use an eval-safe scripting language like awk rather than Tcl or sh (though Tcl definitely gives you the best tools to deal with eval of any string-based language I know... and that's no accident). Awk doesn't eval strings, if you quote something it stays quoted: talking about quoting problems in awk is seriously missing the point, and the fact that Larry Wall brought them back in Perl (which is basically awk on steroids and Guinness) is one reason I don't want to use Perl any more than I have to.
I'm not sure what you're getting at. If they're providing an argument that I'm going to interpret as a glob pattern, then I'm responsible for making it work right. If they're globbing it, the fact that my program is a script is irrelevant.
And the whole sh-fetish (speaking of which, that's '${1+"$@"}' if you want to be portable
It's a lot more complex, building an eval-safe language, than you think. Tcl-without-eval (or more properly, Tcl-without-implicit-eval... and I've had thoughts on this area myself) wouldn't look like Tcl, it'd be more like Smalltalk or Lisp. Taking out eval means taking out subst and uplevel and a whole lot of other stuff as well, so your "{$a $b $c}" mistake wouldn't happen... you wouldn't be able to write that. The XSL example someone posted that used [...] to interpolate queries into strings? That's eval, too. The XSLT examples, with ? More eval. Javascript embedded in XML? More eval.
There's other eval-safe scripting languages. Javascript (the language itself), Icon, I've already named a few. I chose awk because it's one that's already being used, but I really do like javascript a lot... as a language in its own right. It's a pity that it hasn't been given a chance to really sing.
Good boy. Pretty soon you'll catch on that I have been asking for nothing else but an example of a scripting language that understands that (or something comparable to it) all day.
The rest of us will use the appropriate tool for the job. XPath for XML, SQL for relational databases.
That is a complete description of the traditional password file. More recent versions have added more fields, like turning gcos into a two to four fields or adding password expiry information... but it's always still a relation. In no system, anywhere, is this a hierarchical structure.
There are files in
You are focussing on what tools you are using to process /etc files
/etc would be. So far it looks like the answer is one of: (a) larger and harder to read and manage bootscripts, (b) having every script load a large XML handling library before doing any work, or (c) redesigning the boot process around a Java VM, to avoid having to reload the VM or *ML handling libraries every time.
No shit, sherlock. That is the topic.
I have sort of lost track of what we are debating!
I don't know what you're debating. I'm asking what the advantage of going to XML for the files in
XML has areas where it's a massive improvement over traditional practice. This isn't one of them.
OK, show me the tools.
[XML I don't want to re-quote to feed back to
That's about half again as much code to just do the extraction. It's not significantly shorter than my SQL example. It's significantly harder to read than the original awk version, and that used fixed constants. And you're not actually doing any processing on it, you're just getting the home directory. Awk contains a compact C-styled scripting language that's eval-safe and turing complete. What's the scripting language in XSL?
[pause for googlage]
Let's see... on the one hand I use awk and write "foo(arg1, arg2);" and on the other hand I write something like...
No thanks. A real scripting language is not an optional feature.
/etc/passwd[@uid >= 1000]/@home
/etc/passwd is a relation. Whether it's stored in XML, in a UNIX-style flat file, or a table in a relational database, it's still a relation. If treating an XML file as a relation is "forcing it into a relational model" then parhaps XML is not the best tool to use.
How do I use that in a script?
XSL already exists, and is much better suited to XML's heirachical structure than trying to force it into a relational model so you can use SQL on it.
Sounds like crossing smileys and x-face.
show me a general awk script that will retrieve, say, the value of the second parameter in any file under /etc.
/etc a viable option. Surely that's not so hard?
I'm not sure I understand how that's meaningful.
What you're saying is "with XML, you don't need to write a different two-line parser for each file".
What I'm saying is, "with XML, you still have to do something useful with the data". And "doing something useful" involves enough work that the potential savings from the file format are negligable. And "doing something useful" is actually harder with XML because the tools that do said useful stuff don't seem to exist for XML.
What I'm also saying is that there are things that XML is very good at. I would certainly use XML over the dots and backslashes of nroff, for example. The way Apache uses XML to provide metadata markup in its configuration files is great.
Why should I be limited to 30-year-old scripting languages?
I haven't asked you to be "limited to 30 year old scripting languages". I apologise for misleading you, or giving you a convenient hook to use to dismiss my comment, whichever applies.
I gave you a couple of examples of scripting languages, and asked you to show me something like that that allows me to script operations using XML as easily as I can script operations using traditional UNIX table-like files. It doesn't need to be awk, it could be Icon, Tcl, ECMAscript, SQL, or even standalone replacements for grep/sed/whatever that could be integrated into shell scripts.
Just show me something that does as good a job as what I have. I'm not even asking for something better, just a tool that makes integrating XML into
No, really, I'm not kidding.
Defining your dependencies in Prolog or some similar language makes all kinds of sense.
When's the last time you wrote a shell script that fully dashed its actual intended arguments to programs?
./". Also, "fully quote all variables" to take care of the "file with a space or shell metacharacters in its name" problem.
Never. I write scripts that actually work on real systems. The solution to the "file beginning with -" problem is "start every pattern with
This problem has nothing to do with XML, by the way. It's a language issue, and it's shared by every language that includes an "eval" operation, and it's never found in any other languages. The problem is (a) recognising when you have an eval and (b) knowing how to deal with it. Writing your Makefiles in XML doesn't change the fact that eventually you're going to pass the script you've generated to an eval to call an external program.
Don't get me started on SQL
SQL has a very simple set of quoting rules. It's one of the least painful script interfaces I've ever had to deal with.
Oh, you're just blatting out XML with printf? You deserve to lose, then.
Which brings us back to the evil that is ant. It's XML that in the ordinary course of using it is edited by humans in ordinary text editors.
That's a horribly inappropriate use of XML. Pure XML is not the worst format to write code in that I've ever used... after all, Make has its own significant problems, but it's close to the worst one I've run into that wasn't deliberately designed to be hard to understand.
Just stick to the simple XML guidelines and any XML parser can parse them.
/etc/passwd in two lines of awk. Your DTD is going to be longer than that.
I showed you an example of parsing
All this is already there. There is a standard XML dialect called XSL...
Show me. Show me something that's already hooked into a conventional scripting language (sh, awk, etc) that can be used without major dependencies (so that it can actually be used during boot) and that doesn't involve writing XML to parse XML.
Solutions involving Perl or CPAN, or Java, or that are only usable with a webserver installed, will not be considered responsive. When you're sitting at a single-user prompt and the only software available is what fits on your emergency boot floppy solutions like those tend to elicit violent reactions.
But many programs are much more dynamic than UNIX's /etc/passwd file. For files that are likely to be needed in a new format at some time, XML coupled with XSL provides a flexible way to move you data programatically into the new format.
/etc, it's harder to write a description for them so that you can parse them using a general purpose XML library than it is to parse them using the traditional UNIX tools. For files where parsing the content is inherently difficult (document markup) or where you need self-documenting content (general databases) XML has a definite place... but the grandparent article was specifically talking about /etc/passwd and its kin.
I don't think you're disagreeing with unless you're dealing with files that are very much more complex than the ones in
As for denmark:havarti:3.99:104... if you have humans editing your files directly then XML produces different failure modes, but my recent experience with ant and Tomcat is that it can be a LOT harder to track a problem down. If the files are edited by programs, you're better off in either case... and it's really more critical in XML than in other kinds of files.
And in practice, the price of cheese in denmark would be maintained by someone filling a field somewhere that lead to a statement like "UPDATE inventory SET price='3.99' WHERE country='denmark' AND cheese='havarti';". XML lives in the demimonde between scripts and databases, it's useful for sparse markup in documents, it's useful for data interchange, but in Java it somehow has become the format of choice even where it's wildly inappropriate and where its actual strengths are not even used.
it assumes that there are no fields that contain a : character.
That's because there are no fields that contain a ":" character. If there could be fields containing a ":" character, then ":" wouldn't have been chosen as a separator. If you put a ":" character in there, then you've created an invalid password file. The standard utilities won't create a file like that, vipw will refuse to write it, and so on.
If you were dealing with CSV files, you'd have the same problem with a malformed CSV file.
Same with XML.
The best you could hope for would be a better error message, but you couldn't even depend on that. Even with full-on relational databases and SQL queries, you still get applications coughing and dying over invalid tables, and you still have to go in with forensic tools and look for the corruption.
God knows I've done enough of THAT.
And converting them to XML won't change that, because unless you're dealing with files that are very much more complex than the ones in
Any suggestions?
Prolog.
No, I'm not kidding. Solving complex dependency trees is what Prolog's *for*.
Somewhere in the Java code base are the seeds of the first computer virus to ever cross from the silicon to the meat brain. It's the only conclusion I can come up with to explain all the terrible design in just about every significant Java project I've seen so far. Ant is merely the tip of the iceberg.
most of our internal tools require IE6 or Windows only patch-distribution systems.
It's an Own Goal! Scooooooooooore!
What could possibly be the benefit of OS X as a desktop computer?
Lower support costs. Higher productivity. Less downtime.
I don't get another $129 hit each successive year for the latest, greatest, faster, more complete OSX.
Stick with Jaguar.
It's still better than XP.
It was recently revealed that OSX for PPC is compiled with the optimize-for-size option.
And the problem with this is what? You'd rather they unroll loops and blow out the cache more often?
I may upgrade to Tiger on my own desktop, a friend of mine keeps telling me how cool Spotlight is, but so far Panther has handled everything I've thrown at at, and I've only upgraded to Panther to run some Apple software that requires it... everything else runs on Jaguar.
... $130 (Retail) ... $50 (macsales.com) ... $180
... $180 (Pricegrabber.com) ... $50 (Retail) ... $20 (Retail) ... $20 (Retail) ... $270
You can buy Panther from resellers for $50 now. So if you (as I did) installed Jaguar in early 2003, your upgrade cost would be $50... which as it happens is about the same as the antivirus software I have to buy regularly to keep up my A/V updates.
OS X Jaguar
OS X Panther upgrade
Total
Windows 2000 Pro
Norton Antivirus
LiveUpdate
LiveUpdate
Total
Actually, I've found cheaper copies of Panther since.
Anyway the time I need to upgrade to Tiger, it'll be $50. Of course, by then I'll probably have had to upgrade to XP (doesn't that look like a "poison" emoticon?).
It takes a non-trivial amount of futzing to just get the god damn file and printer sharing working.
Huh? Macs support Windows file and print servers, you don't need to set up AFS or LPD servers if you don't want to.
but there is just too much useful software that works in GNU/Linux w/o a compatibility layer
I don't understand what you're talking about here, you don't need to run UNIX apps in any Linux compatibility mode. They run natively on OS X, because it is UNIX under the hood. I've never had to load up a "compatibility layer" to run UNIX software on my Mac.
Veterans Health Information System and Technology Architecture
This is why Open Source is Evil, right?
Quick, someone come up with a retronym for "LONGHORN".
All those are a couple of keystrokes away after you hit control-escape or control-alt-delete. It seems a waste... the Command/Apple key on the Mac is actually used to provide functionality throughout the system in every application and lets me do things like keyboard cut-and-paste in CLI windows.
It's a shame, since Windows otherwise has so much better keyboardability than the Mac OS... but everything Microsoft's done to Windows since 3.11 has made it less effective from the keyboard.
I'm not talking about updates that happen without your knowledge. I'm talking about notification of updated packages that YOU choose to install.
... let's see... less than 20 packages total, and half of them are things like "libtool" that are never executed except during the installation of other packages.
Oh, like the Rad Hat spam-me-harder service? Boy, was that a waste of money.
and keep a ready inventory of all the installed programs (including those installed by dependancy)
If you don't use a package mangler, you don't have any programs installed by a dependency unless you do it yourself.
i've got better things to do with my time than spend it constantly checking for seucrity issues.
That's why I limit the number of packages I install. And why I use an operating system that actually HAS a core operating system, rather than being a bunch of packages flying in close formation... so when I set up a typical internet-facing server it's only got