Extensible Programming for the 21st Century

Anonymous Cowardly Lion writes "An interesting article written by a professor at the University of Toronto argues that next-generation programming systems will combine compilers, linkers, debuggers, and that other tools will be plugin frameworks [mirror], rather than monolithic applications. Programmers will be able to extend the syntax of programming languages, and programs will be stored as XML documents so that programmers can represent and process data and meta-data uniformly. It's a very insightful and thought-provoking read. Is this going to be the next generation of extensible programming?"

Comments

Go Greg!

[xcham]

The document is mirrored here to help compensate for the bandwidth deluge.

been done

[studboy]

programs will be stored ... so that programmers can represent and process data and meta-data uniformly.

Yup. Back in the day, we called this "Lisp". It was about as readable as XML, but a hella lot more fun.

Re:been done

[Anonymous Coward]

The S-expression people put the attributes in as arguments; the element data is also just another argument, which is certainly a more uniform treatment.

Ex:

<a b="1" c="2" d="4">blahblahblah</a>

becomes something like:

(a "blahblahblah" (b 1 c 2 d 4))

or this:

(a "blahblahblah" b 1 c 2 d 4)

or this:

(a (b 1) (c 2) (d 4) "blahblahblah")

Take your pick.

The point stands that you don't need XML's syntax to get these sort of behaviors, and that's not really surprising; any sufficiently general syntax usually can accomodate any paradigm in another sufficiently general syntax. This probably has some metaphysical voodoo link with the Church-Turing thesis, formal languages, and computability, but I'm too lazy to care.

You could dispense with the element names if you didn't care about such things, although it makes it harder to reorder your arguments, of course. Then again, XML can't support this sort of ordered attribute list, now can it? Not directly, anyway. ;)

I'm not a Lisp or XML zealot, and I agree with you that if it's machine-generated, it doesn't really matter, but the Lisp people have a point that anything you claim to be able to do now with XML could have been done before with Lisp. Heck, DSSSL is based on Scheme.

* Note that I didn't quote the arguments; I'm assuming this is being fed to the read procedure, and not the evaluator.

Re:been done

[boots@work]

This has been proposed and implemented several times as an alternative syntax. I think Arc is meant to have this, but it's hardly the first. I don't think it's ever really caught on.

Why?

- Sometimes it's nice to put code into things that don't reliable preserve whitespace, such as, say, comment fields on web sites.

- Parens are well-established in lisp. If you change it you give an additional barrier to people coming from other lisp dialects, without particularly helping people coming from elsewhere.

- Whitespace by itself is not enough. Do you want to write (+ 3 (* 4 5)) across 3-5 lines? Python ends up with fairly complex rules about backslashes, open parens, etc.

- One advantage of lisp is that it's easy to write out from a program. This is really not true of Python.

- If you accept that we need paren syntax, then you can wonder whether indentation should be supported as an alternative. But having two different syntaxes for one language, though an interesting idea, is likely to cause a lot of practical confusion.

So I think all you really want is an editor that ensures the indentation is always valid, and that can highlight parens and do other things. emacs goes a long way, but it could be better -- for example by making outer parens larger, as in TeX-printed or handwritten mathematics.

In my humble opinion what Lisp needs to take from Python is not semantic indentation, but rather a single standard dialect with good OS bindings. The last thing we need is yet another slightly incompatible dialect that can't bind to existing code. Sheesh; I love lisp but lisp implementers really exasperate me.

dealing with XML

[Yobgod Ababua]

That's exactly one of the author's points! You shouldn't (and in his vision won't) have to deal with the XML directly, -unless- you are one of the people actually writing new plugins rather than just using them.

His suggestion is primarily that we start using editors that transparently present the 'code file' in our choice of format rather than forcing us to edit it byte-by-byte. It's like the syntax-highlighting you probably use now, only effecting more than just colors.

Using XML for the underlying syntax is mostly irrelevant to his proposal, but he suggests it merely because it is currently popular, well suppoerted, and well suited to it's primary job of presenting data in an easily MACHINE READABLE format.

His proposal is, in fact, exactly the opposite of requiring coders to pop open a hex editor, and he likens our current ASCII-only coding methods to doing exactly that at one point.

Compiler extension (was:Can't wait)

[real gumby]

It would be great, if instead, I could hook into the compiler and tell it exactly how it should handle vectors.

Well of course that's what templates are. Yes, their syntax is horrendous but that's what comes of trying to wedge the concept into the existing crannies of C syntax (or when, as Stroustrup remarked to me once, "the ecological niche was already polluted").

If you hanker for a language in which metasyntactic extension is natural, you need Lisp macros (or here and here for a more complex example), Scheme "hygenic" macros or the CLOS MOP.

But if you really want to consider "hooking into the compiler" as you say then you should look at the reflective programming work, the ground work for which was laid down almost 25 years ago by Brian Cantwell Smith and was even implemented, by me and others, back then. Although a lot of work continued in this area that vein pretty much got mined: unless you can think up a completely new control structure there's not a huge amount more you can do with such a system than you could with a normal metasyntactic extension mechanism.

HTH
-d