Extensible Programming for the 21st Century

Anonymous Cowardly Lion writes "An interesting article written by a professor at the University of Toronto argues that next-generation programming systems will combine compilers, linkers, debuggers, and that other tools will be plugin frameworks [mirror], rather than monolithic applications. Programmers will be able to extend the syntax of programming languages, and programs will be stored as XML documents so that programmers can represent and process data and meta-data uniformly. It's a very insightful and thought-provoking read. Is this going to be the next generation of extensible programming?"

Comments

Go Greg!

[xcham]

The document is mirrored here to help compensate for the bandwidth deluge.

been done

[studboy]

programs will be stored ... so that programmers can represent and process data and meta-data uniformly.

Yup. Back in the day, we called this "Lisp". It was about as readable as XML, but a hella lot more fun.

Re:been done

[Anonymous Coward]

The S-expression people put the attributes in as arguments; the element data is also just another argument, which is certainly a more uniform treatment.

Ex:

<a b="1" c="2" d="4">blahblahblah</a>

becomes something like:

(a "blahblahblah" (b 1 c 2 d 4))

or this:

(a "blahblahblah" b 1 c 2 d 4)

or this:

(a (b 1) (c 2) (d 4) "blahblahblah")

Take your pick.

The point stands that you don't need XML's syntax to get these sort of behaviors, and that's not really surprising; any sufficiently general syntax usually can accomodate any paradigm in another sufficiently general syntax. This probably has some metaphysical voodoo link with the Church-Turing thesis, formal languages, and computability, but I'm too lazy to care.

You could dispense with the element names if you didn't care about such things, although it makes it harder to reorder your arguments, of course. Then again, XML can't support this sort of ordered attribute list, now can it? Not directly, anyway. ;)

I'm not a Lisp or XML zealot, and I agree with you that if it's machine-generated, it doesn't really matter, but the Lisp people have a point that anything you claim to be able to do now with XML could have been done before with Lisp. Heck, DSSSL is based on Scheme.

* Note that I didn't quote the arguments; I'm assuming this is being fed to the read procedure, and not the evaluator.

Re:been done

[boots@work]

This has been proposed and implemented several times as an alternative syntax. I think Arc is meant to have this, but it's hardly the first. I don't think it's ever really caught on.

Why?

- Sometimes it's nice to put code into things that don't reliable preserve whitespace, such as, say, comment fields on web sites.

- Parens are well-established in lisp. If you change it you give an additional barrier to people coming from other lisp dialects, without particularly helping people coming from elsewhere.

- Whitespace by itself is not enough. Do you want to write (+ 3 (* 4 5)) across 3-5 lines? Python ends up with fairly complex rules about backslashes, open parens, etc.

- One advantage of lisp is that it's easy to write out from a program. This is really not true of Python.

- If you accept that we need paren syntax, then you can wonder whether indentation should be supported as an alternative. But having two different syntaxes for one language, though an interesting idea, is likely to cause a lot of practical confusion.

So I think all you really want is an editor that ensures the indentation is always valid, and that can highlight parens and do other things. emacs goes a long way, but it could be better -- for example by making outer parens larger, as in TeX-printed or handwritten mathematics.

In my humble opinion what Lisp needs to take from Python is not semantic indentation, but rather a single standard dialect with good OS bindings. The last thing we need is yet another slightly incompatible dialect that can't bind to existing code. Sheesh; I love lisp but lisp implementers really exasperate me.

Ah.. So the professor likes Eclipse

[the_skywise]

And suddenly he's propheysing the future?

Editors like Emacs, Visual SlickEdit and even the loved/loathed MS Visual Studio have plug-in frameworks.

As for XML being the "glue" for holding things together... No. It'll be a data neutral "modulator" you emit your data from your program by name in a particular format. Transmitting and receipt by the other programs will be handled by a remodulator. In between it might be XML, it might be binary, it might be whatever you feel like using that day.
(and no I haven't read the artile (FORBIDDEN)

Re:Ah.. So the professor likes Eclipse

[Anonymous Coward]

Or you can check out IDEA which does all that eclipse does, and more, and faster (MUCH faster), and with less bugs.

Check out their PSI (program structure interfaces) for plugins.

Re:Can't wait

[TwistedSquare]

Actually I believe this particular example might well be optimised out by the compiler to the code you mention. But your point is probably valid for trickier problems.

Summary Of The Summary

[Pan T. Hose]

An interesting article written by a professor at the University of Toronto argues that next-generation programming systems will combine compilers, linkers, debuggers, and that other tools will be plugin frameworks, rather than monolithic applications.

This is not the next generation of programming systems but rather the present one for pretty much everyone except for those using Microsoft tools.

Programmers will be able to extend the syntax of programming languages,

Again, nothing new.

and programs will be stored as XML documents so that programmers can represent and process data and meta-data uniformly.

There is no way in hell that would ever happen. Ever.

It's a very insightful and thought-provoking read. Is this going to be the next generation of extensible programming?

No.

Now, I will read the entire article, but somehow, I am not holding my breath...

Re:Next generation for ME

[Greger47]

So whats wrong with CORBA? Here's one among several implementations for PHP: http://sourceforge.net/projects/universe-phpext/

/greger

yup

[Janek Kozicki]

that next-generation programming systems will combine compilers, linkers, debuggers, and that other tools will be plugin frameworks, rather than monolithic applications.

yup, it already happened. more than 10 years ago. it's called Rule of modularity and Rule of Composition. In case you don't know. It's the Basics of the Unix Philosophy

Re:Can't wait

[exp(pi*sqrt(163))]

Actually, this example often isn't optimized away which is why the Blitz++ library exists. In fact, the author of the library, Todd Veldhuizen, has written at least one paper spelling out what compiler developers need to do in order to ensure stuff like this is optimized away. I've done lots of experiments myself. As soon as you put something like my example line inside a template that is instantiated from another template etc. I've found compilers start missing what you think ought to be easy optimizations. It's very frustrating, I just want to tell the compiler myself how to do its job because it's often just a mindless application of a bunch of rewrite rules.

Re:Next generation for ME

[warriorpostman]

.NET provides "remoting" and Java provides RMI. These are essentially Remote Objects. CORBA provides this cross-platform. If I understand exactly what you're asking for, it already exists. Clients can instantiate objects remotely, and maintain virtually local control of these remote objects, using the aforementioned technologies.

However, if you're using PHP, the whole concept of remote object has to be patched together (just like in ASP) simply because there is no true "state" in web applications.

Re:No one here wants to hear this...

[Anonymous Coward]

Uh... and emacs, that extensible editor/IDE thing written in a Lisp dialect...

Re:Next generation for ME

[Anonymous Coward]

Ruby does all this very transparently already with the Distributed Ruby library which is part of the standard Ruby.

Basically all you do is this:

# Server
# (Sorry about the bogus underlines, Slashdot kept wanting to remove my pretty whitespace!)
class Calculator
_ def add(a, b)
_ _ return a + b
_ end
end
DRb.start_service('druby://server:9000', Calculator.new)

# Client
calculator = DRbObject.new(nil, 'druby://server:9000')
calculator.add(2, 2) # => 4 -- Note that the calculation will be done by the server!

It's as simple as that. You can run the Server and the Client on entirely different computers, thousands of miles away and everything will just work. There are layers that build on top of this which provide master server functionality (clients and servers can find each other via an application server), firewall layers and more. All those however keep in the spirit of the simplicity of the example code above.

There's more information about this available at this article. Just one more quote I would like to highlight:

Ho hum, you say. This sounds like Java's RMI, or CORBA, or whatever. Yes, it is a functional distributed object mechanism---but it is written in just 200 lines of Ruby code. No C, nothing fancy, just plain old Ruby code.

Interesting operators

[jfdawes]

What maybe a few people have missed is that there will be some incredibly interesting "hardware" out there in the future.

Some people have already demonstrated things like using DNA computers to solve travelling salesman problems, Quantum Computing and Grid Computers.

Perhaps what this article is suggesting is one way for developers of entirely new "hardware" to easily supply operators and types (syntax) to any programming language.

It would be interesting to be able to write program a that talked directly to the nervous system using fairly standard <your language of choice> syntax, that when compiled produced a real piece of nano "machinery".

dealing with XML

[Yobgod Ababua]

That's exactly one of the author's points! You shouldn't (and in his vision won't) have to deal with the XML directly, -unless- you are one of the people actually writing new plugins rather than just using them.

His suggestion is primarily that we start using editors that transparently present the 'code file' in our choice of format rather than forcing us to edit it byte-by-byte. It's like the syntax-highlighting you probably use now, only effecting more than just colors.

Using XML for the underlying syntax is mostly irrelevant to his proposal, but he suggests it merely because it is currently popular, well suppoerted, and well suited to it's primary job of presenting data in an easily MACHINE READABLE format.

His proposal is, in fact, exactly the opposite of requiring coders to pop open a hex editor, and he likens our current ASCII-only coding methods to doing exactly that at one point.

Inventing Lisp again

[richieb]

Paul Graham said that all languages hope to become Lisp. This sounds like just another attempt.

Why not just use Lisp?

Compiler extension (was:Can't wait)

[real gumby]

It would be great, if instead, I could hook into the compiler and tell it exactly how it should handle vectors.

Well of course that's what templates are. Yes, their syntax is horrendous but that's what comes of trying to wedge the concept into the existing crannies of C syntax (or when, as Stroustrup remarked to me once, "the ecological niche was already polluted").

If you hanker for a language in which metasyntactic extension is natural, you need Lisp macros (or here and here for a more complex example), Scheme "hygenic" macros or the CLOS MOP.

But if you really want to consider "hooking into the compiler" as you say then you should look at the reflective programming work, the ground work for which was laid down almost 25 years ago by Brian Cantwell Smith and was even implemented, by me and others, back then. Although a lot of work continued in this area that vein pretty much got mined: unless you can think up a completely new control structure there's not a huge amount more you can do with such a system than you could with a normal metasyntactic extension mechanism.

HTH
-d

Re:Next generation tools...

[E_elven]

Just for clarification: you don't actually code in XML. This is what it means:

&lt;function&gt;
&lt;name&gt;foo&lt;/name&gt;
&nbs p; &lt;abstract&gt;Blah blah&lt;/abstract&gt;
&lt;args&gt;
&lt;arg&gt;
&lt;name&gt;bar&lt;/name&gt;
&nbs p; &lt;type&gt;int&lt;/type&gt;
&nbs p; &lt;purpose&gt;Blah blah&lt;/purpose&gt;
...

When you fire up your IDE, you'll see:

// abstract:
// Blah blah
// parm:
// bar -Blah blah
int foo(int bar)
{
...
}

Joke was funny, though.

Related Discussion...

[Meijer]

on Lambda the Ultimate.

This will be in Longhorn

[objwiz]

IMO, this is nothing new. Microsoft's Longhorn presentation, at the PDC in LA last fall, made it pretty clear that Longhorn will be shipped with the compiler and an extensive class framework "installed".

C# is part of the operating system and the file system. The operating system will generate "partial classes" for new file "types" and other functionality on demand.

Since the compiler will be part of the OS, I could write an application that could generate code and run it, leveraging existing classes already on the system.

Re:Glaring error at beginning of article

[voodoo1man]

If sucess is measured by longevity, COBOL & Fortran are the most sucessful programming systems in history.

The problem here is in the definition of system. FORTRAN and COBOL are programming languages - the systems for programming in them have changed drastically over the years, from punched cards to teletypes to full-screen glass teletypes. The Unix command line, on the other hand, has pretty much stayed the same since it was introduced. You can only type in the current prompt, editing the prompt is awkward, and previous input and output scrolls up off the screen. The three biggest changes have been job control (so you can edit your command line with vi), pipes, and editing-friendly shells (tcsh has nice key-bindings borrowed from Emacs for moving point, deleting, etc.). I think they were introduced in that chronological order, although I could be mistaken.

Re:The horror... the horror...

[hyc]

"half human-readable" is just a bit too wishy-washy for me. HTML is human readable, and it does its job pretty well (aside from true horizontal positioning control, instead of faking it with Lists and Tables) but that's because the 'T' stands for TEXT and it's intended primarily for human readable text.

For anything that is meant to be machine-processed, like a low level data format, human-readability is a non-issue. For anything that is meant as general purpose, large capacity data storage, human-readability is a liability because the volume of metadata eclipses the volume of data.

As a slightly better example, ASN.1 is sometimes tedious and even overly verbose, but it is far more compact than XML, is at least as expressive, and is trivial to parse with a simple state machine.

What annoys me about "XML" is that a "Markup Language" is supposed to describe data *for presentation*, that's what Markup is all about. Using it as a general purpose database format is truly a misuse of the SGML technology.