Lightweight Scripting/Extension Languages?

Andy Tai asks: "Extension languages are designed to be embedded in applications to support customization of the application behavior. Common scripting languages, like Perl and Python, are fairly 'large' with powerful run-time engines and libraries and are widely available and 'script' writers usually assume their stand-alone existences in the deployment environment. However, if one is looking for a language that's small enough so its source can be embedded in the distribution of and built as part of the application, Python and Perl may be 'overweight.' For the real lightweight choices there are Lua and Tinyscheme. Are there others? What are people's preferences and opinions regarding lightweight extension languages?"

Comments

Anything but odd/new language...

[cymen]

Is PHP/Perl/Python too big? Just use them. I don't care about a couple more megs of system RAM. RAM is cheap. Learning a new language takes time.

Unless there are extremely good grounds for using some annoying new/odd language, use something mainstream.

Re:Anything but odd/new language...

[awgriff279]

I would agree with this, except that it makes for somewhat inefficient programs when one blows a couple extra megs of ram. Efficient, small programs are better long run because they are easier to maintain and run faster. However, for a hack, macgyver job nearly anything can be used.

Re:Anything but odd/new language...

[JohnFluxx]

I agree - especially as python can be really tiny.

Also, read this:

Extend vs Embed

Often people only think of putting python(etc) inside their program, rather than the other way round - putting their program inside of python. Read the above link.

Re:Anything but odd/new language...

[Christopher Cashell]

There are dozens of available library implementations of reasonably well known languages. The trick is simply finding which one fits best for you. And, depending on the application, and how it's used, you might want to consider created a more generalized plug-in architecture, and allow for multiple scripting languages (possibly using something like CORBA).

One thing I would strongly urge, is to pick a "real" language that already exists. Ideally, something like those listed below. Even if it's a fairly uncommon language that you personally aren't familiar with, the fact that it's in use in the real world means that more people will be familiar with it, and that it will likely be a more stable and usable solution.

Creating a brand new language just for your application is a gauranteed way to annoy people because there is no chance that they will know the language before using your application.

Here are some possibilities, depending on what you're looking for:

Scheme/Lisp* based:

• librep, used by Sawfish
• Guile, the official GNU extension language
• Elk, designed from the start as a small embeddable library
• tinyscheme, another small implementation designed for embedded use
• SIOD, a very small implementation that can be easily customized for your application
• Kawa, a Scheme enivornment written in Java, that compiles Scheme code to java byte-codes, is an excellent choice for use in a Java application
• Embeddable Common Lisp, while not as small or lightweight as most Scheme implementations, does a pretty decent job

For larger (and more common) languages, you can choose from:

• JavaScript/ECMAScript, this is a great choice as it was designed to be used as an extension language, and there are at least a couple of free/open source implementations available. . . and it's use in web browsers/web pages has made it a very well known language
• eperl, an embeddable form of Perl
• Python, though not ideally setup for this, can be embedded in another program
• tcl, tcl was designed parly for use as an embedded scripting language, and works fairly well in this regard

Some less well known, but still "real" languages that can make an excellent extension language include:

• ficl, an implementation of Forth that's designed for embedded use
• Lua, a language that was designed specifically to provide a small but powerful language for embedded use
• S-Lang, a progammers library that provides a console GUI library and an extension language (used by jed, slrn, newt, mutt (optional), etc
• cint, a C language style interpreter could probably be used easily enough
• ch, another c interpreter
• Bean Scripting Framework, BSF, is a great choice for java applications, as it provides a framework to use a numer of different languages, providing greater choice
• Scriptix, an extension language with a C/C++ style syntax that evolved out of MUDix
• ferite, an interpreter with "similiarities to perl, python, C, Java and pascal", and designed for embedded use
• bsh, or BeanShell, is a small embeddable Java interpeter, written in Java, and has some extra "bonus" features of particular interest to scripters

* Scheme, due to it's small size and powerful, easily extendable syntax, seems to be quite common when it comes to extension langauges. A list of programs that use Scheme (or a similar lisp dialect) off the top of my head includes: SIAG Office, The Gimp, Emacs, TeXmacs, Gnumeric, AutoCAD, Sawfish, GnuCash, Snd Sound Editor, etc.

The need for "extension languages"

[be-fan]

Is usually a sign that you are using too low-level a language for the application itself. There is no reason you can't write your whole app in something like Python (or better yet, a compiled high-level language).

I can see why you might want to stay with C/C++ for a major commercial application (not because of speed reasons, but because the maturity of the implementations of alternative high-level languages) but it pains me to see tons of OSS software, especially non-speed-critical GUI apps, still being written in C!

Re:The need for "extension languages"

[msuzio]

I don't think you're visualizing the situation correctly.

I don't see how the need for an extension language indicates that at all. An extension language (IMHO) is usually needed so that the end user of the application can script that app and extend it to do things you, the application engine writer, did not supply. The needs of the engine and the needs of the extension language are totally different.

I'm going to want to write the engine to be fast,
efficient, and easy for me to maintain. The extension mechanism should be simple, allow for ease of re-use (so the user community can exchange code snippets amongst themselves), and powerful enough to allow the sort of add-ons and control structures I (the app designer) *think* the user will need.

Perfect example is Neverwinter Nights (although the darn app always crashes on me... but I digress ). Powerful engine, with a scripting language bolted on. You would never try to write that application *in* the scripting language. You might ship a large amount of *content* with the app pre-written using the scripting language, though.

Re:The need for "extension languages"

[kruntiform]

I think that's the difference between C and Python programmers - C programmers enjoy programming.

That's crazy talk.

Python programmers don't, at least everything my Python-loving collegues say suggests that the thing they like about it most is that they never get to understand what's going on under the hood.

Supposing that you are not misrepresenting your Python-loving collegues, then yes, they are probably not good programmers (at least not great programmers). But so what? What have they got to do with anything? I don't even know these people. What about the Python programmers who do know what is going on under the hood (like me :D)? Are they bad programmers too? Really, if you are going to critique some technology, then please stick to technical issues. It's easy to make logically fallacious arguments: C programmers are bad because I know some C programmers that only know what goes on under the hood. They don't know anything else beyond that. And they smell. Therefor C is bad. (Note to any particularly dimwitted moderators out there: I don't believe that is an accurate representation of C programmers at all! I think it is quite untrue in fact. It is just an example of a logically fallacious argument used to illustrate a point. Please don't mod me down, you dipshit moderator!)

Re:The need for "extension languages"

[be-fan]

C programs don't force hardware to resemble a PDP-11, but the C machine model doesn't resemble the actual hardware as much as C programmers like to believe.

- C is entirely serially, and the compiler has to infer paralellism from the code. Modern hardware is significantly parallel.

- C is entirely memory-based, while modern hardware does everything it can to avoid referencing memory.

- C has no primitives to describe branching, to aid things like speculative execution.

- C has no vector types, while most modern processors have vector units.

etc,etc,etc.

C has a very nice, simple machine model, but C programmers don't really understand what the computer is doing "under the hood."

Why another language?

[Anonymous Coward]

Extension languages are designed to be embedded in applications to support customization of the application behavior. Common scripting languages, like Perl and Python, are fairly 'large' with powerful run-time engines and libraries and are widely available and 'script' writers usually assume their stand-alone existences in the deployment environment.

Well you're not obligated to provide the libraries. For instance, Blender, which uses Python, did not include Python libraries ; however it was possible for people which really wished it, to set paths to point to libraries. Forcing the users to learn a new language, just to cut the size of the distribution is misguided IMHO. Just provide the extension language (& support) as an optionnal individually downloadable shared library.