Lightweight Scripting/Extension Languages?

Andy Tai asks: "Extension languages are designed to be embedded in applications to support customization of the application behavior. Common scripting languages, like Perl and Python, are fairly 'large' with powerful run-time engines and libraries and are widely available and 'script' writers usually assume their stand-alone existences in the deployment environment. However, if one is looking for a language that's small enough so its source can be embedded in the distribution of and built as part of the application, Python and Perl may be 'overweight.' For the real lightweight choices there are Lua and Tinyscheme. Are there others? What are people's preferences and opinions regarding lightweight extension languages?"

Comments

Guile

[crmartin]

God knows the words "lightweight" and "Gnu" don't generally go together, but how about Guile?

Re:Guile

[aspjunkie]

just Guile!? Don't forget Ryu, Chun Li, and Dhalsim! ;)

TinyTCL

[the_womble]

what about Tiny TCL

Its easy to learn and use (very good if the people using our embeded language are not programmers). Its mature, supposed to be extensible, and if you decide you need something more powerful later on even the current full verion is not that bloated.

Re:TinyTCL

[Twylite]

Tcl/Tk proper is not a bad idea either. If one does not need the GUI part (Tk) then Tcl is relatively easy to integrate into a project, and is under 200k.

HowToEmbed from the Tcl Wiki is a good starting point, and MkTclApp may help.

Tcl has very consistent and simple syntax (although it can get rather confusing at times), and it is very simple to add new command into your application that are exposed via Tcl. One of the nicest aspects of Tcl is that is it seriously multi-platform.

Of course this all depends on the requirements: how powerful does the language need to be, what functions must it provide, what types do you need it to be able to handle, how small is "lightweight", etc.

Re:TinyTCL

[ianezz]

One of the nicest aspects of Tcl is that is it seriously multi-platform.

Another nice aspect of Tcl is that it can easily evaluate code in a different stack frame (example: in the caller's context), and source code can be easily passed to procedures as strings between braces (as it is usually done), so extending/reimplementing the language control structures is as simple as writing a new procedure (and no special/ugly syntax is required).

That's as close to Lisp macros as you can get.

The need for "extension languages"

[be-fan]

Is usually a sign that you are using too low-level a language for the application itself. There is no reason you can't write your whole app in something like Python (or better yet, a compiled high-level language).

I can see why you might want to stay with C/C++ for a major commercial application (not because of speed reasons, but because the maturity of the implementations of alternative high-level languages) but it pains me to see tons of OSS software, especially non-speed-critical GUI apps, still being written in C!

Re:The need for "extension languages"

[be-fan]

C programs don't force hardware to resemble a PDP-11, but the C machine model doesn't resemble the actual hardware as much as C programmers like to believe.

- C is entirely serially, and the compiler has to infer paralellism from the code. Modern hardware is significantly parallel.

- C is entirely memory-based, while modern hardware does everything it can to avoid referencing memory.

- C has no primitives to describe branching, to aid things like speculative execution.

- C has no vector types, while most modern processors have vector units.

etc,etc,etc.

C has a very nice, simple machine model, but C programmers don't really understand what the computer is doing "under the hood."

Javascript!

[Geek Boy]

ECMAScript/Javascript seems like a logical choice. it was made for just that - embedding. It's quite small to implement, and there are many opensource implementations out there already. It can be procedural or OO, and everyone knows how to use it if they've done some basic web work.

Re:Javascript!

[Anonymous Coward]

Mozilla implementation of JavaScript in C is available at http://www.mozilla.org/js/ and pure Java implementation is at http://www.mozilla.org/rhino/

And btw, ECMAScript/JavaScript can be pure functional as well.

ICI

[tpv]

ICI is very lightweight and generally quite usable by semi-programmers.

It's sufficiently C-like to suit anyone who has done C/C++/Java/Perl/... but high level enough to be a feasible extension language.

Of course it depends on what type of person you expect to use your extension language. Are they programmers?

REBOL

[the_greywolf]

didn't see anyone else mention it, so i thought i would.

REBOL is a ridiculously simple and easy-to-learn web-oriented language. so easy, in fact, that i wrote a fully-functional IRC dice bot in under 400 lines, overnight. and if that weren't impressive, then might i add that i was running on ZERO caffeine, learning the language for the first time, learning IRC Client protocol for the first time, and came up with a few unusually witty statements and insults to boot?

now the bad part: REBOL is not open source. poo. (i really was a bit disappointed.) but REBOL/Core is free (for any use, i gather), and the license fees for View and Command seem rather reasonable.

the nice part: it has been ported to and runs on about 43 platforms, last i checked, and is light enough that the executable weighs in at around 250kb for the win32 release. (haven't used the other platforms, so no comment.)

it runs on just about every unix i've heard of, on every relatively common configuration, and works beautifully and seamlessly. and, after a quick glance, i see it runs on serveral major embedded systems, including WinCE, QNX, and Linux, and will even run on my friend's dated Amiga.

enjoy and happy coding. :)

Re:REBOL

[swdunlop]

I had a very guilty love for REBOL, but it hasn't changed much in the last three years.. Many of the warts on REBOL/View are still there, and it's still not fully documented. They also haven't finished their port to OS X, so it isn't as perfectly cross platform as I would like..

And.. Finally.. How does one embed REBOL in an open source application?

Developing for windows?

[KILNA]

TinyPerl

Object Forth?

[samjam]

ficl.sourceforge.net

A portable object oriented forth at less than 100K for a full implementation, able to handle calls from your apps as well as call your apps fgunctions or other dll's.

Under a BSD-ish license.

The upside:
1) Its tiny
2) The forth code you write will be tinier

Downsides:
1) You need to learn forth (I mean properly) to appreciate it.

Forth is unlike most other languages, it was designed to avoid the debug, recompile cycle that is so common.

Sam

How about JavaScript?

[Rich]

I've been working on a system for embedding JS in KDE apps (amongst other reasons it is a about 1/10 the size of a python interpreter). You can find lots of information at http://xmelegance.org/kjsembed. The interpreter in KDE has no dependency on KDE/Qt or anything else so you might find it usable in your app.

Copy of an advogato article

[DavidNWelton]

I guess atai must have submitted the same question here as well, but the "original" discussion took place on advogato:

http://www.advogato.org/article/735.html

It would be useful if he could state what his requirements and limits are, in detail, because that's a very necessary part of evaluating what would work.

Why another language?

[Anonymous Coward]

Extension languages are designed to be embedded in applications to support customization of the application behavior. Common scripting languages, like Perl and Python, are fairly 'large' with powerful run-time engines and libraries and are widely available and 'script' writers usually assume their stand-alone existences in the deployment environment.

Well you're not obligated to provide the libraries. For instance, Blender, which uses Python, did not include Python libraries ; however it was possible for people which really wished it, to set paths to point to libraries. Forcing the users to learn a new language, just to cut the size of the distribution is misguided IMHO. Just provide the extension language (& support) as an optionnal individually downloadable shared library.

Consider Ruby

[urbangipsy]

What about Ruby (ruby-lang.org). It's easy to learn, powerful and has a reasonable set of classes and methods built-in. It's also quiet widespread nowadays, lightweight compared to Python and easy to embed via libruby.so
see the Ruby book (section Embedding a Ruby Interpreter

Re:Anything but odd/new language...

[Christopher Cashell]

There are dozens of available library implementations of reasonably well known languages. The trick is simply finding which one fits best for you. And, depending on the application, and how it's used, you might want to consider created a more generalized plug-in architecture, and allow for multiple scripting languages (possibly using something like CORBA).

One thing I would strongly urge, is to pick a "real" language that already exists. Ideally, something like those listed below. Even if it's a fairly uncommon language that you personally aren't familiar with, the fact that it's in use in the real world means that more people will be familiar with it, and that it will likely be a more stable and usable solution.

Creating a brand new language just for your application is a gauranteed way to annoy people because there is no chance that they will know the language before using your application.

Here are some possibilities, depending on what you're looking for:

Scheme/Lisp* based:

• librep, used by Sawfish
• Guile, the official GNU extension language
• Elk, designed from the start as a small embeddable library
• tinyscheme, another small implementation designed for embedded use
• SIOD, a very small implementation that can be easily customized for your application
• Kawa, a Scheme enivornment written in Java, that compiles Scheme code to java byte-codes, is an excellent choice for use in a Java application
• Embeddable Common Lisp, while not as small or lightweight as most Scheme implementations, does a pretty decent job

For larger (and more common) languages, you can choose from:

• JavaScript/ECMAScript, this is a great choice as it was designed to be used as an extension language, and there are at least a couple of free/open source implementations available. . . and it's use in web browsers/web pages has made it a very well known language
• eperl, an embeddable form of Perl
• Python, though not ideally setup for this, can be embedded in another program
• tcl, tcl was designed parly for use as an embedded scripting language, and works fairly well in this regard

Some less well known, but still "real" languages that can make an excellent extension language include:

• ficl, an implementation of Forth that's designed for embedded use
• Lua, a language that was designed specifically to provide a small but powerful language for embedded use
• S-Lang, a progammers library that provides a console GUI library and an extension language (used by jed, slrn, newt, mutt (optional), etc
• cint, a C language style interpreter could probably be used easily enough
• ch, another c interpreter
• Bean Scripting Framework, BSF, is a great choice for java applications, as it provides a framework to use a numer of different languages, providing greater choice
• Scriptix, an extension language with a C/C++ style syntax that evolved out of MUDix
• ferite, an interpreter with "similiarities to perl, python, C, Java and pascal", and designed for embedded use
• bsh, or BeanShell, is a small embeddable Java interpeter, written in Java, and has some extra "bonus" features of particular interest to scripters

* Scheme, due to it's small size and powerful, easily extendable syntax, seems to be quite common when it comes to extension langauges. A list of programs that use Scheme (or a similar lisp dialect) off the top of my head includes: SIAG Office, The Gimp, Emacs, TeXmacs, Gnumeric, AutoCAD, Sawfish, GnuCash, Snd Sound Editor, etc.

Check out Ferite

[FireChipmunk]

Ferite is a scripting language and engine all in one managable chunk. It is designed to be easily extended in terms of API, and to be used within other applications making them more configurable and useful to the end user. It has a syntax similiar to a number of other langauges but remains clean and it's own language.

I love that it was designed from the Start to be Thread Safe and Embeded, unlike PHP or Python.