Slashdot Mirror
Lightweight Scripting/Extension Languages?
Andy Tai asks: "Extension languages are designed to be embedded in applications to support customization of the application behavior. Common scripting languages, like Perl and Python, are fairly 'large' with powerful run-time engines and libraries and are widely available and 'script' writers usually assume their stand-alone existences in the deployment environment. However, if one is looking for a language that's small enough so its source can be embedded in the distribution of and built as part of the application, Python and Perl may be 'overweight.' For the real lightweight choices there are Lua and Tinyscheme. Are there others? What are people's preferences and opinions regarding lightweight extension languages?"
Is PHP/Perl/Python too big? Just use them. I don't care about a couple more megs of system RAM. RAM is cheap. Learning a new language takes time.
Unless there are extremely good grounds for using some annoying new/odd language, use something mainstream.
God knows the words "lightweight" and "Gnu" don't generally go together, but how about Guile?
Its easy to learn and use (very good if the people using our embeded language are not programmers). Its mature, supposed to be extensible, and if you decide you need something more powerful later on even the current full verion is not that bloated.
Just how much customising do you need?
From my experiance if you only need a little customisation then you can do most of that stuff from a config file, .ini if you want it to be easy to edit, xml if your going to write a gui to do it or your users are compertant. Any more than that your better off going with something like python or perl, they are widely known, and have rich libraries to let the people using the app do some really imaginative things.
The other thing that you can do is write your app in such a way that it is easy for scripting langs to get there hooks into. For example, on windows you can do this by providing com interfaces to all the things you want to have control over, then you can use any lang you want to control your app.
Is usually a sign that you are using too low-level a language for the application itself. There is no reason you can't write your whole app in something like Python (or better yet, a compiled high-level language).
I can see why you might want to stay with C/C++ for a major commercial application (not because of speed reasons, but because the maturity of the implementations of alternative high-level languages) but it pains me to see tons of OSS software, especially non-speed-critical GUI apps, still being written in C!
A deep unwavering belief is a sure sign you're missing something...
Gentlemen, this one is a dupe. The question is: Was this ask slashdot submitted twice, or did the editor failed to remove it from submission queue last time?
ECMAScript/Javascript seems like a logical choice. it was made for just that - embedding. It's quite small to implement, and there are many opensource implementations out there already. It can be procedural or OO, and everyone knows how to use it if they've done some basic web work.
I find s-lang a very nice extension language. www.s-lang.org
It's sufficiently C-like to suit anyone who has done C/C++/Java/Perl/... but high level enough to be a feasible extension language.
Of course it depends on what type of person you expect to use your extension language. Are they programmers?
Read more of this story at Slashdot.Read more of this story at Slashdot.Read more of this story at Slashdot.
i recommend you look at paxscript.
it's the most complete and quickest of any embedded source stuff that we've investigated.
supports basic, c, objectpascal (ie delphi) and javascript.
not free, but you get the source when you purchase it (US$179).
delphi and borland c++ builder only.
nostrils
didn't see anyone else mention it, so i thought i would.
REBOL is a ridiculously simple and easy-to-learn web-oriented language. so easy, in fact, that i wrote a fully-functional IRC dice bot in under 400 lines, overnight. and if that weren't impressive, then might i add that i was running on ZERO caffeine, learning the language for the first time, learning IRC Client protocol for the first time, and came up with a few unusually witty statements and insults to boot?
now the bad part: REBOL is not open source. poo. (i really was a bit disappointed.) but REBOL/Core is free (for any use, i gather), and the license fees for View and Command seem rather reasonable.
the nice part: it has been ported to and runs on about 43 platforms, last i checked, and is light enough that the executable weighs in at around 250kb for the win32 release. (haven't used the other platforms, so no comment.)
it runs on just about every unix i've heard of, on every relatively common configuration, and works beautifully and seamlessly. and, after a quick glance, i see it runs on serveral major embedded systems, including WinCE, QNX, and Linux, and will even run on my friend's dated Amiga.
enjoy and happy coding.grey wolf
LET FORTRAN DIE!
TinyPerl
Error: PANTS NOT FOUND. Press <F1> to continue.
ficl.sourceforge.net
A portable object oriented forth at less than 100K for a full implementation, able to handle calls from your apps as well as call your apps fgunctions or other dll's.
Under a BSD-ish license.
The upside:
1) Its tiny
2) The forth code you write will be tinier
Downsides:
1) You need to learn forth (I mean properly) to appreciate it.
Forth is unlike most other languages, it was designed to avoid the debug, recompile cycle that is so common.
Sam
blog.sam.liddicott.com
I've been working on a system for embedding JS in KDE apps (amongst other reasons it is a about 1/10 the size of a python interpreter). You can find lots of information at http://xmelegance.org/kjsembed. The interpreter in KDE has no dependency on KDE/Qt or anything else so you might find it usable in your app.
I guess atai must have submitted the same question here as well, but the "original" discussion took place on advogato:
http://www.advogato.org/article/735.html
It would be useful if he could state what his requirements and limits are, in detail, because that's a very necessary part of evaluating what would work.
http://www.welton.it/davidw/
Well you're not obligated to provide the libraries. For instance, Blender, which uses Python, did not include Python libraries ; however it was possible for people which really wished it, to set paths to point to libraries. Forcing the users to learn a new language, just to cut the size of the distribution is misguided IMHO. Just provide the extension language (& support) as an optionnal individually downloadable shared library.
I know it's not directly relevant to the question, but FWIW the rather excellent Freedom Force uses Python as its scripting language.
Life is like a sewer; what you get out of it depends on what you put into it...
"...learning LUA takes exactly 1 hour."
I couldn't read the reference manual in one hour. I couldn't learn the quirks of co-routines in one hour.
I think there is a big, big misunderstanding about computer languages. The "quick and dirty" ones are incomplete and limited. The complete languages are necessarily complex. At one time, Perl was like Lua. Then it needed this and that until now it is beginning to be as complex as C++. The same will happen with Lua, I'm guessing, if it remains popular.
It seems that every self-motivated serious programmer writes at least one editor or one language or one compiler. Even I wrote a language and compiler -- for Hewlett-Packard data acquisition equipment.
I think Larry Wall didn't foresee the future when he started Perl. Did he begin Perl with the idea that it would eventually be object-oriented? Did he expect that he would spend his entire life developing his "report language"?
I'm tired of learning new languages. I don't see the point in it. I'll bet there are hundreds of thousands who learned Pascal in college who wish they had not wasted their time; except for Delphi, Pascal is dead.
For those who want an embedded scripting language, how about a C interpreter? Try Cint or Ch.
Or, how about EiC, now hosted at Sourceforge. "EiC is pointer safe."
I'd reccomend siod, or perhaps guile.
http://people.delphiforums.com/gjc/siod.html
I've done a fair bit with siod lately, and can say that while it's not the most sophisticated scheme interpreter around, it dovetails very nicely with C code. It's trivial to write new functions, and defining new data types isn't that much more difficult at all.
That's as close to Lisp macros as you can get.
For people looking for Lisp-style macros, siod includes a simple implementation of a feature that works very much like macros. If the symbol in the function spot of a list itself refers to a symbol, the referred to symbol's binding is evaluated as a macro rewriting function.
Oh, and as an example of its simplicity, the whole thing fits in something like 10,000 lines of code. That's garbage collection, a library of simple string functions, a Scheme evaluator, file I/O, and a bunch of stuff I'm probably forgetting...
IIRC, SCM and Guile are both derivations of the original siod code base. For that matter, so is STk, as it it itself based on SCM. Either of those would likely be a good choice too.
What about Ruby (ruby-lang.org). It's easy to learn, powerful and has a reasonable set of classes and methods built-in. It's also quiet widespread nowadays, lightweight compared to Python and easy to embed via libruby.so
see the Ruby book (section Embedding a Ruby Interpreter
#define WAIT() do { } while( rand() % rand() )
Then I just sprinkle in a few WAIT()'s wherever my code runs too fast.Yes, I'm aware that rand() can return 0. :)
Hint: I never said my code doesn't crash!
On Mac OS X you definitly will use AppleScript. Regardless of programming language used for your application. OTOH, you can use any language that offers a library to send high levle events to your apps(not sure about that in our days).
t ml
... as soon as you start working with SWIG you will realise that making an application scriptable menas to craft special hook functions to access only the wanted part of the application, likely via strings only arguments and return values.
On Windows you likely will use Visual Basic for Applications or Visual Basic Script or any other Active Scripting language.
If you write ONLY Java, like I do in our days, its easy: the scripting language will be either Java(Dynamic Java, see: http://koala.ilog.fr/djava/ or Bean Shell, aka bsh, see: http:www.beanshell.org) or Java Script, aka Rhino, see: http://www.mozilla.org/rhino/ or any other language running on a JVM see: http://grunge.cs.tu-berlin.de/~tolk/vmlanguages.h
If you write your program ina C-ish language I would consider to use CORBA --- yes indeed!! --- to be able to script it with everything.
If you want to use traditional scripting languages like TCL, PERL etc. you likely will want to look into SWIG, see: http://www.swig.org.
But
The bigger your applicatin gets, the more complex will be the access layer for the scripting languages.
When you have reached that point you likely regret not to have used something more suiteable, like CORBA or the Java stuff I sketched above or you are to the point where you throw it away and rewrite it in a "embed" approach where you will use a scripting language as development language and write certain modules as plugins into that language.
Then however you are bind to the scripting language as host language.
BTW: there are a lot of free mini orbs which make CORBA a toy. And writing IDL is not much harder than writing SWIG config files. If you like to shield your customer from CORBA give him a lib for his scripting language, which hides the fact that CORBA is used.
Regards,
angel'o'sphere
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
If the app runs on JavaVM, then JavaVM-based languages (such as Jython and Groovy) may be good choices, since the VM and its large set of libs are there anyway.
Perl 4, while dated for newer Perl programmers is a very effective language and weighs in around 400k.
I can think of a couple of big, $$ applications that embed it.
Conformity is the jailer of freedom and enemy of growth. -JFK
Nasal is a very well designed small embeddable language with a Javascript-like syntax and an LGPL license. I prefer it to Lua because I am not fond of Lua's Pascal-style syntax. Be aware that Nasal is not very battle-hardened yet and does not have even a 1/100th of the developer community of Lua.
lua is a great scripting language when it's used well. When i was studying game development in school, we were introduced to it as a good way to drive out games. Its easy to add functionality for it... its fast... it simple. I could write all my rendering/sound/network/input code in c++, and just have interfaces for everything from lua... this way, when i wanted to drive a game app, i could do the following: (pseudo lua... hmm... pseudo scripting :) )
load level
load char at xyz
load enemy at xyz
go
then, my c++ code could handle all the collision detection/AI/proper sounds for things... it all comes down to good design. Something like lua is great for having somethign high level drive a low level engine.
Wouldnt you like to be a pepper too?
When I was answering this question, I ended up looking at nasal. While not perfect, it was the closest to what I wanted ... and was under a compatible license (LGPL). But then I wanted something small, lua is way too big.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
Assuming that end users are going to be writing things in this language, you have to use one that there are already books/courses about, that their friends already know how to program in, etc.
This pretty much forces things like Javascript or VBScript, which may not be pretty but have the mindshare. You could use something more elegant and exotic, but will anyone bother to learn it? Will you be willing to support them when they haven't?
Pascal wasn't just any language. It was the most taught computer language in the world at one time. It died within a few years. No one seems to be mourning it. So, how many languages do we really need, if one can die without causing problems?
My vote: Put all the good stuff into one language.
Lua as a library is 60K on x86. Nasal is not much smaller - 45K. Lua can be smaller if you do not register some builtin functions. They are basically the same sizewise, but Lua interpreted code runs several times faster.
Is there a place that describe all the major programming languages with descriptions?
I agree, but I've already had a lot of fun and brain exercise doing programming. I'm looking for a permanent relationship. I want to get married.
It's under 2 megs when stripped.
That's small, isn't it?
for you windows guys, better check out winbatch. It's BASIC - but you can compile it. There are "extenders" for just about every task you can think of. You can access OLE/COM - but chances are there is already a library for what you need. There is an IDE with built in debugging. Context sensitive help. Fantastic documentation too - great support community as well.
Stolen from their webpage:
WinBatch is:
A complete programming language designed for all utilities
A perfect tool for power users and system administrators
A LAN utility development tool
An Internet utility development tool
A business process automator
A high level programming language that is easy to learn and use, without the learning curve and complexity of traditional programming languages such as C, C++, and VB.
No - it's not free - but it's cheap. The compiler is fairly expensive at $500 though.
It's a fantastic tool - couldn't imagine managing windows boxes without it. Screw VBScript, why should I mess with objects to send something to the console or get at an environment variable?
You know... most of the size of embedding Python is from the default library, which does not need to be included for embedded scripting. The actual binary of the python interpreter is barely over 20k.
There's no reason *not* to use Python for your embedded scripting, IMO.
"Times have not become more violent. They have just become more televised."
-Marilyn Manson
BeanShell is the original lightweight scripting extension language for Java.
http://www.beanshell.org/
Pat
Small (without the Tk part), multi-platform, truly interpretive, and created to be embeddable. From John Ousterhout's introduction to his book about his language:
What else do you need? Oh, and if you are still thinking of making your own -- keep rereading the first paragraph of the quote above, until the silly thought leaves your mind...
In Soviet Washington the swamp drains you.
Until the 1.8 release the whole Ruby source code and libraries could be distributed on a 1.4MB floppy with room to spare. With 1.8 a lot of libraries have been added so it's about twice as big. (you can always leave out libraries you don't need to keep it light)
Ruby is also easy to embed as well, but we decided to use swig to wrap our C++ classes so that they're accessable from Ruby and then script the whole thing in Ruby. This offers several advantages for our application since we wanted to allow users to script it anyway.
Another project involves including Ruby and libs on a CD with a Ruby script that is used for license detection/management and installation. The Ruby script queries the rpm repository (on Linux) or registry (on windoze) and makes certain determinations about the system on which the actual software on the CD is to be installed. Turns out that including Ruby and most of it's libraries on the CD took much less room than the statically linked executables we used to include for doing the same thing (which were developed in C++). At any rate, the potential user of our CD doesn't need to have Ruby installed on their system because we include it on the CDROM and start up the Ruby script from a shell script (or a bat file on Windows.). Oh, and the Ruby code for this particular project that replaced C++ code is about 1/3 as big and only took us 1/4 of the time to develop & debug as compared to the previous C++ solution. And the Ruby-based system is much more flexible than the previous C++ one - when new requirements come in we can usually have the Ruby system updated & tested in a matter of hours as compared to days for the previous C++ based system. The productivity gains alone were well worth the conversion to Ruby.
But TCL is showing it's age these days. TCL is really a pretty ugly language (has 'let' assignment syntax as I recall - oh the pain, I don't want to remember). It was fine in 1988, but now programmers hope for more. Seriously, there are much better choices now. I'm partial to Ruby myself (http://www.ruby-lang.org). It's quite easy to embed in a C/C++ app, or you can use SWIG to wrap your C/C++ code for access from Ruby. Ruby is pretty small as well, the sourcecode and libraries are about 1MB (well, it's a bit larger now with the 1.8 release, but you don't need all of the libraries that are included now).
COM/ActiveX works swimmingly in this regard in that you have a choice of languages for both producing and consuming COM/ActiveX components, and COM allows you to pass object references across the barrier while ActiveX allows you to work with GUI widgets in this way. And it is true multi-language, not the fake multi-language of the .NET languages.
On the other hand, COM/ActiveX pretty much limits you to Windows, you have to be careful of the lowest-common-denominator effect of which COM data types work in which languages (the more restrictive set of "Automation" types compared to the wider set of C-compatible types and a more restrictive situation for scripting languages and Web browsers than with Visual Basic 6). Also, COM/ActiveX can get ugly in some environments (C++ and MFC/ATL -- you are really stuck taking the Wizard-generated code at face value unless you really know what you are doing), and as a component developer you have to pick up some IDL basics.
Is there anything out there that even comes close? With the Python/C-C++ meld, you are pretty much restricted to Python on one end and C++ on the other hand, and the C++ end is, shall we say, a bit lower level than the Python end, although there are things like SWIG that look amazingly like, hey, IDL and Visual Studio Wizard code.
Another thing I have looked into is the Matlab-Java meld -- a lot of academic engineer and scientists use Matlab as their all-purpose scripting tool, but I wouldn't want to develop high power apps in it. The Matlab-Java meld is surprisingly clean, the Java side can be a Swing app and it runs in its own Swing windows. Again, you are restricted to Matlab on the scripting side, Java on the custom app side, but the interesting thing is that it is pointedly the sort of thing SUN isn't interested in. According to the SUN world view, Java is everything you would ever need and the idea of scripting Java from anything else doesn't enter into their universe.
Is there anything else out there that scripts Java? I suppose there is Jython. Anything else?
One aspect of the big scripting languages that has been deeply unsatisfying in my own personal search for a language to embed has been the lack of robust sandboxing support, and generally weird hacks to support multiple simultaneous interpreters.
My own particular application requires executing scripts entered in by unprivileged users. Basically, the idea is that users can log into the service and run (simple) programs on the server to perform mundane tasks. I suppose the closest idea you could compare it to would be the use of procedural languages in SQL DBs (the actual idea was inspired by MU*s with scripting languages, specifically the MUSH lines).
Most of the big scripting languages are far too powerful for this task, leaving open possibilities for compromises and exploits galore. Efforts to sandbox these advanced scripting languages generally seem to fail due to the sheer complexity of possible interactions.
As much as I'd like to support something everyone is familiar with, it seems like the only feasible approach is to develop my own interpreter from the ground-up, keeping in mind my security and resource management goals from the very start. I'm sure there's a scripting language invented like that already, but it's (1) hard to find, and (2) probably isn't documented well enough for me to have confidence in its design. Without a big mass of similar developers who have successfully used the source code, I'd have to vet the code myself anyway.
So there you have it, an embedding problem which you can't solve by simply 'rewriting the application in the scripting language' or 'embed popular engine X'. It seems to require a scripting language specifically designed for embedding, sandboxing, and multiple interpreter instances from the very start. I think Scheme might work pretty well.
I can't believe LISP wasn't suggested. LISP has one of the most simple syntaxes in the world to parse. A lisp program basically consists of a list of functions. (functionName arg1 arg2 arg3) ....
(function)
such that each arg is! you guessed it, a function. parsing is so easy a 4 year old could handle it. implement some of the more basic methods needed for a useful application and you are pretty much done.
- tristan
'Nuff said.
Ferite is a scripting language and engine all in one managable chunk. It is designed to be easily extended in terms of API, and to be used within other applications making them more configurable and useful to the end user. It has a syntax similiar to a number of other langauges but remains clean and it's own language.
I love that it was designed from the Start to be Thread Safe and Embeded, unlike PHP or Python.
I agree. So, how could those features be sensibly added to C++?
Why do we have to abandon a language and invent new ones, with new quirkinesses?
It takes great ability to put stuff into a language, and do it right. I'm not talking about doing it wrong.
If you use Java, possible the best choice is Groovy
http://groovy.codehaus.org/
Groovy is very tightly integrated with Java, you can use Groovy classes from Java and Java classes in Groovy." Groovy supports both static and dynamic typing so Java objects are seamlessly available in Groovy and vice versa. I can derive/overload Java classes in Groovy that are available to Java developers. i.e. Groovy is Java really; just using a Ruby-ish syntax." - James Strachan
SUN and Java is in the one language is all you need for anything camp. Python is perhaps in the 2-language camp -- use Python for scripting, extensions, and even for major parts of the system, use C/C++ for core modules.
You forgot a big one: Tcl. There is a pure Java Tcl interpreter Jacl. It's trivial to embed and use. I've used it to add scripting to tools, and to run the same Tcl scripts on Windows, Linux, Solaris, OS X, all from a single Java jar running on all of those.
Remember, Tcl does stand for "tool command language:. :-)
Lua will not get more complicated because it is designed to be a lightweight embedded scripting language. The only complexity is that which the embedders choose to add to make the language useful within their programs. I read the Lua mailing list and it looks like it's going to remain as simple as it is (~10k lines of the common subest of ANSI C and C++).
True story.
And so does Unix -- if you listen to Windows zealots -- the 1970-ies technology...
Yes, and penguins are pretty ugly creatures. It is a matter of perception and -- as you mention -- partiality.
SWIG can generate TCL wrappers (and Python, and Perl) just as well. TCL, however, was created to be embeddable. And its syntax allows for easy to understand application configuration files. Ruby may well be good, but that does not mean TCL is not.
In Soviet Washington the swamp drains you.
Now here is a good thing, the spammers can bombard us with junk about language extensions instead of extensions to a part of the male anatomy, which at the moment seems to constitute 90% of all spam.
The only way I can do this is by using a database instead, and a lot of bothersome fiddling about. Neither a database nor a spreadsheet come close to what I really need, an entirely new dimension of programming capability is needed.
Even better if it would only work in OpenOffice.org, and under Linux or BSD, it would then provide a lever to move the management....
Some way of actually putting shell scripts into cells, and connect the variables to other cells, would be the ultimate perfection, because then any other functionality needed would be accessible.
I know what I need, but am not a good enough programmer to do it.
Now, I hope that this is not completely off topic, it is after all an extension, even if not to a language as such. I also hope it will inspire competent people to progress spreadsheets to the next dimension and get way ahead of M$.... Maybe a super-extended spreadsheet would be the very thing to make several open source spreadsheets give M$ Office the sound thrashing that it deserves.