Slashdot Mirror
Lightweight Scripting/Extension Languages?
Andy Tai asks: "Extension languages are designed to be embedded in applications to support customization of the application behavior. Common scripting languages, like Perl and Python, are fairly 'large' with powerful run-time engines and libraries and are widely available and 'script' writers usually assume their stand-alone existences in the deployment environment. However, if one is looking for a language that's small enough so its source can be embedded in the distribution of and built as part of the application, Python and Perl may be 'overweight.' For the real lightweight choices there are Lua and Tinyscheme. Are there others? What are people's preferences and opinions regarding lightweight extension languages?"
God knows the words "lightweight" and "Gnu" don't generally go together, but how about Guile?
Its easy to learn and use (very good if the people using our embeded language are not programmers). Its mature, supposed to be extensible, and if you decide you need something more powerful later on even the current full verion is not that bloated.
Just how much customising do you need?
From my experiance if you only need a little customisation then you can do most of that stuff from a config file, .ini if you want it to be easy to edit, xml if your going to write a gui to do it or your users are compertant. Any more than that your better off going with something like python or perl, they are widely known, and have rich libraries to let the people using the app do some really imaginative things.
The other thing that you can do is write your app in such a way that it is easy for scripting langs to get there hooks into. For example, on windows you can do this by providing com interfaces to all the things you want to have control over, then you can use any lang you want to control your app.
i recommend you look at paxscript.
it's the most complete and quickest of any embedded source stuff that we've investigated.
supports basic, c, objectpascal (ie delphi) and javascript.
not free, but you get the source when you purchase it (US$179).
delphi and borland c++ builder only.
nostrils
didn't see anyone else mention it, so i thought i would.
REBOL is a ridiculously simple and easy-to-learn web-oriented language. so easy, in fact, that i wrote a fully-functional IRC dice bot in under 400 lines, overnight. and if that weren't impressive, then might i add that i was running on ZERO caffeine, learning the language for the first time, learning IRC Client protocol for the first time, and came up with a few unusually witty statements and insults to boot?
now the bad part: REBOL is not open source. poo. (i really was a bit disappointed.) but REBOL/Core is free (for any use, i gather), and the license fees for View and Command seem rather reasonable.
the nice part: it has been ported to and runs on about 43 platforms, last i checked, and is light enough that the executable weighs in at around 250kb for the win32 release. (haven't used the other platforms, so no comment.)
it runs on just about every unix i've heard of, on every relatively common configuration, and works beautifully and seamlessly. and, after a quick glance, i see it runs on serveral major embedded systems, including WinCE, QNX, and Linux, and will even run on my friend's dated Amiga.
enjoy and happy coding.grey wolf
LET FORTRAN DIE!
TinyPerl
Error: PANTS NOT FOUND. Press <F1> to continue.
ficl.sourceforge.net
A portable object oriented forth at less than 100K for a full implementation, able to handle calls from your apps as well as call your apps fgunctions or other dll's.
Under a BSD-ish license.
The upside:
1) Its tiny
2) The forth code you write will be tinier
Downsides:
1) You need to learn forth (I mean properly) to appreciate it.
Forth is unlike most other languages, it was designed to avoid the debug, recompile cycle that is so common.
Sam
blog.sam.liddicott.com
Assuming you already know at least one other language, learning LUA takes exactly 1 hour. That's it. Spend an hour on it, and you are a master LUA programmer. That's a big part of its appeal.
I've been working on a system for embedding JS in KDE apps (amongst other reasons it is a about 1/10 the size of a python interpreter). You can find lots of information at http://xmelegance.org/kjsembed. The interpreter in KDE has no dependency on KDE/Qt or anything else so you might find it usable in your app.
I guess atai must have submitted the same question here as well, but the "original" discussion took place on advogato:
http://www.advogato.org/article/735.html
It would be useful if he could state what his requirements and limits are, in detail, because that's a very necessary part of evaluating what would work.
http://www.welton.it/davidw/
And btw, ECMAScript/JavaScript can be pure functional as well.
Or, how about EiC, now hosted at Sourceforge. "EiC is pointer safe."
What about Ruby (ruby-lang.org). It's easy to learn, powerful and has a reasonable set of classes and methods built-in. It's also quiet widespread nowadays, lightweight compared to Python and easy to embed via libruby.so
see the Ruby book (section Embedding a Ruby Interpreter
I think you misunderstand.
say I have a program, written in C, and I want to add full python scripting support.
One way would be to try to add in the python interpreter into the program, and this is the way you immediately think of.
The second way would be to have a small python program that calls your program (so all you rewrite in python is your main() function and the function that does event handling.)
The link I provided gives reasons on why they consider the second way better.
Nasal is a very well designed small embeddable language with a Javascript-like syntax and an LGPL license. I prefer it to Lua because I am not fond of Lua's Pascal-style syntax. Be aware that Nasal is not very battle-hardened yet and does not have even a 1/100th of the developer community of Lua.
lua is a great scripting language when it's used well. When i was studying game development in school, we were introduced to it as a good way to drive out games. Its easy to add functionality for it... its fast... it simple. I could write all my rendering/sound/network/input code in c++, and just have interfaces for everything from lua... this way, when i wanted to drive a game app, i could do the following: (pseudo lua... hmm... pseudo scripting :) )
load level
load char at xyz
load enemy at xyz
go
then, my c++ code could handle all the collision detection/AI/proper sounds for things... it all comes down to good design. Something like lua is great for having somethign high level drive a low level engine.
Wouldnt you like to be a pepper too?
When I was answering this question, I ended up looking at nasal. While not perfect, it was the closest to what I wanted ... and was under a compatible license (LGPL). But then I wanted something small, lua is way too big.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
If you accept the notions that
1. scripting languages speed development
2. high-level code does not need to be optimized
Then for an app like Neverwinter Nights, it would make sense to use a scripting language for the high-level code (e.g. menu handling code), while leaving the performance-critical code (e.g. 3-D rendering) in C, C++ or assembler.
Coded like this, NN might not crash as frequently because either
- The crashy code would be implemented in a scripting language, and so memory handling would not be a problem
- The crashy code would still be in C/C++, but since the other code had been written more quickly, there would be more time to fix it.
The current issue of the C/C++ Users Journal is all about mixed-language development.
I'm not talking about Python. Python is positioned as a scripting language, and hence doesn't have a powerful compilation infrastructure. Although, Pysco does some very cool dynamic optimizations for Python.
In contrast, there are many high-level languages like Common Lisp, Dylan, Scheme, ML, Ocaml, etc, that have very powerful native compilers. They do optimizations that C/C++ compilers simply cannot do, because of the low-level C memory model. Literally decades of research has gone into making these compilers, and they have optimizations that (while not quite magical) are very impressive.
Variously:
- There are type-inference optimizations that eliminate the overhead of dynamic dispatch.
- There are heap-analysis optimizations that stack-allocate objects whenever possible, to avoid heap allocation.
- There are analysis that avoid heap-allocating closures.
- There are analysis that eliminate type checking and array bounds checking.
- There are analysis that perform large-scale optimization of class heirarchies, to eliminate the over head of OOP.
- There are memory allocation analysis that reduce the overhead of garbage collection (region inference).
- They do method specialization, allowing the C++ template advantage of generic functions optimized for a given type, without actually having to deal with explicit type parameters.
Some useful pointers:
Apple Dylan Wiki
Lisp vs Java vs C/C++ performane
Bigloo Scheme Compiler
Gwydion Dylan compiler
CMU Common Lisp Compiler
UW Vortex Compiler
MLKit ML Compiler
Ocaml Compiler
A deep unwavering belief is a sure sign you're missing something...
The main reason why applications are developed in C is cross platform compatibility. That said, scripting languages are getting better about being everywhere--but they can never be everywhere where you might need a truly portable application (embedded platforms and wacky operating systems come to mind).
I agree that writing your basic GUI applet in C is probably counterproductive; scripting languages can certainly be much more portable than C, as they hide a lot of the messy machine internals. However, for core infrastructure like server daemons, core utilities like grep, and other system-level programming tasks should be done in a portable language like C, a compiler for which is one of the first tasks facing the designer of a new architecture (after the core operating system and a few basic libraries). Interpreters like Python and the rest, after all, are written in C.
Also, C is the de facto language for library bindings and all the rest--you can be pretty much guarantee being able to link a library into any other language if it exposes a C API. The same wouldn't be true for a Python or Perl module, C++ library, or Java class file.
Finally, C gives you much more control than higher-level languages. That can be both a blessing and a curse, but at least it means you don't need to hack the C compiler to get a feature that you want. In many corner cases, you'll need an extra C function or two to implement lower level functionality. One of the strengths of Python (the main strength, in my POV) is that it makes it easy to script C-based programs and libraries.
COM/ActiveX works swimmingly in this regard in that you have a choice of languages for both producing and consuming COM/ActiveX components, and COM allows you to pass object references across the barrier while ActiveX allows you to work with GUI widgets in this way. And it is true multi-language, not the fake multi-language of the .NET languages.
On the other hand, COM/ActiveX pretty much limits you to Windows, you have to be careful of the lowest-common-denominator effect of which COM data types work in which languages (the more restrictive set of "Automation" types compared to the wider set of C-compatible types and a more restrictive situation for scripting languages and Web browsers than with Visual Basic 6). Also, COM/ActiveX can get ugly in some environments (C++ and MFC/ATL -- you are really stuck taking the Wizard-generated code at face value unless you really know what you are doing), and as a component developer you have to pick up some IDL basics.
Is there anything out there that even comes close? With the Python/C-C++ meld, you are pretty much restricted to Python on one end and C++ on the other hand, and the C++ end is, shall we say, a bit lower level than the Python end, although there are things like SWIG that look amazingly like, hey, IDL and Visual Studio Wizard code.
Another thing I have looked into is the Matlab-Java meld -- a lot of academic engineer and scientists use Matlab as their all-purpose scripting tool, but I wouldn't want to develop high power apps in it. The Matlab-Java meld is surprisingly clean, the Java side can be a Swing app and it runs in its own Swing windows. Again, you are restricted to Matlab on the scripting side, Java on the custom app side, but the interesting thing is that it is pointedly the sort of thing SUN isn't interested in. According to the SUN world view, Java is everything you would ever need and the idea of scripting Java from anything else doesn't enter into their universe.
Is there anything else out there that scripts Java? I suppose there is Jython. Anything else?
"It's a good rant but it's not really very relevent - writing your app in python (at least in the way he describes) simply isn't plausible if you're writing an app for commercial or even mass distribution - you can't rely on an existing Python installation and if you've got to package one with your installer then theres hardly any more work to embed an interperter instead."
This is old enough it will probably never be read, but that isn't true. There are utilites that do this in a snap, in addition to finding any extension package or dll deps that your python is using automagically in almost all cases. Just today I took a small app written in python using WxWindows ran it through py2exe and it runs on all flavors of windows, without python or wxWindows libs installed on either one. Took me about ten minutes.
McMillian Installer will do the same trick, tho I haven't used it recently. Mcmillan also does the same thing for Linux, where I wish a lot more people were making packages available as no dep binaries as well as more traditional source packages, but that is just me.
Now, as for your other points, if you happen to want to write your code in Python, well, that is a different matter. But if you choose to, it is very easy and fast to distibute your program binary without deps.
7. What we cannot speak about we must pass over in silence.
Ferite is a scripting language and engine all in one managable chunk. It is designed to be easily extended in terms of API, and to be used within other applications making them more configurable and useful to the end user. It has a syntax similiar to a number of other langauges but remains clean and it's own language.
I love that it was designed from the Start to be Thread Safe and Embeded, unlike PHP or Python.