Slashdot Mirror
The State of the Scripting Universe
r.jimenezz writes "Via PragDave's blog I learned of an article by Lynn Greiner on the state of scripting languages, a.k.a. scripting dynamic languages. A number of influential personalities (Guido von Rossum, Damien Conway, PragDave himself and others) were interviewed and it's interesting to see how much their opinions coincide despite being interviewed separately. A lengthy but worthwhile reading."
The research firm reports that over 41 percent of the 666 developers surveyed use Perl, 32 percent use PHP, and 15.6 percent use Python
The Number of the Devil of Programmers.. looks like it's as I always said, scripting languages are evil..
"Victory means exit strategy, and it's important for the President to explain to us what the exit strategy is." G.W.Bush
I had to refresh a few times and still nothing! Then I tried submitting this and got a 500 server error and thought "Ah, now I understand!".
I have often wondered what makes a particular language a "scripting" language v.s. a "real" language. One thing scripting languages all have in common is that they are interpreted. However Basic is interpreted (even though some may dissagree that it is a "real" language), and there have been interpreted versions of Pascal and others.
The best I can come up with is that a scripting language is an interpreted language that is normally attached to an application in order to allow the end user to automate functions within that app. So for example BASH programming is "scripting" since most of what you do is items that would normally be done at the command line. Same with Word/Excel macros. Also, TCL was designed to be attached to other programs (although the typical use is stand-alone). But I wouldn't put PERL in that catagory, anymore than GW-Basic or interpreted Pascal are scripting languages.
So, what does a language need to be called a "real" language, other than being compiled?
Scripting languages will always have a place , Its a simple case of the right tool for the job. .If i do have a larger project that requires a bit more power then i can prototype it in perl or shell then easily transcribe it to C .
As a sysadmin most of my Programming is done in either ruby(im difrent) , shell(bash or ksh) or perl. Using C for most of the things i have to automate is as i said "Taking a sledge hammer to a penut" , It would be total overkill and waste my time and the systems and it would be more error prone
Scripting will always be an important part of my work , and with the increasing power and decresing price of computer equipment over the last few years , it has become viable to program many many things in languages such as python , things which would not so long ago, would have requierd the power of C/++.
Scripting is in a fine state , and will continue to fill its end of the market , just as compiled languages will always have their place
The only things certain in war are Propaganda and Death. You can never be sure which is which though
This article:
0 0.asp
http://www.devsource.com/article2/0,1759,1778148,
talks about some of the cool stuff that Tcl does. My favorite thing about the language is that it hits a real sweet spot with its level of abstraction. Python has an event loop now, in Twisted, but Tcl's had the same thing for ages, and it's very easy to use.
http://www.welton.it/davidw/
For projects below a certain complexity level, I've always found scripting languages to be more productive for the programmer (very fast edit-run-debug cycles) even if they are less efficient for the CPU. And with the advent of JIT compilers, the efficiency of scripting is much improved.
Given the low-cost of fast CPUs and high-cost of programmers, I'd rather waste CPU cycles than waste developer labor.
Two wrongs don't make a right, but three lefts do.
I have a final project in my computer engineering diploma in which we are trying to monitor and control a few security devices over the web. The focus is about split equally between design of a PCB w/ microcontroller and getting the code written. Since my only formal training was in C, and at first I considered using C for the majority of the programming. However after some research I stumbled upon scripting languages.
The result is that instead of a cumbersome 1000 line C program which would be a huge time sink, I have several small C programs connected with script. I'm using a combination of Bash script, PERL, and Java. I had to learn the basics of each but in the end that was faster and easy to debug. The time I saved went into more hardware hacking and making sure that the electronics portion worked better. In the end it made for a project that was much more fun.
Incase anyone missed the joke, its a parody and a very funny one
Mike Roca--
I don't think it's necessarily a question of project size, though, so much as the nature of a project. I have worked on some very large tools for which a scripting language is a much better choice. As far as I am concerned, C is best used when you're working with very large and rapidly changing data sets (C structs will probably always be lighter weight than objects) or you need to crunch a lot of numbers really fast.
I think the way of the future is static languages like C for critical stuff, and dynamic languages like Ruby, Io, or some functional language for most of the program. . . much the way that programs used to be written in C with some inline assembler for the performance-critical stuff.
I wish i could find a good scripting language like python and perl where the interpreter (and standard libraries) are distributed as a single executable file. CGIpython comes very close, but unfortunately it doesnt work in interactive mode (at least not the windows version). Oh well....
I agree totaly, me just rushing my posts there , I really should of used better wording.This is exactly when i use C ,Interpreted languages will never compete on data crunching tasks(though the line is blurring)-
The only things certain in war are Propaganda and Death. You can never be sure which is which though
Those basterds ignored us again!! (no I didn't seriously expect to be noticed).
Philosophy.
"Flamebait"? As I said here, Slashdoters (or maybe just those with "mod" points) are humor impaired.
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
Yes, but BASH is a manly scripting language, where as...
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
Instead of a single 1000 line C program, you have a basket of C programs and a basket of programs in different "scripting" languages... And this is better than a single C program?
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
One important advantage of statically typed language is the ability to do code insight... I mean the little pull down menu which pops up as soon as you hit the dot character after a variable name, displaying a list of available methods. This saves you about 80% of the time spent on browsing documentation. As a side effect, it prevents typing errors.
Also, JBuilder for example supports incremental compiling (i.e. compiling while you type), which immediately detects practically all syntactic errors during input. This means you end up nearly always with a syntactically correct program before you even think of compiling... the problem with scripting languages is that syntax errors (e.g. illegal method calls) only crop up at runtime.
Lets say you have to added a feature to the following function:
updateCustomerAccount( customer, newInfo )
Looks nice, with very descriptive function name and parameters. But it isn't. Is customer a string name, a an accountId, or a data structure? Does it return anything (success or failure code)? And a little more debatable, does it throw any errors.
The usual cases are:
But even if you can always rely on one of these solutions, you're missing out on the beauty of statically typed languages: IDEs. By formally typing objects in the language grammar, you have not just documented your code to your peers, but to the machine itself. And allowing the machine to reason about your code means it can help you write it. It keeps you in the code, with instant access to API, documentation, and refactoring. Now you have mentally stepped beyond the code into the problem space.
Anyone who has written Java in a solid IDE like Eclipse can atest to this. I've had I've had people look over my shoulder in amazement as "the computer writes [my] code" for me. Every function appropriate to the context (and their documentation) usually within 6 keystrokes, without memorization.
Even the best Python IDEs (strongly typed, but still dynamically typed) pale in comparison to this experience.
When will the scripting world learn what they're missing?
Anm
(For the record, my prefered langauge is Java, but my current work has me doing C, C++, Python, and Perl as necessary. While I haven't worked in it, I assume C# is in the same boat as Java: statically typed, runtime reflexective, dynamically loadable, useful exceptions/stack traces, and large library including the source compiler itself.)
(emphasis mine)
His point about all languages being interpreted at some point doesn't make sense to me, even with the parenthetical note about processor architecture. What about CPU architecture these days makes a language interpreted? Would this include ASM then (which is really just a human-readable version of machine code)?
I thought compiled languages compile once to, you know, machine code prior to execution, whereas interpreted languages compile code on-the-fly for every program execution? Yet Thomas is suggesting that if I write "hello world" in C, the machine code compiled from that will in reality be interpreted by the CPU?
My CS profs didn't cover this!
Is Capitalism Good for the Poor?
Ruby has RubyScript2Exe to collect a script and all dependencies includint the Ruby interpreter into a single Windows .exe file.
.exe.
AllInOneRuby uses the same technique to package the Ruby interpreter and core and standard libraries into a single
I'm not sure if Perl and Python have something similar.
I agree with the right tool for the job sentiment. I'm familiar with Bash, Python, Tcl and Ruby. I initially looked at doing something with Ruby, but the RSS module tutorial was somewhat confusing. I ended up with the beautifully simple feedparser module for Python, and got the job done very quickly.
If I'd been somewhat biased, I could've spent a lot of time banging my head trying to figure out how to use the Ruby version, but instead, in this instance, went to something that I found simpler and quicker to get into.
Man watching 6 MSCE's around a sun box, looks alot like the opening scene's of 2001:space odyssey...
I really should of used better wording.
"should have used".
yes that was the joke
The only things certain in war are Propaganda and Death. You can never be sure which is which though
This is more of a nit pick, but I can think of a few interpreted languages geared at serious crunching.
APL -- serious number crunching is the only reason people use it.
PIK (Ok, it's a whole environtment, not just an interpreter) -- Lots of record crunching gets done by PIK systems.
IMHO, all languages should be 'scripted' as well as 'compiled'. The same language syntax can be both. Why not use our advanced computational tools to do the grunt work of optimizing functionality? We've automated every other engineering discipline, but can't seem to do that for our own. (I know I'm overstating the argument, handwaving and idealizing - it's a dirty job, but somebody's gotta do it.)
What I'd like to see: As I enter code in the language, the system does syntax checking and builds the necessary data structures, prompting and possible asking questions about ambiguities and any necessary library functions either not found or not called correctly. In fact, this process might as well be graphical in nature - NeXTstep's Interface Builder perhaps came closest to this in my experience. The code should be runnable immediately, in an interpreted environment - a la Perl/Python/PHP etc.
Thus language processing system would act as an intelligent assistant, looking over the system as I design and build it and making suggestions. It might even make performance predictions based on its analysis of the code-so-far.
Once the functionality is complete, I can pass the system to an even smarter optimizing compiler function, which does the necessary things to make it efficient, small, fast, etc. In the process the compiler may ask more questions about how to resolve various issues that didn't come up earlier. Since the functionality is already defined, the compilation has a mostly-known target for functionality. This process can take as long as necessary, perhaps while I go home for dinner; using more heuristics, AI, whatever, to further improve and clean up the system for the defined functionality.
Except for the AI features, this combined capability is actually old. Back in the Long Ago, Harris computers ran the Vulcan OS, which had a n interactive FORTRAN system. It did line-by-line compiling, doing syntax checking on each line and even running the code. Once your program was complete and ran, the interactive compiler could output correctly-formatted FORTRAN that could be compiled using the standard FORTRAN compiler.
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
As an off-topic aside, I'd like to learn APL just for the hell of it...any pointers to good sites with tutorials, or good books?
C structs will probably always be lighter weight than objects
In C++, if a class has no virtual functions, there's no reason it can't have exactly the same memory footprint as a C POD struct. Member function calls should similarly incur no additional overhead vs. a C function call. That's the underlying philosophy behind C++ with regard to C - you only incur penalties for things (like virtual function indirection, exceptions, etc.) if you actually use them in your code.
Downmodding is the refuge of the weak. Don't downmod, make a better argument!
www.lua.org
It is designed to be linked into other programs. It has specific features for dealing with C/C++. It's also fun to use. The definitive book about Lua is avaiable on line. What's not to like?
I would suggest learning J instead. It's essentially APL, but replaces all of APL's special characters with ASCII characters.
Modern Perl is probably not a scripting language, but I'm not familiar enough with it to say so for sure.
However, the last time that I gave Perl more than a cursory look (in the last millenium), it was using UNIX commands (executables) to implement many of its operators.
For example, I think that it used expr to do RE matching, mv to rename files, etc.
Perl hid the use of (many of) these commands from the user.
That's how I came up with the "in between" comment: It used many system commands, but it wasn't (very) obvious to the casual user.
Modern Perl (version 4? version 5?) probably doesn't use executables any more than any other language (although I don't know that for sure), so I guess that I wouldn't consider it to be a scripting language at all.
How's that for decisive commentary?
Those who sacrifice security to condemn liberty deserve to repeat history or something. - Benjamin Santayana
Smalltalk, Forth, Haskell...
bar = map( lambda x: x[1], foo )
Maybe it's possible to do that with static typing, but if so it would add so much grief in making sure all the type related stuff fits together as to be barely worth doing. And it is worth doing.
I'm not sure why you think static typing implies grief for this kind of code. Now I'm not sure, as I've only typed about 10 lines of Python in my life, but it looks like you're mapping a list of arrays to a list of the second elements of those arrays. Here's the equivalent code in OCaml:Being a bit more general, we can write:To which OCaml assigns the type:That is, firsts takes a list of arrays of some unknown type 'a, and maps it into a list of elements of type 'a. No grief, plenty of static typing... Issues like this have nothing to do with static v. dynamic typing.
-30-
Wonderful, thanks very much.
that being said, i love perl for quick stuff, i/o and text processing, and lots of other things. php is great for web and i am definitely interested in looking into python...
Lisp started out like a dynamic language -- interpreted, no data typing, everything is a list -- but it evolved to having a compiler, having arrays as well as lists, and way to specify data type or for the compiler to make a data type inference from the context. When it acquired those features, people could make the statement that "Lisp is as fast as C", an arguable point now but not even arguable back in the early days.
The parent post speaks to reading and understanding code with and without static data types, and I suppose data type along with (the horror, the horror) Hungarian notation have their pros and cons with regard to reading and maintaining code. But there is also a use for static data types in getting efficiency.
A dynamic data type means that data needs tags on it to determine the type and the meaning of operations performed on the data, and testing the tags means slower execution. Doesn't matter too much on how these tests are implemented -- test and branch, function pointers in a virtual method table -- you still have some form of tags and some form of tests. People will say "what about Moore's Law" and I will say "yes, what about it?" Moore's Law talks about growth in gates apart from speed, but the gates made available by Moore's Law have been used effectively to speed up programs based on static data types. Not only do static-data programs keep getting executed faster, the speedup of static-typed over dynamic-typed programs also keeps increasing. Yes, dynamic-typed programs execute faster on new hardware, but not anywhere near the speedup afforded to static-typed programs, and the gap keeps getting wider. It does not matter that a Python program is 100 times slower than a C program -- we are told to wait X years until hardware gets 100 times faster -- 100 times faster for a C program, but maybe only 10 times faster for a Python program, and we still have a factor of 10 shortfall.
Why is this? The current generation of processors utilize various forms of pipeline parallism, which benefits from tests and branches being sparse rather than at every instruction. With static typing, not only do we lose the overhead of the tests, the absence of the tests means that we can keep our pipelines full.
A static data type, whether spelled out with a type declaration or "discovered" by the compiler is a big efficiency win. An array type (basically a static list) is also a big help for efficiency, especially with the lists making up waveform samples for audio or bitmap images for raster displays.
All data does not have to be static-types, but if you have the provision for static scalar and array types along with sections of code that operates on those types with arithmetic ops, assignments, and indexed loops, you can translate those operations into fast-running code. You can even have array bounds checking of the looping statements are such that the bounds checks can be done on the first and last index values without having to do tests on every index in between. Things like Numeric or NumArray in Python don't count because not only are the implemented as C extension modules, you have to write your algorithms that do looping on those structures as C extension modules as well or give up on efficiency.
I can write a C extension module, you say. Back in the day of Fortran or even Turbo Pascal, one could write inline assembly language for speed-critical parts, but when optimizing compilers came along we gladly said goodbye to such tweaking and patching.
Once a dynamic language gets a way of specifying static-typed scalars and arrays, there is nothing standing in the way of optimization to achieve performance in the ballpark of C for things like audio, images, and looped numerical algorithms, all without C-language extensions, and at that point one will have duplicated what distinguished Common Lisp and other Lisp dialects of that generation from first-generation Lisp.
Sometime I'll take another look at Squeak, which seems to be the anointed successor to Smalltalk. I have a friend who's been a total fanatic about Smalltalk for many years - it has affected his career in interesting ways. A few months ago he got me interested in its descendant Squeak, because of Alan Croquet 3D system written in Squeak. So far, I've just bounced off when I've tried to do anything with it. I've been doing procedural programming for a long time. I think it was Dykstra who said (approximately), "Anyone who is proficient in FORTRAN can quickly learn to write FORTRAN in any language." I may try again soonish, although it's not
:O) I managed to get Croquet running on my system, but it was so slow (I'm running at 650 MHz) that it was unusable, and I diskliked the little buttons around the edge. Tthey seemed counterintuitive to me, but of course I grew up in a different UI paradign so who am I to say.
This note speaks to my interest - IMHO the next, and most interesting, phase of UI design will be in the use of 3D to create not just usable but "amenable" user interfaces.
For me, that will have to wait until I can get a better machine, preferably one I can use with polarized 3D glasses!!
One problem I didn't have time to figure out at the time was that Croquet's OpenGL driver setup was hardwired into the code, and didn't accept my OpenGL setup (I'm running DRI). But that's another story.
It's easier to be a result of the past, but more fun to be a cause of the future! http://www.spacefinancegroup.com/
Why don't we write the whole thing in C? Well, the shell allows us to write quickly and hack around while the C approach has an edit-compile-link-test loop. Why don't we write the whole thing in shell or Perl or whatever? Because we need to optimize the slow parts.
So a given application has high-level parts that benefit from the flexibility of dynamic data types, access to a library of functions, running a line at a time to see what happens. It also has low-level parts to express looping algorithms in an efficient manner.
The big question is not whether you separate your application into high and low level parts; it is whether the high and level level parts are all written in the same language or in two languages.
The one-language approach has the advantage of having to be skilled in only one language and of easier piecing together of the high and low level parts. The one-language approach as the disadvantage that unless your OS is set up for that one language, there will always be something, some OS call or feature that will require interfacing to a C-style function.
The two-language approach has the advantage of a high-level language tailored for the high level, a low-level language tailored for the low level. The two-language approach can have the problem of impedance mismatch between the two languages, requiring a lot of adaptor plumbing, especially at the low-level language end.
The adapter plumbing in Unix consists of function calls to C-style plumbing along with pipes, accomplished with some more function calls. The adapter plumbing in Windows is COM, which is straightforward enough until you get into the full complexity of the barely-documented raft of interfaces required to implement an ActiveX control along with the ham-handed Microsoft approach to C++, where they throw classes, macros, templates, and machine-generated code at the problem, all at once. Python with its dictionary-dispatch of object methods is somewhere in the middle.
I think that the people doing a given dynamic language need to make a choice. If they are committed to the two-language approach, they need to work on having a streamlined approach to low-level extensions. There is a tendency to wall off the low level extensions into libraries and tell people "here are the (numeric, graphics, audio) libraries) -- they do everything you need and you don't need to write your own extensions, but if you need to write your own extensions, here are the C header files, and good luck." Well, the library-coded extensions cover a lot of cases, but sooner or later you are going to need to write extensions, and writing C extensions is often harder than writing straight C programs on account of the impedance mismatch of object models and data types between the high and low levels.
If you are going to the "One language to rule them all" approach, that language is going to have to be efficient on the low level in the manner of Common Lisp or related high-level one-language approaches.