State of the Onion 11

chromatic writes "Larry Wall's State of the Onion 11 address is now online. Every year, he describes the state of Perl and its community through metaphor and analogy. This year, Larry explored the history of scripting languages, from their dimly-lit beginnings to their glorious future. Along the way, he also describes several of the design principles invoked in the design of Perl 6. 'When I was a RSTS programmer on a PDP-11, I certainly treated BASIC as a scripting language, at least in terms of rapid prototyping and process control. I'm sure it warped my brain forever. Perl's statement modifiers are straight out of BASIC/PLUS. It even had some cute sigils on the ends of its variables to distinguish string and integer from floating point. But you could do extreme programming. In fact, I had a college buddy I did pair programming with. We took a compiler writing class together and studied all that fancy stuff from the dragon book.'"

Comments

scripting

[Lord Ender]

I really wish the term "scripting language" would die. Can't we just call them "very high level" languages, instead? Isn't that sufficient to distinguish the Perls, Pythons, and Rubys of the world from the "high level" languages like C and C++? It is perfectly possible to compile Python programs, for example, to a pyc binary. They aren't any more "scripts" than a.out. The difference between "very high" and "high," to me, is the fact that dynamic datastructures (lists, hashes) are native, so programmers don't have to worry about mundane memory address and pointer nonsense.

Re:scripting

[Ed Avis]

Python, Perl, Lua etc. are very high level languages, at least from a slightly old-fashioned perspective that regards assembler as the baseline and something like C or Pascal as high level. But there are other very high level languages which are not scripting languages, for example Haskell, Erlang, Prolog, C++ with insane template nonsense, even SQL and XSLT. So the term scripting language is still useful I think. The definition is a bit fuzzy, which is what Larry points out in his talk.

Re:scripting

[Anonymous Coward]

Agreed. I code almost exclusively in Python at work (I say "almost", because I dabble in bash), and I'm so damn glad I don't have to deal with memory allocation nonsense like in C and C++. Even Java gets on my nerves with its static typing.

Languages like Python make it easier to design and implement algorithms without having to worry about other concerns.

Re:scripting

[Yvanhoe]

Perls, Pythons, and Rubys are interpreted languages as opposed to compiled languages. A .pyc is not an executable file.
Every people use the term differently. Here is mine : I am doing a script when I give directives to launch programs or functions written in another language. When the CPU spends 90% of its time outside my program, I consider that this is a script.
Python's philosophy is that it is a scripting language in the sense that if you spend more than 10% of your CPU time interpreting some python code, you are doing something wrong. They encourage you to rewrite these image processing functions in C/C++ if you are serious about performances.

And yes, given this definition, one can write scripts in C.

Re:scripting

[Reality Master 101]

It doesn't. It does, however, require a 32 bit integer data type (which the OP said). It doesn't. It does, however, require a 32 bit integer data type (which the OP said).

Close, but not correct. C requires *at least* 16 bits for a short, and *at least* 32 bits for a long. It actually doesn't require an exactly 32 bit integer datatype. Well, to be really pedantic, the C Standard specifies a range of values that a datatype must support, so technically a binary machine is not required.

Re:scripting

[Chandon Seldon]

Hand compiling is only moderately more complicated for Perl than it is for C. Just as the C function turns into a specific pattern in assembler, the various things that Perl provides can be turned into assembler patterns too. The stuff that Perl does for memory management implicitly is only moderately more complex than the stack manipulation that C does implicitly for lexical variables. Even the two data structures that Perl provides - vectors and hashes - are just very simple data structures that would always translate into the same simple pattern of assembler.

Basically the only thing that cleanly differentiates a language like Perl from a language like C is the "eval" function - which requires access to a Perl interpreter/compiler to work.

Re:scripting

[anomalous cohort]

To me, the term scripting has nothing to do with the level of abstraction. Rather, it is more about whether or not the source is directly interpreted or compiled into an intermediate form. Having said that, I must admit that it would be awfully silly to create a low level language that is interpreted.

From most of the languages that he lists, it looks to me that what he is really talking about are Dynamic Scripting Languages. Scripting languages that are also dynamic provide a lot of flexibility and can genuinely be categorized as different from static languages in an interesting way. I have blogged on this recently as part of a larger article on OSS.

Re:scripting

[Ed Avis]

I must admit that it would be awfully silly to create a low level language that is interpreted.

Not at all. Open Firmware (used in the OLPC among other devices) is built around a Forth interpreter which can be used to write device drivers and hardware initialization code. Pascal was originally implemented as a p-code interpreter. And so on.

Re:scripting

[chromatic]

I have not used Perl for a while but when I did I would have called it a scripting language. It just didn't lend it's self to large programs.

Why not? I've used it for large programs that are still in place. You're reading the output from one of them right now.

(You won't hear me arguing that Slash is and has always been a good example of maintainable Perl code, but you won't hear Cmdr Taco arguing that he's a good example of a programmer either.)

Perl 6: The Language of the Future (... Forever)

[joe_n_bloe]

Every year Larry talks about what interesting things have been going on with Perl 6. These interesting things never include "release."

Re:Perl 6: The Language of the Future (... Forever

[Otter]

Every year Larry talks about what interesting things have been going on with Perl 6.

And this year he barely even did that!

Fine by me

[winkydink]

Perl 5.8 does everything I need it to. There are other languages I use when Perl doesn't do what I need it to. I don't need one language for all of my needs.

I view Perl 6 as an continued employment mechanism for those who write books about Perl and teach Perl to others.

Re:Fine by me

[bzipitidoo]

I'm not a Perl wizard, but I have used it. I find a lot more limitations than just slowness.

Calling on external C functions from Perl is a pain. You can't ignore all those C functions, there's way too many useful ones out there. There's UNIX system calls, XWindows and higher level GUI libraries, socket and networking stuff, file manipulation, and everything in libc. Perl has a good bit of that covered natively, but suppose you want to use OpenGL? Then you're stuck trying to figure out Frozen Bubble for example. Or you could do it yourself and grind out the XS manually, or fix some automatically generated XS. If you're lucky, someone else has already put together a module, and that module works. Or you say the heck with it and just use C.

Maybe I'm not being flexible, but I found Perl's OOP confusing. The return from a function could be anything from a simple integer to an object that's an instance of a class in a very big ugly hierarchy, and you can't tell what class it is. Lazy undisciplined programmers can very quickly make a horrendous mess. What, some class doesn't have a function called "draw" because it didn't inherit from the correct branch? Just add one, and who cares about keeping the hierarchy sane! I've heard Perl called a "write only language".

Re:Perl 6: The Language of the Future (... Forever

[rubycodez]

and every year the design for Perl 6 becomes more and more contorted and ultra-complicated, basically taking every cool feature Larry sees in other languages and mashing them together into an incoherent Mulligan stew. If Perl is like "whale guts everywhere" then Perl 6 is like taking a whole Oceanarium of sea creatures and dropping them through the dual rotors of a crane copter 10,000 feet over Manhattan.

Yup... and he doesn't apologize for it

[krog]

The Perl 6 community is more focused on getting it right than getting it into the marketplace ASAP. While this is frustrating for many -- it seems like every other day, there's a Perl 6 feature I want to use in my Perl 5 program -- it has contributed positively to the development of Perl 5 and Perl 6 alike. Perl 6 is painstakingly designed and planned as truly a next-generation dynamic language, and as features are completed, they often spill down into Perl 5. (See the perldelta for v5.10, out very soon, for some good examples.)

Unless someone is willing to finance full-time development on Perl 6, this is the best we get. I think it's pretty good.

Re:Yup... and he doesn't apologize for it

[rubycodez]

getting it right???? Perl 6 is a pathologically overcomplicated mess of anything and everything Larry saw in other better designed languages over the last decade, all mixed up with no rhyme or reason. It's like a swiss army knife with condiment and shaving cream dispenser, cell phone and vibrating butt plug. The specs are a nightmare to read, the syntax takes 25% more effort to construct something compared to other scripting languages, and there's no end in sight. Time to realize Perl 6 is like the Ripley-Alien clone freaks, only merciful thing would be to hit it with a flamethrower and start over.

Re:Yup... and he doesn't apologize for it

[Nimey]

So it's the computer-language equivalent of English?

Re:Yup... and he doesn't apologize for it

[theskipper]

I once made a bet with a friend that some day we would see the terms "Perl 6" and "vibrating butt plug" in the same sentence.

Kicking myself for not saying paragraph instead of sentence.

In other news...

[nycguy]

Duke Nukem Forever team announces that they are reimplementing everything in Perl 6.

Re:In other news...

[Ignominious]

FTA:

"Then there's Duke Nukem Forever, a nice clean design. It has some issues, but in the long run Duke Nukem Forever might actually turn out to be a decent platform for running Perl 6 on. Pugs already has part of a backend for Duke Nukem Forever, though sadly that has suffered some bitrot in the last year. I think when the new Duke Nukem Forever engines come out we'll probably see renewed interest in a Duke Nukem Forever backend."

Thanks, and see ya!

[stuntpope]

Thank you, Mr. Wall, for providing a language that caught my interest and led me into a profitable career.

However, I moved on several years ago. One of those Python guys inspired negatively by Perl. Much of what keeps me away from Ruby, in fact, is the Perl resemblance. I still have a legacy Perl application to maintain, but I don't do any new Perl work.

I'd think a regular "State of the Onion" pronouncement would be an avenue to discuss where we are today, and where we are headed, with Perl. Instead, it's a rambling short history of "scripting" languages, and a rundown of various language design choices with "Perl 6 will have [x]" statements.

I guess I really don't get the purpose of the essay.

As to TMTOWTDI, I've concluded TOABWTDI (There's Often A Better Way To Do It).

Re:Thanks, and see ya!

[kisrael]

I'd think a regular "State of the Onion" pronouncement would be an avenue to discuss where we are today, and where we are headed, with Perl. Instead, it's a rambling short history of "scripting" languages, and a rundown of various language design choices with "Perl 6 will have [x]" statements.

I guess I really don't get the purpose of the essay. I think the purpose of the essay might be to be a rambling short history of "scripting" languages, and a rundown of various language design choices with "Perl 6 will have [x]" statements.

It was also (IMO) a damn fine read, with lots of intriguing rhetorical flourishes (I also learned a little C. [...] That's because a little C is all there is) and thought-provoking concepts, like how most human languages can express anything, but they differ in what you MUST express.

I think most people have a rough idea where Perl is now (present, though likely slipping as a % of interesting code being written) and where it's going (a guess about how the new perl 6 would be received when it finally shows up)

As to TMTOWTDI, I've concluded TOABWTDI (There's Often A Better Way To Do It). Better than Perl, or in general?
If the latter, well sure... there will almost be another way that is better in some subset of the parameters you could use to measure "Betterness". One tradeoff you always have to make is how much time and conceptual effort do you put into optimizing that...

BASIC/PLUS

[Sloppy]

Oh wow, BASIC/PLUS on a PDP-11 running RSTS. That's how I started too. And yet, I became a Python guy. ;-)

Re:BASIC/PLUS

[Suzuran]

My high school had a calculator policy of "You may bring any calculator you like to calculator-allowed tests."
So one day I decided that my calculator was GLAXIA, my PDP-11/44 which ran RSTS/E (V8 or V7, I forget which...)
I packed the whole thing on a cart; the system (Two BA11s), RA81 disk, and LA-120 teletype, and wheeled it into the classroom.
The teacher asked me what it was - "It's my calculator." The look on his face was priceless.
It was loud as hell, but the teacher allowed me to complete the test with it. I forget what I scored.
Thereafter the calculator policy was changed to read
"You may bring any calculator you like to calculator-allowed tests, provided it does not dim the lights when powered on."

Old hardware rocks!

Re:Perl 6: The Language of the Future (... Forever

[timster]

That sounds great, until you're trying to work with someone else's Perl code and it turns out that they have a special fondness for those Perl features which were inspired by awk. A language with a clean design means that you can collaborate with others.

Re:Perl 6: The Language of the Future (... Forever

[timster]

You said that it's OK for Perl to have an excessive number of features, because you don't have to know them. I'm pointing out that the truth is that you do have to know the whole language to use it. So no, I don't have to "blame myself for not knowing the language", I blame the language for being poorly designed.

High level != "automated memory management"

[Anonymous Brave Guy]

Can we please stop bashing C++ memory management? I write C++ for a living, yet very rarely use what the critics typically call "manual memory management". Either it's really not that hard to do things in better ways, or I guess I must be a superhuman programmer, because according to all the checking software, I haven't introduced a memory leak since... no, actually, I've never introduced one in as long as I've worked here. If you want to talk about the advantages of garbage collection, knock yourselves out, but please stop treating C++ and C as if they're the same in terms of memory management. They are different worlds.

In any case, garbage collection is far from the biggest benefit of using a scripting language (or whatever we want to call them) over something lower level. As others are pointing out, the more important properties exhibited by most of the modern scripting languages that make them "high level" include first class data structures, first class functions, and dynamic typing.

Re:High level != "automated memory management"

[Anonymous Brave Guy]

Before anyone else points it out, I realised that my final sentence in the parent post reads as though I think dynamic typing is necessary for a high-level language. I don't think this is so in general, but in the context of scripting languages, I think it's one of the key features that lets you write higher level code more easily. In a statically typed language, some sort of type inference serves a similar role, keeping the code generic and cutting out unnecessary boilerplate code.

Re:High level != "automated memory management"

[smittyoneeach]

Can we please stop bashing C++ memory management?

No.
If we don't insist on treating the tools themselves as the end product, then how will we perpetuate mis-information, and sell "new" products, which are, dared we look at them objectively, just re-shufflings of what has come before?

wanna be annoyed? try this

[wardk]

attend a local Perl user group meeting seeking basic knowledge like maybe a few tips about hashes and namespaces

your question may be answered if you are willing to sit through 3 hours of the alpha geeks sparring over who understands Damian Conways latest obsfucation most, hopefully with many examples of their own variations to programs that have no real use. meetup at break with someone who didn't participate in that discussion and note they don't know what the hell they are there for either

besides, anyone who doesn't know enough about perl to make changes to it's internals is a lamer

I love using perl, but in my experience, the community around it is lead by pretentious people who think you suck

ok, I admit I haven't attended one of these in years, but now you know why

your mileage may vary, hopefully much better results than mine were

Perl 6. why again should any mere mortal even give a damn??

Worst presentation in a while.

[harrkev]

Is it just me, or is this his worst presentation in a while? I think that I missed last year's, but I remember thinking that a couple of his presentations were quite original. There is the one where he based his entire presentation on an optical-illusion picture (three years ago, as I recall), and one where his presentation was based on the default Red Had screensaver collection (two years ago, if I am not mistaken). I had always thought that listening to him present one of these would be kind of like geek-stand-up-comedy. This one was downright plain by comparison.

Re:Perl 6: The Language of the Future (... Forever

[nuzak]

> Thankfully, Perl 6 follows the same principle as previous Perls

Except for actually existing.

perl 5.10 should not be neglected (Re:Fine by me)

[doom]

perl 5.8 does everything I need it to. There are other languages I use when Perl doesn't do what I need it to.

perl 5.10 is about to be released, and it has a number of significant improvements over perl 5.8. Off the top of my head: it has a real "switch" statement included (as originally designed for perl 6), it has recursive regular expressions that can be used to do Text::Balanced sorts of things (if for some reason that now-standard module doesn't do it for you), and a number of new modules have been added to the standard library.

I view Perl 6 as an continued employment mechanism for those who write books about Perl and teach Perl to others.

At the present, I look at Perl 6 as an attempt at turning perl into a saner, more rational language, by keeping Larry and Damien busy with something else.

Even so, I'm still interested in seeing where it goes. Unlike commercial projects, open source projects can continue to stagger forward to success long after many people have given up hope (e.g. Mozilla/firefox).

Re:Pair programming?

[sohp]

Pair programming isn't 2 people both trying to use the computer at the same time. Pair programming is 1 person working on writing the code and doing the keyboard tasks while the other person looks on and thinks about what's happening within the context of whatever functionality is currently being added to the code. They talk to each other, help each other notice stupid 1-off errors and typos, and bounce ideas off each other. Every so often, whenever one of them is tired of typing, or the one not typing has an idea that can be more easily expressed by typing out the code rather than talking about it, they switch.

Maybe one of them is a whiz the language and tools, and but is a bit fuzzy on the domain, and the other one is new to the language, but is a domain expert. At first the whiz does most of the keyboard work and the domain expert handles the meta stuff, but gradually they learn from each other and switching off helps them both progress. The domain expert might come to a point where he is trying to explain something complex, so decides to ask for the keyboard and type in some code that does it. It might not be syntactically correct or use the language structure in the best way, but the whiz, looking on, can help clean it up once the ideas are down.

But really, you can't fully appreciate it until you try it with someone with experience in pair programming.

the provocative larry wall

[doom]

we all know that language snobs dislike perl -- why they appear to dislike it more than the far-worse PHP is hard to discern, but clearly they do

Actually, it occurs to me that I probably do know why perl provokes such rabid responses, I think it's because of "The State of the Onion" talks.

"Larry has turned what was a gentleman's war into guerrilla tactics. He consistently and cleverly (I never said he wasn't brilliant) slanders other languages and language communities, and encourages this behavior in his lieutenants." -- Steve Yegge

I don't know where Yegge got this "gentleman's war" stuff -- when I got started in this game, Nikalaus Wirth was making pronouncements about how a generation of programmers have had their brains irreparably damaged by programming in Basic (and Larry Wall alludes to this in his talk).

I think what's really going on here is that like most "gentleman's" rules, it's only intended to protect you if you're a member of the club. The club got very upset at Wall for daring to suggest that there might be some limitations to their worldview (most "computer scientists" are really mathematicians trying hard to pretend that they're still doing math). That's the really unforgivable thing.

In contrast, the PHP culture is just happy they're doing better than ASP, and makes no explicit philosophical statements about language design.

GENTLE_ART_OF_PROGRAMMING

Put up or shut up, please

[coryking]

I'm disappointed with the state of perl. I used to be a huge perl nut and have a major project that is all mod_perl. But I'm growing increasingly fustrated by the lack of modern programming tools, especially compared to other modern languages. Even PHP has a better choice of editors. Heck, I can write syntax colored, intellesense'd python in Visual Studio!

Perl6 is a text book example of why rewrites are bad. While these people are busy writing the Programming Language to End All Programming Languages, .NET has stolen the market for the very thing these people where trying to make. Pugs, or parrot or whatever, sounds a heck of a lot like .NET/Mono. By the time Perl6 gets out, if it ever does, nobody will care about it because the open source market will be dominated by Mono. At this rate, the perl crew might be better served by just compiling down to MSIL and leveraging Mono for cross-language compilation.

So please, put up or shut up.

See also: Netscape.

Re:Put up or shut up, please

[skrolle2]

You are, sadly, wasting your breath. :-(

There is a large amount of programmers who found their comfort zone in certain loosely-typed languages, typically they did a lot of C, moved on to Perl and realized that they were much more productive in Perl, but not really realizing why, only thinking that Perl must be some holy grail. They are programmers who have never used Visual Studio or Eclipse or similar IDEs for strongly typed languages, they have *no* *idea* what refactoring is, and what it can do for your productivity, or how easy it makes your job of reading other's code with functions like "find usage".

They're used to getting no help from their editors, they're used to manual search and replace, and they're used to small-team projects where that kind of tools actually work. And since they're stuck in that comfort-zone with those languages where you *cannot* make tools like VS or Eclipse, they will never learn their power.

The lesson here is that we who use modern IDEs, we will get bypassed by something in the future, and we will be stuck in our comfort zones. Will we move on?

Re:Perl 6: The Language of the Future (... Forever

[sootman]

Congratulations on the best mental image, evar. And yeah, it's taking a while: here he is talking about Perl 6 over five years ago. (Home of the famous "big knob" quote.)

I wonder how this guy turned out: "Given this approach to learning Perl (just for a general working knowledge, maybe light usage,) is it really worth spending a lot of my time learning Perl now, or should I wait for the big Perl6 revision?" :-)