Wikipedia Chooses Lua As Its New Template Language

bonch writes "In an attempt to tackle the inefficient complexity of its current template system, Wikipedia will be adopting the Lua scripting language. Known most for its use in videogame scripting, particularly World of Warcraft, Lua is lightweight and designed for easy integration into existing applications. The transition is expected to begin after the release of MediaWiki 1.19, possibly in May." Basically, the template system started turning into an ugly programming language. There was debate over using Javascript or Lua; Lua ultimately won due to implementation concerns. The mailing list threads announcing the decision and discussing the change have further details.

Comments

Re:Lua

[Anonymous Coward]

As opposed to Javascript?

Stop delaying the inevitable.

[MadKeithV]

"the template system started turning into an ugly programming language" - ah, any sufficiently complex system eventually evolves to contain a limited, broken version of Common Lisp.
Stop delaying the inevitable!

Re:Stop delaying the inevitable.

[dkf]

"the template system started turning into an ugly programming language" - ah, any sufficiently complex system eventually evolves to contain a limited, broken version of Common Lisp.

This includes Common Lisp, which contains itself as a proper subset.

Re:Stop delaying the inevitable.

[goldaryn]

"the template system started turning into an ugly programming language" - ah, any sufficiently complex system eventually evolves to contain a limited, broken version of Common Lisp. Stop delaying the inevitable!

LEEEEEEEROOOY JEEEEEENKIIIIIIIIINNSSS

Re:Stop delaying the inevitable.

[David Gerard]

The trouble with domain-specific languages is that they are Turing complete. This is a fatal trap: your hammer may be a great hammer, but if it's Turing-complete you will (this is a law of the universe) one day be forced to use it as a screwdriver, spanner, soda siphon, and nail. You will end up having to build a working full-scale replica of the Titanic from toothpicks and spit, complete with iceberg.

Your rule is more like - any domain-specific language will eventually evolve into brainfuck. ParserFunctions certainly did.

Re:Stop delaying the inevitable.

[jbolden]

That's why DSL's in LISP are nice. They just admit the problem and include LISP underneath for when you want to do something different.

Sounds exciting

[YutakaFrog]

Lua has some notable differences from more prominent languages like Java, but as a World of Warcraft addon developer, I find it a surprisingly robust and fun language to program in. I look forward to this change to Wikipedia and hope it works well for all of their contributors.

Re:Sounds exciting

[Talderas]

I want a DPS meter addon for wikipedia.

Re:Sounds exciting

[shutdown -p now]

Lua is a language with neat syntax and clear semantics which tend towards minimalism without sacrificing usefulness, that seems to be designed by sane people for a change. Compared to some other *cough* JS *cough* it's practically a godsend.

Re:Let's Discuss having a Discussion about a Decis

[Trepidity]

This seems to be at least partial evidence that that's not really the case: it was discussed for a while, a decision was made, and implementation rather than further discussion is now happening.

Raw- or OOP-base Lua?

[Phrogz]

I'll be interested to see if they go for WoW-style "raw", imperative Lua (gobs of functions) or a more OOP-style Lua (NB: my site).

In designing the Lua interface for an old Game UI authoring product I originally went with OOP-style Lua. It was (IMHO) a rather elegant wrapper on our DOM. However, we soon found that the memory thrash of using Lua's lightweight userdata to go back and forth between C++ and Lua resulted in poor performance on consoles, and I ultimately had to redesign the interface to be more WoW-like for our next release.

It was a shame, putting more onus on the scripter to manage objects (tables of properties in Lua) based on a 'pointer' passed around to uniquely identify each element in the DOM, and passing that pointer to all relevant functions. But the performance increase was dramatic.

Re:Raw- or OOP-base Lua?

[Phrogz]

BTW, my personal opinion on Lua:

It's a fun language to learn, because at the core it is *so* simple. In less than a week a good scripter can fully wrap their head around everything that Lua has to offer from the scripting side (not the C++ side; that might be another week). It's rather elegant, really, with convenient syntax for integer-based for-loops that automatically create a new copy of the loop variable on each pass for simple closure creation.

However, when you get down to actually typing in itwell, it's not as verbose as Java, but there's some real RSI danger there. With it's simple core come decisions like "not only will we not give you foo++, we won't even give you foo+=1". Try typing things like "frameCounter = frameCounter + 1" many times and you'll start to scream. Every day I scripted in Lua at work I would long for the times when I could use Ruby to actually get something done.

For those who know JavaScript and want to get a glimpse of what Lua is like, I have a page on my site: Learning Lua from JavaScript.

Re:Raw- or OOP-base Lua?

[Short Circuit]

I hope they're going with a PHP extension, and not implementing LUA in PHP. I'd rather have the execution happen in C code than PHP, for execution speed reasons, and I imagine their server operators feel similarly. (Though there's the obvious counterpoint that you're adding a new chunk of less-tested code that has fewer barriers to cross to exploit some vulnerability...)

In any case, I can't say I'm upset at the change--I'm actually a bit giddy. MW templates are a royal PITA.

Re:Raw- or OOP-base Lua?

[nahdude812]

There is a LUA PHP PECL extension: http://pecl.php.net/package/lua

It's relatively new, but this kind of attention could really skyrocket the extension forward. It's a great idea at large, there are a variety of situations where you want to defer decisions to your customer. Historically that meant creating a kind of pseudo DSL with a bunch of forms to fill out for the customer, with hopefully most major options covered, but usually failing to satisfy a variety of corner cases.

Another alternative is the V8JS extension (JavaScript). The advantage of JS is that more people know it already, and in may ways, JS is surprisingly elegant (not that Lua isn't). It won't perform as well as LUA though, and requires more resources to maintain the VM.

Re:Raw- or OOP-base Lua?

[Darinbob]

Look at how it's implemented though. Lua is small and efficient. Ruby looks like a committee has been working on it, and Python is even bulkier. That's why you see Lua in embedded systems so much more than other interpreted languages. It doesn't get in the way of the application like say Tcl does. Yes it comes with very little in the way of "standard library" which is also a benefit in many ways. I've seen some extremely elegant OOP systems for Lua that are short and do what is needed and no more, while simultaneously others re-created their favorite OOP style and end up writing more code to do that than their application code itself. Lua works because it keeps it simple.

Re:Raw- or OOP-base Lua?

[pnot]

However, when you get down to actually typing in itwell, it's not as verbose as Java, but there's some real RSI danger there. With it's simple core come decisions like "not only will we not give you foo++, we won't even give you foo+=1". Try typing things like "frameCounter = frameCounter + 1" many times and you'll start to scream.

Which, for me, immediately raises the question "Are there any good Lua IDEs?". I mainly code in Java, and it's true that it can often read like the Book of Deuteronomy -- but fortunately I don't have to type all that shit out, because NetBeans autocompletes a lot of it for me. Is there anything similar for Lua?

Re:Lua

[Anonymous Coward]

Javascript is web scale. Lua is not web scale. Also: Lua comes from Brazil. You know what else comes from Brazil? Waxed balls. I wouldn't trust a programmer that waxes his balls. If he can't make good decisions involving his nutsack, can he make godo decisions involving language design? (Just look at PHP!)

Re:Lua

[james_van]

That's a really valid argument. Im inclined to agree.

Re:Yay!

[Trepidity]

You don't generally need to be an administrator to edit the scripts, with the exception of a few scripts that are used on so many pages that they're vandalism magnets. And even for those, you can propose changes on the talk page, which are usually made if they're reasonable. There is not really a whole lot of politicking around the content of scripts, although admittedly that's partly because the home-rolled language sucks so much that very few people care to figure out how to edit pages that look more like line-noise than classic Perl did.

There's sometimes politicking about whether a particular one should exist or be used at all; some people find the proliferation of infoboxes, footer boxes, succession boxes, portal boxes, etc. too much clutter and not very useful. But the internals, afaik, aren't one of the hotbeds of debate.

Not a language problem

[zarlino]

Wikipedia could stick to PHP or switch to any other language. But that's not their problem. Their problem is the messy markup language they slowly created. I know cause once I tried to render their markup inside another app. Basically, they have all sorts of tags that reference obscure server-side behaviour and everything is so entangled that creating a new renderer is basically impossible. This is sad because they are wasting the work of volunteers.

Re:Not a language problem

[JDG1980]

This actually is a serious problem that has been discussed on the Wikipedia development and foundation mailing lists. Because Wikicode is not rigorously defined like real HTML/XML, the only definition of correct output is "whatever the current parser generates." This not only makes it nearly impossible to independently implement Wikicode in other products besides MediaWiki, but it also makes it far more difficult to create a WYSIWYG editor that doesn't break things. And doing the latter has been a goal of many people high up in Wikipedia for some time.

Re:Not a language problem

[David Gerard]

That's the precise problem. 1. the language was never designed, it accreted, and is mathematlcally impossible to describe fully in most sensible formats. 2. we can't throw it away because there's billions of words of text in it accumulated over ten years. 3. we can't throw it away because the existing editor base demand it stay because they're used to it.

So WMF is (a) throwing money as well as brilliance at the problem, and (b) has put Brion Vibber onto sorting out what is to be removed from wikitext, because he's one of two people (Tim Starling the other) that people will accept the opinion of on this matter. All proceeds well :-)

So now the problems are with seriously complicated things like doing bidirectional text properly - a hard requirement for an international project, and one that is not done quite properly by anyone else. Something where mere dev brilliance has half a chance :-)

Re:Lua

[Hognoxious]

Lua is a terrible language. It is also an excellent language.

FTFY (see WP:NPOV)

Lua is Great for Configuration

[alexbirk]

Embedding Lua for configuration or building templates is it's real strength. I've used it many times in programs that require pretty extensive configuration and it's a joy in that environment. I think it's a great choice for this.

Re:Lua

[Anonymous Coward]

I've never been able to get into Lua, whereas Python and Javascript I have. I prefer C/C++, Java, C# however. Though, if I had to pick a scripting language to use I'd use Python. Ruby is unbearable for me, I've tried to like it just can't. Lua isn't unbearable so much as I'm generally too bored with it to try it.

Things I generally don't like about Lua:
1) Lua replaced the ! symbol with ~ for Not. This doesn't make any sense. Not that ! made much sense either. I do like that they added a "not" keyword. That at least makes sense.
2) It feels very BASIC with its do, end, then, end etc. That's not a good thing. We should be moving away from that garbage.
3) I'm not a particular fan of any language that wants me to type one of; local, let, or var. The parser should be able to figure out when I'm assigning at all times. And, they still use == in this language which is silly since the whole point of using local, let, or var is so you can distinguish = from assignment or condition. It's annoying when languages that do this. Having said that, you could just pull a Pascal instead := or something if you absolutely must avoid ==, I'd complain a lot less. At least it's a clever and convenient way of addressing the problem. Let's face it, == only makes sense to a programmer, a mathematician would look at you weird. Then again they'd look sideways at := too, whereas Let in math has a meaning. Really I think math should teach := and drop the let, it'll save room on whiteboards.
4) Why should you have to prepend = to evaluate something? Like: = 2+3, or even = 3 == 3. Why can't I just type 2+3, or even 3 == 3? It's weird.
5) They went so different with everything, but they kept % for modulo? Doesn't make sense, why not just type mod? The symbol never made much sense. If they were being really adventurous they would have tried harder IMHO. Having said that, I respect that they switched ^ to power instead of xor.
6) I'm not a fan of using .. to concatenate strings either. "a".."bc" looks godawful goofy. I much prefer "a" + "bc". I get that it can cause problems when you need to mix in math, but that's what parenthesis are for. I see why they didn't use + though, they support "coercion", like 100 + "7" becomes a 107 integer. Which is a pretty cool concept I guess. But, it forces you do something weird like .. for concatenation which is a minus against it. Maybe they could have done just "a" "bc" becomes "abc". Then it'd be as simple as two strings like str1, str2 just being placed like... str1 str2 in order to concatenate them. Then they could still use + for coercion.
7) I don't mind nil for NULL. It's better than Python's Nothing IMO because it's shorter. Though, I see where Python was going, they were trying to make it obvious when read by a user who doesn't know programming. However, the rest of Python isn't obvious to a non-programmer anyways, so that defeats the purpose. Whereas, nil is at least is less typing, doesn't require an uppercase, and it has an English and Latin definition so as far as being a real word it's wins against Nothing in at least simplicity. I give them props for using this.
8) tonumber("10") is goofy, they're going for English I guess. Python's int("10") is less typing so I'd prefer that. I think when it comes to English vs typing less, as long as it isn't a weird symbol that makes no sense, I lean towards typing less. However, I could live with tonumber, it's not one of the worst things in this language.
9) Having to type function or def in a language to declare a method is annoying. If I was forced at gunpoint to choose, I'd probably go with function because at least it's obvious, less typing didn't win out here, def is weird looking and doesn't make sense at all. The def keyword reminds me of Sub from BASIC, in a bad way. I realize it's a bastardization of defmethod from Lisp. If we absolutely have to have some way of parsing methods (I think there should be a way around it IMO) startin

As opposed to a Wordpress style engine?

[PortHaven]

Seriously, Wikipedia's #1 fault and the reason I ceased actively contributing is that it requires humans to use a mark-up language for what is essentially a simple text based document.

And all such edits would be handled much easier via a WYSWIG editor. Yes, elitist monkeys with far too much time on their hands love that feel of doing something complicated for the sake of it.

Those more intelligent and or beings who have furthered the race through reproduction tend not to want to waste time.

Implement a simple editor that facilitates editing. And let computers do what they do best, process. And humans do what they do best collate ideas and knowledge.

First rule of computers. Don't waste time doing what a computer can do better than you.

Re:As opposed to a Wordpress style engine?

[Hatta]

I assume you mean something like HTML or such. But I also suspect this does not apply to something like a word processor document, where you likely use Word or something similar?

He's talking about LaTeX. It's both a word processor document AND code. And it's better than Word in every way except the steepness of the learning curve.

Re:As opposed to a Wordpress style engine?

[epine]

Wikipedia documents are supposed to be word processor documents.

Huh, I never got that memo. I thought it was a system for collaborative editing and mass distribution. In my own experience, I've always found WYSIWYG turns far too quickly into WYGIWYSW (what you get is what you're stuck with). And besides, the collaboration system depends on a diff tool with a human-accessible interpretation. It's not as if authoring is the only mission-critical task.

I started writing some WordPress posts recently in the visual editor, and man does that suck. A simple paste carries all kinds of format from the source document I usually don't want. When I press "undo" to use the clunky "text paste only" widget, the undo scrambles my window scroll position, and sometimes takes out a piece of my previous edit as well. It's a PITA to add text to a paragraph that ends with link text. CR SPACE BACK-ARROW BS FORWARD-ARROW seems to work to get me out of the link text format zone. This is better than raw markup? How, exactly?

Different strokes for different folks, I guess.

I've been following LuaTeX for some while. It's a lot of things, but compared to base LaTeX being a speed daemon is not one of these. Should be plenty fast enough for Wikipedia, though.

One thing that turned me off Wikipedia was the asymmetry of the quantification system: there's no way to add the statement "it has never been said that ..." to a valid article. Of course, this is not a problem if someone notable has bothered to make a trivial observation in that mode on the record, but surprise ... it turns out that notable sources often have better things to do than state the obvious. Kind of like Godel's theorem: in any attestational system, there is something blindingly obvious no-one has ever bothered to note for the record because it's too trivial to bother with. The same thing bugs me with peer review: positive results circulate, negative results vanish without a trace. I find it a burden to create balance working in a half a predicate logic, but I'm weird that way, let me be the first to admit it. In a way, a person of my temperament never really belonged there in the first place. I graciously retired when I discovered the fault was on my side, though I do still fix howlers whenever they cross my path.

I was also frustrated with how Wikipedia lost traction on leverage. (I don't want to belabour that just now.) Hopefully this is the beginning of reversing the tide.

Re:It's called "the inner-platform anti-pattern"

[Lennie]

As I understand it:

MediaWiki is written in PHP and they wanted to create a sandbox for the templates scripts, so they choose to use Lua as a PHP extension.

Because Lua is very suitable for embedding, as that is what it's general purpose in life is.

There is no PHP in a sandboxing inside PHP as far as I'm aware of.

Yay lua

[Osgeld]

the only language that would use more words to describe the article than what's in the article

Re:Lua

[Anonymous Coward]

1) In standard logic, ~ means NOT. Since LUA uses functions for it's bit-wise operations, I see no issue with using this.
2) Why? People don't consider C to be BASIC and it allows pre and post-test WHILE loops.
3,4) I got nothing
5) You should really make up your mind here. They use the normal % for modulo and you complain? Come on now.
6) PHP uses . to concatenate strings, seems like you're splitting hairs here for no particular reason other than to complain.
7) Wow, you found something good?
8) LUA is dynamically typed, why would they have an int()?
9,10) You really shouldn't be using LUA... go with a language that you like. It sounds like you prefer Python anyways.

Just as a note, I haven't programmed anything in LUA before but I've looked at the code and looked over the reference manual. Looks like a fine language; I just have no use for it as of yet.

Re:Lua

[Xtifr]

Disclaimer: I mostly like Lua, though I wouldn't say I love it. Like all languages, it has advantages and disadvantages. Lua's strength is embedding, though, and that's where it shines--the only other language that comes close is Tcl, and Lua is cleaner, IMO.

1. I mostly agree with you that ~= for not equals was a mistake. As another poster pointed out, it's a somewhat justifiable mistake, but we had nearly managed to standardize !=. Lua is the first recent language to ignore this near-standard. After 11, below, this is probably my biggest complaint about Lua.
2. BASIC? Really? Dude! Those keywords don't come from BASIC! (In fact, I've never used a version of BASIC that supported any of them--oh, and get off my lawn!) Those come from Algol, probably by way of Pascal or Modula, and they're great, which is why BASIC ripped them off--they're probably the only good part of any flavor of BASIC, because they're not BASIC features. :)
3. I'm not sure what you're on about with "local, let and var" (though I think I disagree), but the rest of your rant (about "==") is inconsistent with your earlier complaint about "~=". Like "!=", "==" has become more-or-less standard, and I'm glad Lua didn't decide to innovate here.
4. What are you on about? "print(2+3)" works just fine, as does "a = {1, 2+3}".
5. Man you're stretching! And again, inconsistent with your point 1.
6. Matter of taste. I think I actually prefer "..", but it's not something I feel strongly about either way. Using "+" for concatenation tends to work better in an OO language with operator overriding. Lua's more of a low-level embedded language.
7. Like, whatever. Is this really worth even discussing? 47 different languages do this 47 different ways, and all of them are fine.
8. tonumber() is consistent with the other coercion functions, and if you really hate typing that much, you should probably find another line of work. I certainly wouldn't want to hire you. People who complain about extra typing are generally the ones who write opaque, cryptic, incomprehensible code with no comments.
9. Oh. My. God! If you ever design a language, I will pray that I am never, ever forced to use it! :) Oh, and "def" doesn't come from Lisp--it's simply short for "define". P.s. if you really want to use just one delimiter everywhere, try Tcl. It's not a bad alternative to Lua if you're looking for an embedded language, and it uses curly braces for everything--even function arguments. P.p.s. those aren't methods, because Lua's not an OO language. Those are functions. (Or procedures, though Lua, like most modern languages, doesn't distinguish between the two.)
10. I don't think I've ever encountered this quirk, so I won't comment, except to say, if you don't like that, don't do it!
11. You only had 10 points on your list, and I'm truly amazed you left out the one biggest issue most people (especially those familiar with C, C++, Java, Perl, Python, Ruby, etc.) will trip over--one-based arrays! If you're going to rant about Lua, how can you possibly ignore the exasperating one-based arrays? Are you even a programmer? :)

Anyway, Lua's not really competing directly with perl/python/ruby. Its strength is that it's small, fast, and easily embeddable. The ease with which you can call back and forth between Lua and C is what really makes it shine. Some of its quirks seem to be choices made for performance reasons, and I'm willing to live with that. Overall, I like its style and flavor better than tcl, which seems to be its main competition.

Re:Let's Discuss having a Discussion about a Decis

[svick]

Using Lua instead of the current template syntax will not mean much for editors of articles and nobody claimed it would. It will only make (huge) difference for those who currently write templates.

On the other hand, there is also some work going on to make editing of articles easier using a WYSIWYG editor.

Re:Lua

[glwtta]

I wouldn't trust a programmer that waxes his balls. If he can't make good decisions involving his nutsack, can he make godo decisions involving language design?

Seems like a perfectly good decision to me. There really is nothing like a shorn scrotum, it's breathtaking; I suggest you try it.

Re:Wrong problem

[JasterBobaMereel]

The problem you are seeing is with the editor you are using ....not with MediaWiki..

Try dragging a URL into a text editor and it will fail ...what a surprise ?