Slashdot Mirror

At least according to some rough microbenchmarking I've done myself. My workload is to write raw CSV to disk as fast as possible. In testing, NILFS2 was nearly 20% faster than ext3 on a spinning disk.

It was also smoother. Under very heavy load ext3 seemingly batched up writes then flushed them all at once, causing my server process to drop from 99% to 70% utilisation. NILFS seemed to consume a roughly constant percentage of CPU the whole time, which is much more in line with what I want.

NILFS2 is not for everyone or for every purpose. But it suits my purpose. As usual, you should do the engineering thing: consider your needs, test the alternatives.

Well, the same is true also for Perl, PHP, Python and even Lua, so that's nothing radically new.

If you're partial to Lua, it has SQLite bindings (and various other DB engines).
I've not used LuaSQLite personally so I can't give you a code sample, but I wouldn't imagine it would be much more code than the Python example.
As a bonus, Lua was primarily designed as an embedded language (of course Python can be embedded too).

...speed, when you notice that you're losing it, is important.

The Bayesian analysis in spam filters only works on text. Spammers realized that they could get around it by filling the text portion of the message with some random passage from a Project Gutenberg file, thus making it seem innocuous, and then putting the real advertisement in a GIF or PNG file that would be displayed by HTML-capable mail readers. Bayesian analysis can still work, but only in combination with OCR software.

Bayesian filters (and other statistical filters colloqually known as Bayesian) can work on any features at all; not necessarily text. In particular they can use the markup in the header of the message, the message encoding, and so on. Some of the best-performing filters don't use 'text' at all and simply treat the entire message, images and all, as a bit string; for example, compression-based filters. Another well performing filter, OSBF-Lua, uses orthogonal sparse binomial bigrams rather than individual tokens.

Recent standardized testing shows that these methods work just fine on image spam, without any OCR component.

Lua is already faster than most dynamic typed runtimes, add a JIT to lua and it's register based VM becomes competitive with Java and Mono.

I've seen a tamarin "typed" benchmark and heard people say it is slow on eval, ie: code as data in a dynamic language. From this I assume tamarin is stack based and that the VM is not suited to dynamic languages (like javascript). If I'm right, Mozilla have only chosen tamarin because Macrodobe are willing to donate engineering resources. I'd like to see a benchmark and comparison between neko JIT and tamarin, something similar to this comparison between neko and lua.

Fidelis Assis (who has now gone solo after having participated in the CRM114 project) shows great results for his recent solo effort: OSBF-lua Bratko's PPM spam filter -- the one that did great at TREC -- is not yet packaged as a drop-in filter. Same for my DMC spam filter.

The actual TREC 2005 tests referred to in TFA are here.

> sourceforge.net, freshmeat.net, tinyapps.org, packetstormsecurity.org

Yup, and rubyforge and LuaForge as well. One hard part is sorting out duplicates... getting a list of the actual projects shouldn't be too hard since most project aggregation sites have RSS feeds with that info.

Lighttpd embeds lua for use as a front end cache. There's also been a few attempts at a mod_lua for Apache. Lua is cool, one of my favorite languages and I much prefer it to ruby or python. If you haven't already, check out luajit and of course there's nekovm (& mod_neko) that looks like it's ready to deliver on some of parrots promises.

People complaining about Lua memory management performance and multi-tasking scalability are behind the times.

The 5.1 development version of Lua, due for final release late 2005, includes an incremental garbage collector and has been enhanced by Mike Pall with optional patches that allow for the efficient multitasking of large numbers of scripts (http://lua-users.org/wiki/ResumableVmPatch and http://luajit.luaforge.net/coco.html). Note that those patches fit a niche similar to stackless Python. Running scripts in different threads was already supported in prior Lua versions with only a small amount of C-side infrastructure.

In addition, Mike has recently released a JIT (http://luajit.luaforge.net/) extension for even better performance.

People complaining about Lua memory management performance and multi-tasking scalability are behind the times.

The 5.1 development version of Lua, due for final release late 2005, includes an incremental garbage collector and has been enhanced by Mike Pall with optional patches that allow for the efficient multitasking of large numbers of scripts (http://lua-users.org/wiki/ResumableVmPatch and http://luajit.luaforge.net/coco.html). Note that those patches fit a niche similar to stackless Python. Running scripts in different threads was already supported in prior Lua versions with only a small amount of C-side infrastructure.

In addition, Mike has recently released a JIT (http://luajit.luaforge.net/) extension for even better performance.

Well it didn't start off as a rant but the gallery site is dead, that's what happens when you get a few thousand requests making shitloads of database queries from a bloated runtime. Gallery, "it's got all the right buzzwords but dies under load", great engineering job guys, now go lookup what caching is all about.

I'm not really this much of an asshole, honest. Check out luajit if you haven't already.

http://luaforge.net/projects/luax/

I didn't really mean for that to be jab at Pygame. The two are completely different projects with different goals.

> if you look at sourceforge stats

Yup, but remember, there are other homes out there that sort of dilute the SF numbers - i.e., RubyForge and LuaForge.