NetBSD To Support Kernel Development In Lua Scripting

An anonymous reader writes "NetBSD 7.0 will support the Lua scripting language within its kernel for developing drivers and new sub-systems. A Lua scripting interpreter is being added to the NetBSD kernel along with a kernel API so developers can use this scripting language rather than C for developing new BSD kernel components. Expressed reasons for supporting a scripting language in a kernel were rapid application development, better configuration, and "modifying software written in C is hard for users." In a presentation it was said that Lua in the kernel will let users explore their system in an easy way."

Comments

users?

[gTsiros]

"modifying software written in C is hard for users."

i thought *users* had no business writing device drivers?

of course if they had said "modifying software written in C is hard for programmers." it would gather a fierce shitstorm.

Re:users?

[TheRealMindChild]

It seems crazy, but this may not be as braindead as it seems. For instance, a keyboard driver. You just need a read key state->have a look up table to the keycodes function, then pass it on. This is redundant, few lines of code with very little difference between most other drivers. You may not want to write your graphic drivers in it, but not every driver needs bare metal functionality

Re:users?

[KiloByte]

So let's invent a sandboxed layer that can interface with the kernel, but can't do damage to it? Also, make the API stable, unlike innards of the kernel. Let's call this novel idea "userspace" or such.

Re:users?

[Mindcontrolled]

The moment you even expect that users should, could, would or even remotely in their most maddened fever dreams WOULD WANT to modify the kernel, you lost it.

Re:users?

[MacTO]

Keep in mind that NetBSD users are quite different from Windows/Macintosh/Linux users (at least on average).

"Hard" may also be a reference to the implementation, rather than the language. Interpreted languages tend to bypass the compile and link part of the development process. This means that interpreted languages are easier to develop with and compiled languages are hard to develop with (at least in some respects). I'd also comment on the arcane nature of C, but I haven't used lua so I don't know if it is any better.

Re:users?

[Lodragandraoidh]

Interesting - failure of user space in this way is exactly why we have zero-days.

I would like to see this happen - but several things make it improbable:

1. Von Neuman architecture. As long as data and instructions exist in the same space - poorly written apps will allow abuse of it.

2. Complexity of current software. The more complex the software, the more likely a bug will exist in it that allows #1. Given how programmers stitch together preexisting modules without understanding what is being done on the underlying system - I only expect that to continue expanding.

It should be instructive that Java was supposed to be that sandboxed layer...and it has so many zero-days it looks like swiss cheese.

Now - how would we avoid that and make an unhackable userspace?

Re:users?

[Anonymous Coward]

I only have 640 KB you insensitive clod!

Re:users?

[jbolden]

i thought *users* had no business writing device drivers?

My non programmer wife wrote a custom device driver in AppleScript, not even knowing what a device driver is. Those sorts of languages can be amazing empowering to end users, and yes there are reasons that end users might want a custom device driver if it is easy enough to accomplish. Though this makes more sense for OSX than NetBSD, I doubt there are many if any non programmers who use NetBSD.

I had to write a custom device driver for Windows once to get something more like a Unix ethernet driver i.e. bypass some normally quite excellent parts of the Windows networking stack. I was doing software not hardware so I guess I'd consider myself an end user. And that was a pain in the ass.

Performance

[Anonymous Coward]

Won't that kill performance?

I mean that's the reason we still write kernels in C. Every cycle counts..

Re: Performance

[Anonymous Coward]

So we use it for performance. Ok I get it now.

Re:Performance

[DarwinSurvivor]

Pfff, writing Linux kernels in Javascript is so last century. Nowadays we RUN kernels in Javascript!

Re:Performance

[stenvar]

Kernel code is like lots of other code: a lot of it is executed rarely and not performance critical. There's setup code, occasional permissions checking, etc.

In addition, this is not to replace plain C modules (although it may be useful for prototyping), it's for complex and dynamic configurability.

Right now, there are two ways of handling that. One is to invent complex configuration files and data structures, sometimes even little interpreters. The kernel is full of those. They are error prone, complicated, and require a lot of effort to maintain. The other is to put complex decision making into user-space, but that involves context switches and other overhead. It also means lots of special-purpose and redundant code, and lots of documentation and complexity. The kernel uses both of those strategies extensively, one reason why it's so big.

Putting Lua+LuaJIT in the kernel is likely faster than either of those approaches, while at the same time also being easier to maintain and document.

Re:Performance

[10101001 10101001]

Kernel code is like lots of other code: a lot of it is executed rarely and not performance critical. There's setup code, occasional permissions checking, etc.

A lot of it is executed rarely because at least half the code is drivers and one usually doesn't have but one sound card, one (or two) network card, etc vs the hundreds of sound card drivers, hundreds of network card drivers, etc. Further, a major point of the kernel is that it, by design, isn't supposed to be executed a lot because most CPU time should be devoted to user space programs. So, to that end, you could say it's not "performance critical". But, that's more in line with the point that precisely because kernels are generally developed so well, they rarely have "performance critical" code. Once you start throwing scripting into it, I think that goes out the window.

In addition, this is not to replace plain C modules (although it may be useful for prototyping), it's for complex and dynamic configurability.

To this, I could see some benefit. But, honestly, I'd feel it safe and saner if it were a language that was reducible to a FSA. The point that "Dangerous code can be disabled at the byte-code level (prevention of endless loop DoS etc)" in the slide really only works in that context, anyways. In any case, Lua is nothing of the sort.

Right now, there are two ways of handling that. One is to invent complex configuration files and data structures, sometimes even little interpreters. The kernel is full of those. They are error prone, complicated, and require a lot of effort to maintain.

Which is an argument for making a good, universal FSA language. It won't remove the being error prone or complicated because that's the nature of the beast and papering over it with a Turing complete language is no sort of answer. But having them all share the same language makes sense from a maintenance standpoint. For example, the BPF was repurposed for syscall filtering--a little admission that I'm not sure entirely where BPF stands language wise although I don't think it's turing complete--which could probably be refactored into a more general filtering language which could be refactored into an even more generic FSA language which could likely be used to refactor a lot of other code.

The other is to put complex decision making into user-space, but that involves context switches and other overhead. It also means lots of special-purpose and redundant code, and lots of documentation and complexity. The kernel uses both of those strategies extensively, one reason why it's so big.

Generally speaking, context switches aren't a big deal in most OSs/kernels today. Yes, certainly having context switches means an extra overhead and stuffing some code or even all code in the kernel would certainly speed it up. And certainly if a lot more code moved into user space would be measurably slower. But kernel design today seems to try to have a balance between the two, especially with considerations of stability and security. The fact is, lots of code is special-purpose because it's for specific hardware. And code being redundant is more a function of maintaining multiple trees--which admittedly having a single kernel allows a lot of unity--but there's no reason that out-of-kernel-space means out-of-the-kernel-source if the code is still rather kernel specific.

After all, the whole point is precisely that a lot of potential kernel code is "executed rarely and not performance critical" yet remaining outside kernel memory gives the system greater protection and hence speeds up development and generally makes the code easier to maintain. Documentation is a wholly separate beast, in any case, and that's more a point of maintainers or distributors demanding good documentation before code is accepted.

Putting Lua+LuaJIT in the kernel is likely faster than either of th

long overdue

[stenvar]

This is a good thing, and it's long overdue. Linux should do the same.

It's actually pretty easy to get Lua running in the kernel and call kernel APIs; but to make it really useful, it needs additional kernel hooks and callbacks, and that requires cooperation from the core kernel developers.

Great idea.

[Meshugga]

I honestly think that this is a great idea. It's a novel approach to extending a kernel, especially considering NetBSD doesn't have that big of a market share and expressly focusses on supporting as many platforms as possible. Prototyping new features or certain drivers in a portable scripting language may give them a leg up in the functionality aspect of the race.

Also, LUA is super fast, small, easy to learn, concise and the C API/embedding it is straight forward.

You may knock it all you want, "because kernel land is C land", but after all bias being said and flamed, this may turn out to be a really fun, interesting and possibly very useful idea.

Tags

[Areyoukiddingme]

Surely this article deserves the whatcouldpossiblygowrong tag more than any other this year. And possibly last year too...

Re:Stupid NOT

[Anonymous Coward]

I don't know how much Unix/BSD/Linus code you have read or written, but I have been developing core code, Scheduler, MM and Drivers since 1980 and this is a great idea, long overdue.

The Abstractions are now well understood and most code dos't need to be quick, but safe, putting PRE in kernels will clean up acres of cockamamie code now written in C, most of which just runs at boot time and is then freed. ... there are exceptions, you cant't have the garbage collector running whenever it likes, eg in the middle of an interrupt routine, but managed memory, not C, C++ scoping is the way to stop slow memory leaks.

Put simply you are an idiot and dont know what you are talking about.

MFG, omb

Re:Stupid

[dalias]

It is potentially stupid, but the reason is much more subtle than most people realize. For all its limitations and headache, C does have one key feature that makes it suitable for kernel programming where most other languages are not: the concept of well-defined storage durations and reserving storage for an object. I really question whether Lua code running as part of the kernel can be made robust under out-of-memory conditions which a kernel must be able to survive.