Slashdot Mirror

``The more I write code and design systems, the more I understand that many times, you can achieve the desired functionality simply with clever reconfigurations of the basic Unix tool set.''

I am glad he figured that out. Unix contains a lot of useful commands and functions. Every programmer would do well to learn them, so they can use them when needed.

What also helps a lot is knowing your programming language well. By that, I mean not just the standard library, but also the fundamentals of the language. Know how things fit together, which things are efficient and which things are costly, and how to design good APIs in that language. Better yet: know this for multiple languages - different languages for different tasks. It's easy to learn a new programming language when you already know how to program, and doing so will provide you with new insights, so go ahead and learn another language.

It is also useful to know tools and libraries that aren't a standard part of the operating system. Of course, there are very many of those and learning them all might be a bit much. Still, knowing a handful and having heard of others can help a lot. No need to write, debug, and maintain your own code for parsing CSV, XML, YAML, or whatever the file format of the season is. Beware of bugs and misfeatures, though. Many libraries, even commonly used ones, contain bugs. Many libraries also contain choices that may or may not be a good idea for what you want to do.

All in all, you can get a long way by just knowing what other people have already done for you. Thanks to free software, you can often use this work in your projects. The world wide web has made it fairly easy to find things that you may use. A lot of software can be made by taking a bunch of libraries and writing some code to stitch it together and make it do stuff. Or, as Mr. Dziuba found out, by mixing together some command-line tools. I am glad he is discovering the strengths of Unix and getting excited enough about them to tell the world about it.

Stitching things together from existing components is not all there is to programming, however. You can go a long way doing just that, and many people make a living that way, but there are some things which are best solved or can only be solved by thinking hard and coming up with some clever code to do exactly the right thing. And somebody needs to write those tools and libraries, too. Compared to putting some existing components together, it's hard work for small results, but I find this to be the most rewarding part of programming. You're really coming up with something new here, making something clever and useful that wasn't available before.

``HTTP was never really designed for this sort of thing.''

Exactly. The World Wide Web is good at what it was designed for: static pages with text and hyperlinks. Go to your favorite search engine, type in some keywords to describe what you're looking for, and it will find you pages with information about that. The Web does this better than any other system I know of.

If you want to build interactive, responsive, graphical applications, HTML and HTTP aren't the right tools for the job. This isn't what they were designed for, and they aren't very good at it, either. Add CSS and JavaScript and you can get a long way - provided the user has a browser that handles what you feed it the way you want it, and the user has a PC that is sufficiently overpowered for the task at hand that it will run smoothly, even though it is programmed very inefficiently. HTML 5 intends to give people who want to go this route some more tools to work with, like the canvas element. However, this is just piling on kludges and throwing CPU cycles at it until it kind of sort of works well enough. Still, the success of, for example, Facebook, shows that there is demand for applications that go beyond what HTML and HTTP were made for.

What I have been advocating for years is creating an open technology that really lets people do the things they want to do that are beyond the scope of HTML and HTTP. Pixel-perfect graphics, lightweight two-way network communication, a complete set of GUI widgets, and a runtime to allow safe and efficient remote code execution, complete with a library system to allow for extensions to be written and distributed efficiently and without stepping on each other's toes. Since neither web standards organizations, nor web browser makers have shown eagerness to work in this direction, I have decided to do it myself. I have just recently begun to write down some ideas, under the name Jungle. It does everything I just mentioned, and while I'm at it, I want to throw in 3D graphics and a generic data encoding designed for efficiency and to be easy to parse (because, frankly, HTML is a mess, and XML, while popular, wasn't the best choice at the time it was invented, and certainly isn't a good choice for some of the things it's used for today).

As I said, I have only just started to write these things down, so these aren't complete specifications yet, and there certainly isn't a working implementation yet. I'm trying to get more time to work on this, but if anyone wants to help, that would be greatly appreciated. My plan was to get a more complete picture online before drawing people's attention to it, but this story seems like a good opportunity to get people's attention, so here goes. If you like what you see, I will be delighted. And if you have a better idea, let's hear it! I will be looking for help getting implementations written once the specifications are up, but that is going to take a while. If you would like to help, though, drop me a line and I'll keep you in mind.

``HTTP was never really designed for this sort of thing.''

Exactly. The World Wide Web is good at what it was designed for: static pages with text and hyperlinks. Go to your favorite search engine, type in some keywords to describe what you're looking for, and it will find you pages with information about that. The Web does this better than any other system I know of.

If you want to build interactive, responsive, graphical applications, HTML and HTTP aren't the right tools for the job. This isn't what they were designed for, and they aren't very good at it, either. Add CSS and JavaScript and you can get a long way - provided the user has a browser that handles what you feed it the way you want it, and the user has a PC that is sufficiently overpowered for the task at hand that it will run smoothly, even though it is programmed very inefficiently. HTML 5 intends to give people who want to go this route some more tools to work with, like the canvas element. However, this is just piling on kludges and throwing CPU cycles at it until it kind of sort of works well enough. Still, the success of, for example, Facebook, shows that there is demand for applications that go beyond what HTML and HTTP were made for.

What I have been advocating for years is creating an open technology that really lets people do the things they want to do that are beyond the scope of HTML and HTTP. Pixel-perfect graphics, lightweight two-way network communication, a complete set of GUI widgets, and a runtime to allow safe and efficient remote code execution, complete with a library system to allow for extensions to be written and distributed efficiently and without stepping on each other's toes. Since neither web standards organizations, nor web browser makers have shown eagerness to work in this direction, I have decided to do it myself. I have just recently begun to write down some ideas, under the name Jungle. It does everything I just mentioned, and while I'm at it, I want to throw in 3D graphics and a generic data encoding designed for efficiency and to be easy to parse (because, frankly, HTML is a mess, and XML, while popular, wasn't the best choice at the time it was invented, and certainly isn't a good choice for some of the things it's used for today).

As I said, I have only just started to write these things down, so these aren't complete specifications yet, and there certainly isn't a working implementation yet. I'm trying to get more time to work on this, but if anyone wants to help, that would be greatly appreciated. My plan was to get a more complete picture online before drawing people's attention to it, but this story seems like a good opportunity to get people's attention, so here goes. If you like what you see, I will be delighted. And if you have a better idea, let's hear it! I will be looking for help getting implementations written once the specifications are up, but that is going to take a while. If you would like to help, though, drop me a line and I'll keep you in mind.

``HTTP was never really designed for this sort of thing.''

Exactly. The World Wide Web is good at what it was designed for: static pages with text and hyperlinks. Go to your favorite search engine, type in some keywords to describe what you're looking for, and it will find you pages with information about that. The Web does this better than any other system I know of.

If you want to build interactive, responsive, graphical applications, HTML and HTTP aren't the right tools for the job. This isn't what they were designed for, and they aren't very good at it, either. Add CSS and JavaScript and you can get a long way - provided the user has a browser that handles what you feed it the way you want it, and the user has a PC that is sufficiently overpowered for the task at hand that it will run smoothly, even though it is programmed very inefficiently. HTML 5 intends to give people who want to go this route some more tools to work with, like the canvas element. However, this is just piling on kludges and throwing CPU cycles at it until it kind of sort of works well enough. Still, the success of, for example, Facebook, shows that there is demand for applications that go beyond what HTML and HTTP were made for.

What I have been advocating for years is creating an open technology that really lets people do the things they want to do that are beyond the scope of HTML and HTTP. Pixel-perfect graphics, lightweight two-way network communication, a complete set of GUI widgets, and a runtime to allow safe and efficient remote code execution, complete with a library system to allow for extensions to be written and distributed efficiently and without stepping on each other's toes. Since neither web standards organizations, nor web browser makers have shown eagerness to work in this direction, I have decided to do it myself. I have just recently begun to write down some ideas, under the name Jungle. It does everything I just mentioned, and while I'm at it, I want to throw in 3D graphics and a generic data encoding designed for efficiency and to be easy to parse (because, frankly, HTML is a mess, and XML, while popular, wasn't the best choice at the time it was invented, and certainly isn't a good choice for some of the things it's used for today).

As I said, I have only just started to write these things down, so these aren't complete specifications yet, and there certainly isn't a working implementation yet. I'm trying to get more time to work on this, but if anyone wants to help, that would be greatly appreciated. My plan was to get a more complete picture online before drawing people's attention to it, but this story seems like a good opportunity to get people's attention, so here goes. If you like what you see, I will be delighted. And if you have a better idea, let's hear it! I will be looking for help getting implementations written once the specifications are up, but that is going to take a while. If you would like to help, though, drop me a line and I'll keep you in mind.

Thank you for your interest! I'm actually working on a number of projects simultaneously, but my big project is Mana. It's also the one that is the furthest away from being ready, because I want to use my experience with the other projects to make Mana really good - or at least avoid as many painful mistakes as I can.

The idea for Mana is to take a look at the programming languages already out there (if you take a look at my (outdated) resume, you will see that I know quite a few, and I've looked at more) and combine some of the good ideas into a single language. To give some examples:

  - Common Lisp offers extremely powerful abstraction mechanisms, effectively allowing you to grow the language to fit your task, until implementing your program becomes practically trivial. Garbage collection, higher-order functions, macros, and a regular syntax make this work, so Mana will have these, too.

  - Haskell and OCaml allow a lot of errors in programs to be detected at compile time. This is made possible by well thought-out type systems and static type checking. Numerous Haskell programmers have had the experience that, once the program makes it through the compiler, it does what you wanted it to do. I would like Mana to also have this ability. Perhaps this can even be extended to handling of errors that can occur at run time, so that a program that makes it through the compiler is guaranteed to not crash (i.e. everything that can happen invokes a code path in the program where the program handles the situation).

Mana is not much more than a rough sketch at this point. In terms of (sub-)projects that are actually useful right now, there is the Voodoo programming language, which has a compiler, as well as a few (as yet unreleased) projects building on it: Antimony, which is basically a set of libraries and utilities to build compilers, and Diamond, which is supposed to become a Ruby-like language with a compiler that can handle enough of the language that I can compile the Voodoo compiler to native code. That's what I'm currently working on, whenever I have time and energy.

In case you would like to discuss about these projects some more, please contact me and we can continue the conversation by email.

Thank you for your interest! I'm actually working on a number of projects simultaneously, but my big project is Mana. It's also the one that is the furthest away from being ready, because I want to use my experience with the other projects to make Mana really good - or at least avoid as many painful mistakes as I can.

The idea for Mana is to take a look at the programming languages already out there (if you take a look at my (outdated) resume, you will see that I know quite a few, and I've looked at more) and combine some of the good ideas into a single language. To give some examples:

  - Common Lisp offers extremely powerful abstraction mechanisms, effectively allowing you to grow the language to fit your task, until implementing your program becomes practically trivial. Garbage collection, higher-order functions, macros, and a regular syntax make this work, so Mana will have these, too.

  - Haskell and OCaml allow a lot of errors in programs to be detected at compile time. This is made possible by well thought-out type systems and static type checking. Numerous Haskell programmers have had the experience that, once the program makes it through the compiler, it does what you wanted it to do. I would like Mana to also have this ability. Perhaps this can even be extended to handling of errors that can occur at run time, so that a program that makes it through the compiler is guaranteed to not crash (i.e. everything that can happen invokes a code path in the program where the program handles the situation).

Mana is not much more than a rough sketch at this point. In terms of (sub-)projects that are actually useful right now, there is the Voodoo programming language, which has a compiler, as well as a few (as yet unreleased) projects building on it: Antimony, which is basically a set of libraries and utilities to build compilers, and Diamond, which is supposed to become a Ruby-like language with a compiler that can handle enough of the language that I can compile the Voodoo compiler to native code. That's what I'm currently working on, whenever I have time and energy.

In case you would like to discuss about these projects some more, please contact me and we can continue the conversation by email.

Thank you for your interest! I'm actually working on a number of projects simultaneously, but my big project is Mana. It's also the one that is the furthest away from being ready, because I want to use my experience with the other projects to make Mana really good - or at least avoid as many painful mistakes as I can.

The idea for Mana is to take a look at the programming languages already out there (if you take a look at my (outdated) resume, you will see that I know quite a few, and I've looked at more) and combine some of the good ideas into a single language. To give some examples:

  - Common Lisp offers extremely powerful abstraction mechanisms, effectively allowing you to grow the language to fit your task, until implementing your program becomes practically trivial. Garbage collection, higher-order functions, macros, and a regular syntax make this work, so Mana will have these, too.

  - Haskell and OCaml allow a lot of errors in programs to be detected at compile time. This is made possible by well thought-out type systems and static type checking. Numerous Haskell programmers have had the experience that, once the program makes it through the compiler, it does what you wanted it to do. I would like Mana to also have this ability. Perhaps this can even be extended to handling of errors that can occur at run time, so that a program that makes it through the compiler is guaranteed to not crash (i.e. everything that can happen invokes a code path in the program where the program handles the situation).

Mana is not much more than a rough sketch at this point. In terms of (sub-)projects that are actually useful right now, there is the Voodoo programming language, which has a compiler, as well as a few (as yet unreleased) projects building on it: Antimony, which is basically a set of libraries and utilities to build compilers, and Diamond, which is supposed to become a Ruby-like language with a compiler that can handle enough of the language that I can compile the Voodoo compiler to native code. That's what I'm currently working on, whenever I have time and energy.

In case you would like to discuss about these projects some more, please contact me and we can continue the conversation by email.

Thank you for your interest! I'm actually working on a number of projects simultaneously, but my big project is Mana. It's also the one that is the furthest away from being ready, because I want to use my experience with the other projects to make Mana really good - or at least avoid as many painful mistakes as I can.

The idea for Mana is to take a look at the programming languages already out there (if you take a look at my (outdated) resume, you will see that I know quite a few, and I've looked at more) and combine some of the good ideas into a single language. To give some examples:

  - Common Lisp offers extremely powerful abstraction mechanisms, effectively allowing you to grow the language to fit your task, until implementing your program becomes practically trivial. Garbage collection, higher-order functions, macros, and a regular syntax make this work, so Mana will have these, too.

  - Haskell and OCaml allow a lot of errors in programs to be detected at compile time. This is made possible by well thought-out type systems and static type checking. Numerous Haskell programmers have had the experience that, once the program makes it through the compiler, it does what you wanted it to do. I would like Mana to also have this ability. Perhaps this can even be extended to handling of errors that can occur at run time, so that a program that makes it through the compiler is guaranteed to not crash (i.e. everything that can happen invokes a code path in the program where the program handles the situation).

Mana is not much more than a rough sketch at this point. In terms of (sub-)projects that are actually useful right now, there is the Voodoo programming language, which has a compiler, as well as a few (as yet unreleased) projects building on it: Antimony, which is basically a set of libraries and utilities to build compilers, and Diamond, which is supposed to become a Ruby-like language with a compiler that can handle enough of the language that I can compile the Voodoo compiler to native code. That's what I'm currently working on, whenever I have time and energy.

In case you would like to discuss about these projects some more, please contact me and we can continue the conversation by email.

Thank you for your interest! I'm actually working on a number of projects simultaneously, but my big project is Mana. It's also the one that is the furthest away from being ready, because I want to use my experience with the other projects to make Mana really good - or at least avoid as many painful mistakes as I can.

The idea for Mana is to take a look at the programming languages already out there (if you take a look at my (outdated) resume, you will see that I know quite a few, and I've looked at more) and combine some of the good ideas into a single language. To give some examples:

  - Common Lisp offers extremely powerful abstraction mechanisms, effectively allowing you to grow the language to fit your task, until implementing your program becomes practically trivial. Garbage collection, higher-order functions, macros, and a regular syntax make this work, so Mana will have these, too.

  - Haskell and OCaml allow a lot of errors in programs to be detected at compile time. This is made possible by well thought-out type systems and static type checking. Numerous Haskell programmers have had the experience that, once the program makes it through the compiler, it does what you wanted it to do. I would like Mana to also have this ability. Perhaps this can even be extended to handling of errors that can occur at run time, so that a program that makes it through the compiler is guaranteed to not crash (i.e. everything that can happen invokes a code path in the program where the program handles the situation).

Mana is not much more than a rough sketch at this point. In terms of (sub-)projects that are actually useful right now, there is the Voodoo programming language, which has a compiler, as well as a few (as yet unreleased) projects building on it: Antimony, which is basically a set of libraries and utilities to build compilers, and Diamond, which is supposed to become a Ruby-like language with a compiler that can handle enough of the language that I can compile the Voodoo compiler to native code. That's what I'm currently working on, whenever I have time and energy.

In case you would like to discuss about these projects some more, please contact me and we can continue the conversation by email.

It's interesting that nobody else seems to have said it yet, but here goes:

What I avoid when working on open source projects is packaging. I'll do design. I'll write documentation. I'll write the code. I'll find and fix bugs. I'll communicate with users. I'll make it as easy as I know how to to get the program working on your system.

But I avoid making a package for any one particular system.

It would probably greatly improve how easy it is to get started with a particular program, but I still can't seem to get myself to do it. Whenever I create a package, it always feels like I'm taking sides, like the platform I'm making a package for is somehow more favored by me than other platforms. And the few times I have created packages, they have not been included in any of the canonical repositories. So nowadays, I pretty much don't bother anymore.

Even though I think that, detach, in particular, would really make a great addition to every Unix-like operating system.

``and random restarts of Windows 7 was my idea!''

Does this satisfy your need? Windows 7 in 7 Screenshots.

``Have a look at the Go Language FAQ. It states very clearly why exceptions were not added to Go.''

Yes, I've read that since. I hadn't bothered to read the whole thing before posting. :-)

It seems to me the reason Go doesn't have exceptions is that "we haven't figured out a good way to implement them yet".

``Have you ever considered, from an assembly language perspective, what exception processing looks like? How handling of multiple stack frames might take place?''

As a matter of fact, yes. I am the creator of the Voodoo programming language, a low-level language intended to be used as an intermediate step to native code generation. I am currently working on mapping the constructs of my Mana programming language (which, alas, is mostly documented on paper as of now) to Voodoo. That includes exceptions, restarts, continuations, closures, tail calls, garbage collection, the whole enchilada. Exceptions aren't exactly straightforward to implement (they break call-return control flow), but it's work you only have to do once and that will save users of your language a lot of headaches. "It's somewhat difficult to implement" has never been a good reason not to have a good idea in a programming language, especially if it _has_ been done before.

``If you want languages that provide safety then you must sacrifice speed.''

Is that so? Care to back up that assertion with some argumentation?

``I, personally, have never found myself enamoured by exception handling. It has its conveniences but you may well find many old school programmers find themselves content without it.''

Oh, I manage just fine in languages without exception handling, too. I'd also manage just fine if all I had was an assembler. In fact, I've done just that for years (back when there wasn't Internet access where I lived). But that doesn't mean I don't think high level languages or exceptions aren't a good idea.

If reduced playback support is your issue, you should not be satisfied with Windows Media player as Microsoft ships with Windows, either.

Also, there is no need to turn the OS upside down to allow playing DRMed content. In the end, the bytes are there, you just have to figure out how to decode them. A regular program is enough to actually do the decoding once you know how.

I'll let you decide which is better, but this is the difference:

detach, as the name implies, detaches the command it runs from the shell and the terminal. There really won't be any association between them anymore. If you close the shell, your command won't receive a SIGHUP.

nohup sets the signal mask for your command so that SIGHUP is ignored, and redirects output to nohup.out. Other than that, it is just a normal command run under the shell. It's listed in jobs. It will also receive SIGHUP when your shell closes.

I'll be honest: I wrote detach because I hadn't discovered any of the alternatives that are listed on its webpage. But there's a reason I hadn't discovered them: they don't do the same thing detach does, and therefore didn't have the keywords I searched for listed with them.

What detach does, really, is to make sure that some program is running, much like the daemons that you will typically find running on a *nix system. In fact, it uses the same mechanism that most if not all daemons use. That is exactly what it does. None of the alternatives I have found do exactly that.

Of all the alternatives I have found, disown comes closest to doing the same thing as detach. Perhaps the end result is actually exactly the same (unlike with nohup); I don't know that for sure. But (1) disown is a builtin in some shells, not a command that can be used from all shells, like detach, and (2) you run disown _after_ starting the command...I, and various other users, find it more natural to type "detach command args" than to type "command args & disown", but that may be a matter of preference.

Finally, I don't like nohup, because it goes and creates files with fixed names (in particular, nohup.out). So even in situations where I could use it, I would prefer detach.

I hope that this clears things up a little. Also see the detach webpage. It lists some possible alternatives and explains how they differ from detach.

Note that I am the author of detach, so of course all of this is fueled by a desire to not have my baby be a useless imitation of some already existing command. ;-) But seriously, I really feel detach does exactly what I want and the others don't. Having said that, the reason I listed the alternatives is so that people can use those if they fit their needs, without having to go through the hassle of downloading, compiling, and installing detach. It's not packaged for any distros I'm aware of (I used to provide Debian packages and OpenBSD ports, but I just don't have the time anymore).

Note that detach isn't dtach. detach detaches the program it starts from the terminal. dtach creates a virtual terminal and attaches the program to that, instead of to the terminal dtach was run from. You can then attach and detach from that virtual terminal (although, unlike with screen, dtach doesn't remember previous output from the command).

dtach and screen are for commands that produce output, where you want to go away and come back to see the output at some point, and for interactive commands, where you have to produce input.

detach is for things like daemons and X clients, where you aren't interested in looking at the output. If you want, you can redirect the output to a file. You can also provide a file to be used as input, but you can't interact with the program yourself. The program really is detached from the terminal.

detach. Start programs and then just forget about them while they keep running. You'll get your prompt back and the program will just keep running in the background, even when you exit the shell you started it from.

``Where does the stupidity stop? When every single thing on the system has been rewritten in someones pet scripting language du jour? When every program requires its own interpreter?''

Because the perfect programming language isn't there yet. Therefore, people continue to create new ones, and others continue to find them useful.

As for your complaint about interpreter creep: I can see why that doesn't make you happy. I guess the reason many languages are interpreted rather than compiled to native executables is that it's easier to write interpreters that run on a plethora of platforms than compilers that target a plethora of platforms (something which I am trying to remedy with TurboVM, Alchemist, and an upcoming low-level platform abstraction language, but this is all still under development).

On the other hand, the interpreter is just a bit of extra code you install that provides functionality a program builds on. In many ways, interpreters are like libraries. They could have shipped the interpreter with the executable, and then you wouldn't have noticed it as a separate piece to install, but that would have prevented reuse...and reuse is a Good Thing. Speaking of which, I believe C isn't particularly good for developing reusable components, so if a large chuck of your software is written in C, you might actually have more duplicated functionality on your system than if much of your software is written in a bunch of different higher level languages.

``Where does the stupidity stop? When every single thing on the system has been rewritten in someones pet scripting language du jour? When every program requires its own interpreter?''

Because the perfect programming language isn't there yet. Therefore, people continue to create new ones, and others continue to find them useful.

As for your complaint about interpreter creep: I can see why that doesn't make you happy. I guess the reason many languages are interpreted rather than compiled to native executables is that it's easier to write interpreters that run on a plethora of platforms than compilers that target a plethora of platforms (something which I am trying to remedy with TurboVM, Alchemist, and an upcoming low-level platform abstraction language, but this is all still under development).

On the other hand, the interpreter is just a bit of extra code you install that provides functionality a program builds on. In many ways, interpreters are like libraries. They could have shipped the interpreter with the executable, and then you wouldn't have noticed it as a separate piece to install, but that would have prevented reuse...and reuse is a Good Thing. Speaking of which, I believe C isn't particularly good for developing reusable components, so if a large chuck of your software is written in C, you might actually have more duplicated functionality on your system than if much of your software is written in a bunch of different higher level languages.

muhttpd uses fork, yet it works on Windows, thanks to the excellent work of the Cygwin project (at least earlier versions of muhttpd worked under Cygwin; I haven't tested recently).

That reminds me of something I thought of a couple of months ago (or is it years by now?). It seems to me that a lot of people who had a great impact on the world have had beards. Does anyone have any theories as to why that might be?

By the way, I have a beard, too. And I'm working on a programming language.

Not knowing why Mersenne primes matter, I looked it up on The Ultimate Source Of Truth. From The Fine Article:

Many fundamental questions about Mersenne primes remain unresolved. It is not even known whether there is a largest Mersenne prime, which would mean that the set of Mersenne primes is finite. The Lenstra-Pomerance-Wagstaff conjecture asserts that, on the contrary, there are infinitely many Mersenne primes and predicts their order of growth. It is also not known whether infinitely many Mersenne numbers with prime exponents are composite, although this would follow from widely believed conjectures about prime numbers, for example, the infinitude of Sophie Germain primes.

Mersenne primes are used in pseudorandom number generators such as Mersenne Twister and ParkMiller RNG.

Mersenne primes were considered already by Euclid, who found a connection with the perfect numbers.

Mersenne numbers are very good test cases for the special number field sieve algorithm

Out of those, I only knew about the connection with pseudorandom number generators, which I became interested in after writing my deadbeef random number generator.

So, yeah. It's lock-in and leveraging one's monopoly in one market to gain market share in another once again.

I would recommend Scheme. Not because I recommend you program in it, but because it will change the way you think about programming and programming languages, especially if you also read Structure and Interpretation of Computer Programs (affectionately known as SICP).

Scheme is a small programming language, so you can learn it relatively quickly. This allows you time to read some of the literature about Scheme. And SICP. This will give you insight in the design of programming languages, and how features (like loops, class systems, etc.) can be implemented in terms of simpler constructs, given the right primitives and powerful enough abstractions.

And, in the end, it's the primitives and abstractions that matter. Given the right primitives and abstractions, you can mold your programming language to be anything you want. Scheme, and other languages in the larger Lisp family, are very good at this. That's probably why they are still around, despite tracing their roots back to the 1950s. Other languages are not so good at it, and you will end up writing lots of boilerplate and repetitive code if you program in them.

I could go on and on and list all the insights Scheme has provided me with, but I'd say you go see for yourself. SICP is a good place to start, because it starts at the basics (so you don't need to know Scheme before you start), but gets up to speed really quickly (so you won't get bored). You could also try my Scheme tutorial.

Note that all this is about the 5th revision of Scheme (known as R5RS). R5RS lacks many primitives that would make it useful for Real World applications (just to name one thing, there is nothing about networking). Many implementations of Scheme (and there are many) provide various such primitives - but that's not the point here. The point is about the fundamentals of programming.

As a final remark, if you are used to Java's "everything is part of a class" curfew, Scheme will provide a refreshing change of perspective. R5RS doesn't even have classes. Instead, the focus is on procedures. So you will be decomposing your system in terms of what it does, more than in terms of what real, imaginary, or "forced on you by the programming language" objects it acts on. In my experience, this leads to many small functions, each doing a simple thing, which can then be composed to build your system - or perhaps a very different system that happens to re-use part of the functionality. Unlike Java's classes, each containing a bunch of method, each of which contains gobs of repetitive code...try to adapt _that_ to do something similar, but slightly different. Again, it's all about primitives and abstractions.

I have been working on a list of games that will run on non-Windows platforms. If anyone has games they would like to be added, please contact me. If you could include both a link to a website about the game and a short description of the game, that would be grand.

I have been working on a list of games that will run on non-Windows platforms. If anyone has games they would like to be added, please contact me. If you could include both a link to a website about the game and a short description of the game, that would be grand.

I've recently done some work on a list of games that do not propagate vendor lock-in. I suppose that, in normal English, that means "games you can run on YOUR system, whatever that system is", although it's not quite _that_ good. The listed games are games I've played and enjoyed myself, but there are links to many games. They can all (I think) be played on Linux/x86, and many of them work on other platforms, too. I don't know if it will satisfy your wishes, but it's a start. Comments welcome.

http://inglorion.net/documents/essays/vorbis-microsoft-apple/

Thanks for mentioning that. Your post has prompted me to finally set up a page for TurboVM, a virtual machine I have been working on. What makes TurboVM interesting is that it's (1) small and simple, (2) not tied into a programming model like typical VMs that are designed with a single language in mind, and (3) FAST.

There is a bytecode interpreter which seems to outperform OCaml's (widely considered fast) by about a factor 2 to 3, and a compiler that compiles bytecode to C which can then be compiled to native code and will run about as quickly as an equivalent program written in C to begin with (everything compiled with maximum optimizations). An assembler and a disassembler are also provided.

All this is in a pretty early stage of development, not very well tested (I think some of the signed operations are wrongly implemented), poorly documented, and lacking features (right now, programs are just instructions...I want something with sections, labels, linkable libraries, etc.), and so on, but you can download it now and it compiles and runs. I hope some of you will be as excited about it as I am. :-D

You can do this with Mailvisa (and, I reckon, most other spam filters that use machine learning).

With Mailvisa, it works like this:

1. You read an article
2. You decide if you liked it or not
3. You put it in a "good articles" directory if you liked it, and a "bad articles" directory if you didn't
4. Once you have collected a number of articles, you train Mailvisa on them
5. Run every new article through Mailvisa. If it says it's spam, you probably don't want to read it
6. Repeat from step 1

Other Bayesian filters vary in the details. With Mailvisa, you want to only train once in a while, because it takes annoyingly long to load and store the databases. With other filters, this may not be the case, so you can train after every message. Some filters should only be trained with messages they classified wrongly (perhaps this goes for Mailvisa, too, but I get 100% precision (less than 1 message in 1000 wrongly classified as spam) and recall in the upper 90%s by training on all messages).

It would be nice if this functionality were integrated with other software (e.g. mail clients, RSS readers, etc.). Mailvisa is a stand-alone program and is not specific to filtering email, but it also isn't integrated with any other software that I am aware of. It would be much nicer if, when viewing a message, you could press a button to indicate that this message was good, or another button to indicate that this message was bad, and the filter of your choice would then be trained with that information.

It is my understanding that chroot _can_ be an effective security tool. It just has its limitations. One such limitation is that processes run as root can break out of the "chroot jail" (why is this, by the way?). Another limitation is that you have to be root to use chroot (and why is this?).

I wrote a little program called chrootexec which allows you to run commands in chroot jails as a normal user, circumventing both these limitations. To do this, it must have the capability to do chroot, setuid to the real user id, and setgid to the real user id; this can be accomplished by installing it suid root.

The question is what sandboxing really solves. Supposedly, we already have processes in isolated address spaces. They interact with the rest of the system through interfaces exposed by the operating system. You can sandbox all you want, but, eventually, you are still going to have some interaction between the sandboxed process and the rest of the system.

I see much more value in writing software in languages where the now common types of exploits can't occur. If we can stop programs from wrongly referencing memory, i.e. going outside the bounds of allocated data structures, that will be a great step, eliminating many current vulnerabilities. And we can do that...we just have to not use the unsafe constructs from C (and not replace them by other unsafe constructs, obviously).

I have a hard time imagining why you would think there could be things that could not be done remotely.

As others have pointed out, you can do a lot of things (I would say every kind of maintenance) remotely over SSH. That basically allows you to do everything that doesn't require a graphical user interface. If you do need the graphical user interface, you're in luck, though. One of the hidden strengths of Unix is that GUI is provided by X, which can be accessed over the network. A convenient and secure way to do this is by tunenling it through SSH (try ssh -X user@host xterm, for example). Even if that isn't enough (e.g. because you're on a machine without an X server), you can even access your desktop through RFB.

Of course, you can't perform any maintenance that requires physical access to the machine remotely. However, in all my years working with *nix systems remotely, I have never needed physical access.

I just recently started a page linking to linux laptop vendors. Check it out, and if you have anything to contribute, feel free to reply to this post.

static uint32_t deadbeef_seed;
static uint32_t deadbeef_beef = 0xdeadbeef;
 
uint32_t deadbeef_rand() {
        deadbeef_seed = (deadbeef_seed << 7) ^ ((deadbeef_seed >> 25) + deadbeef_beef);
        deadbeef_beef = (deadbeef_beef << 7) ^ ((deadbeef_beef >> 25) + 0xdeadbeef);
        return deadbeef_seed;
}

static uint32_t deadbeef_seed;
static uint32_t deadbeef_beef = 0xdeadbeef;
 
uint32_t deadbeef_rand() {
        deadbeef_seed = (deadbeef_seed << 7) ^ ((deadbeef_seed >> 25) + deadbeef_beef);
        deadbeef_beef = (deadbeef_beef << 7) ^ ((deadbeef_beef >> 25) + 0xdeadbeef);
        return deadbeef_seed;
}

``The BBC said they are going to look at other platforms later. They are just making downloads available to the vast majority of the people who paid for it first, this is normal. ''

Normal in that it is common practice, perhaps, but that doesn't mean there isn't a problem with it. The problem is that they are (currently, and that's why there _currently_ is a problem) making downloads available only in a proprietary format. This incurs all the problems with proprietary formats, including, but not limited to locking out would-be users, no guarantee of future access to data, and preventing the great capitalist mechanism of competition from doing its work.

Their decision to use Microsoft's proprietary formats puzzles me, because the BBC has often been in the news for actually using open formats, and they used to actually work on an open format and player, themselves. Why did they decide to go for a proprietary format in this case?

One for Windows:

  - Almost everyone knows how to use it, so it's easy to find somebody to help you if you can't figure out something

I'll also take this opportunity to pimp my essay Hidden Strenghts of Unix. It mostly covers customizability, the shell, and X. Applies to Linux and OS X.

I've been working on something like this for a while. Basically, the idea was to represent common tasks in various programming languages. The tasks are things like performing some operation on every element of a sequence, doing file I/O, defining recursive functions, etc.

Some of what I made is up on http://inglorion.net/documents/plcomparison/. Eventually, I got tired of writing the C and Java programs. On my home machine, I have some more examples; like an implementation of the filter (AKA delete-if-not) function, a math formula internal-representation-to-text translator, and a TCP echo server, but I still have to write some of the programs, as well as the documentation for these.

Anyway, just thought I should share. :-)

Blame goes to both the programmers and the language. Ultimately, it's the programmer's fault if they write insecure code. Even in the very exceptional case where the language does not allow secure code to be written (which I think has been true of PHP a few times), the programmer can still be blamed for choosing to write code in the language. So the programmer is always at fault.

Having said that, the language certainly has an influence on what (security) bugs can be made or are likely to be made. As I've explained in some detail in my essay Better Languages for Better Software, many common vulnerability classes can be eliminated by providing the right language features and APIs. The typical vulnerabilities in PHP scripts (injection vulnerabilities) fall in that category. So PHP is definitely at fault, too.

``I thought DieHard was a random number generator test suite.''

So it is. Speaking of which, does anyone here know how to interpret the numbers it generates? I ran it on the deadbeef random number generator a while ago (test results linked from that page), and my interpretation is that deadbeef_rand does well on some tests and very poorly on others. Am I right? Can one distill from DieHard's output what the weaknesses of the PRNG are?

Oh well. I never actually look at the files that are put in my home dir; there's too much crap there anyway. And dotfiles are doubly evil, because they both clutter the home dir and get in the way when you don't want them to, and are hidden and often ignored, so that you forget to process them when you need to (e.g. when making backups).

I put everything under three subdirectories of my home directory: local, private, and public (with appropriate permissions), with some frequently accessed things having symlinks there, as do files that are expected to be under $HOME, such as dotfiles. public and private are shared among multiple computers, whereas local (which contains lots of temp files, as well as projects which actually are shared, but through subversion) is not.

uint32_t deadbeef_rand() {
    deadbeef_seed = (deadbeef_seed << 7) ^
        ((deadbeef_seed >> 25) + deadbeef_beef);
    deadbeef_beef = (deadbeef_beef << 7) ^
        ((deadbeef_beef >> 25) + 0xdeadbeef);
    return deadbeef_seed;
}

There is a solution to injection vulnerabilities, but it's not validation. Sure, if you validate everything properly, you won't suffer from injection vulnerabilities. However, writing the code for that is cumbersome and error-prone, and thus, in practice, we see that values are not or not properly validated.

The solution I've been championing is structured composition. Instead of verifying that the input won't alter the structure of whatever you're composing, you use APIs that ensure that this won't happen. Some examples of this, as well as other bug-eliminating language/library features, can be found in my essay Better Languages for Better Software.

you have to explicitly indicate if you want to access global variables from inside a function

For simple things, I can live with PHP's warts, but when writing larger or more complex applications, I wish for a Real programming language.

I wrote my own Bayesian filter, Mailvisa, to gain a better understanding of how Bayesian filtering works, and to be able to tweak the parameters. When I last measured it, it caught 93% of spam. Of all the filters I tried at the time (I think it was all filters in Debian sarge), only Bogofilter scored better. This applies to both the amount of spam caught and the filtering speed. The closest thing to false positives I've gotten over the years were a few advertisement mails from my domain registrar.

I have only two problems with it: 1. I have to train it regularly, and 2. nowadays, lots of mail slips through, because it contains words related to programming languages.

Assuming you meant the link to my essay, it's right in my original post, but here goes again: Better Software Through Better Languages. It doesn't actually contain the source for the macro, but the following works:

(defmacro query (&rest parts)
        (format nil "~{~A~}" (mapcar (lambda (part)
                (if (stringp part) part (sql-escape part))) parts)))

where sql-escape should return an appropriately escaped string.

However, I guess it would be a better idea to just use CL-SQL.

``Why are the point and click or turn key solutions so vulnerable to SQL injection in the first place? I had a friend with a PHPBB site that got shot to all hell when some cracker came along and defaced it. Why wasn't it secure out of the box?''

With PHP, virtually anyone can write a web forum. And lots of people do. Unfortunately, not all these people are good coders. Not all of them are aware of the security risks. And PHP doesn't provide particularly secure APIs for SQL (or at least it didn't when I last checked). With better languages (or APIs), injections could be made impossible.

I've written about this before. Basically, SQL injection vulnerabilities would completely disappear if better APIs were used. The problem is that queries are composed as strings, which have no intrinsic structure. The programmer creates structure by inserting certain characters (in particular, single quotes) in the string. However, the exact same mechanism is used to add user-supplied data to the queries. Unless the programmer is very careful, this allows the user to affect the structure (and thus the effect) of the query. An example is in the essay I linked to.

Is that like saying "don't click here"?

``how well does Apple support Play's For Sure devices on its platforms? I'd wager less well than MS supports Apple products...''

Apple won't license FairPlay to anyone, so their DRMed music plays only op Apple products (which they do make for Windows). The Zune (according to what I've read) will use something _other_ than PlaysForSure as its DRM, something that Microsoft doesn't seem willing to license. It remains to be seen if they'll be releasing a player for Mac OS X. However, that's software.

When it comes to hardware, iPods won't play Microsoft's DRMed files, and the Zune won't play Apple's DRMed files. Other players probably won't play either of them. At least, that's the way it looks now. This is a change from Microsoft's earlier position, where they were licensing WMA and PlaysForSure to player manufacturers.

So, the way it stands is: you can buy music from iTunes and play it on an iPod, but not on any other player. You can buy music from one of the stores that sells WMA music (e.g. Napster) and play it on one of the devices that support WMA (not the iPod, and not all devices besides the iPod). AFAIK, there is no store that sells music for Zune yet, but when there is one, that music will play only on the Zune. As you can see, it's all one big mess of incompatibilities. So people will just use MP3 for interoperability.

The one format missing from the discussion is Vorbis, which provides sound quality superior to all of them, and is supported by a growing number of players (including most software players, but also hardware players, e.g. from Samsung), but not by Microsoft or Apple's offerings.

See also my essay on the subject. It's not completely finished, and it will probably need updating soon, but at least I won't have to repeat everything here.