Slashdot Mirror
Swarm — a New Approach To Distributed Computation
An anonymous reader writes "Ian Clarke, creator of Freenet, has been working on a new open source project called Swarm. The concept is to allow a computer program to be distributed across multiple computers in a manner almost completely transparent to the programmer. The system observes the program executing and figures out how the workload should be distributed for maximum efficiency. Swarm is implemented in Scala. Its at an early-prototype stage, and Ian has created a good 36 minute video explaining the concept and the current implementation."
.. was Mosix http://www.mosix.org/
It allowed mosix-running linux computers to distribute their loads over a connected other mosix-running linux computers.
Processes migrate to other nodes transparently. No programming changes were needed.
A friend of mine did a system like this about ten years ago --- hi, Iain! --- called Flit. It had a number of the same features, although using a custom language; it had some rather interesting concepts, such as asynchronous function calls that would return immediately, spawning a new thread, but return a future: a value whose value was not known yet. Accessing the value would cause the thread to be waited upon.
Unfortunately the killer problem that sunk Flit was that of distributed garbage collection. Collecting data over multiple machines is really, really hard, and he never found a usable approach to make it work. I was very disappointed to see that Swarm's garbage collection is still on the to-do list --- he doesn't appear to have started to think about it yet.
I hope he can make Swarm work --- it's something that we could all definitely use. But there are fundamental theoretical problems that have to be solved first...