OpenMosix

Francesco Taurino writes "Moshe Bar has released a new Mosix system: openMosix. From the site: "For thousands of users Mosix has been a reliable, fast and cost efficient clustering platform. There are hundreds of Mosix installations in life sciences, finance, industry, high tech, research and government environments. The goal of openMosix is to give to these users a continued support and an up-to-date platform. openMosix is initially fully compatible with the last Mosix (1.5.2 for 2.4.13) kernel, but is now growing in its own direction. If you would like to contribute to the openMosix project, drop a line to moshe@openmosix.org.""

Comments

1.5.2 isn't latest MOSIX

[NicolaiBSD]

MOSIX 1.5.7 for Linux 2.4.17 (K-MOSIX) is out, according to Freshmeat. Therefore "the last Mosix (1.5.2 for 2.4.13) kernel" seems incorrect.

I never heard of mosix,

[whovian]

and since information is a bit lacking at the link provided, here's a link to the regular mosix FAQ.

Re:Building a mosix cluster

[Hamshrew]

I don't have a link handy, as it's been a few months. I found it to be fairly simple to install... I had 4 PIII machines, set them all up on an internal network and nfs mounted a directory from the head. From there, it was a simple series of steps:

Unpack kernel sources.
Run the Mosix install script.

Did that on each node, then started the mosix service on each.

It worked like a charm for large computations, but had three flaws for normal use.

1) By default, it does not auto-migrate, which was pretty dumb. And getting it to auto-migrate was buried deep in the docs, though it could be guessed from reading up on locks. (echo 1 > /proc/self/lock, I think it was)

2) Migration only occurrs after a certain load average is maintained... if your job involves spawning multiple short-lived processes, like a large compile, it doesn't migrate anyway.

3) Network usage for migration was very heavy over Fast Ethernet.

There you have it. It's the last reason that MOSIX isn't used often in commercial clusters, but it seems well-suited for other distributed computing applications, and has some interesting features, especially for NOW configurations.

What MOSIX can't do

[Anonymous Coward]

http://www.ai.mit.edu/lab/sysadmin/cluster.html

There are several limitations to what MOSIX can
currently do. Java native-threads is one thing,
since MOSIX cannot migrate apps using shared
memory all native-threads applications will stay
on the node they started on, green-thread
based application can migrate though the
internal threads aren't exposed to the OS so no
real parallelism is achieved. MOSIX also can't
migrate sockets so I/O bound problems also
stay at home. Mixed I/O and CPU jobs can
migrate for CPU cycles, but are brought back for
the I/O ops. In the limited testing to date,
processes that can't take advantage of MOSIX
don't seem to be hindered at all by it

Re:What MOSIX can't do

[Hooya]

i'm been working with a 3 node test cluster (mosix) for viability for our apps. we tried mosix first since we don't want to write to a specific clustering API. we already have a ton of programs and introducing a new API into all of those would be a massive headache. plus a number of our developers dont' know much about parallelisms and the headaches it brings to the table.

with mosix, you just 'fork and forget'. that's simple enough. but here are some of the issues that i've run into and hope it's useful for someone else since it took me about a whole week to figure it out.

Sockets don't migrate. fileHandles dont' migrate. ie. for any I/O to occure, the process is brought back home, then upon completion of the I/O, if appropirate it is transparently sent out. what that means is that if the ratio of i/o to other normal crunching stuff is low, the process is probably going to spend most of it's time at home. what i'm going to do next is test if a process migrates at all by just having an open file handle without doing any I/O. i might post it as a follow-up to this comment. if i have time, i'll also play around with mfs. but probably not. if someone has more info regarding I/O using mfs i would appriciate much. ie. if mfs is used to do file I/O can the process still migrate while doing the I/O? apart from file sharing, what other benefits does mfs bring?

Another thing i've tested is java. as you might already know, native-thread java uses shared memory so that can't be migrated. i was using sun's VM so i turned on the '-classic' switch to use green threads. it migrates. for 5 instances of the JVM, on (3) pentium MMX boxes, i had a 70% (roughly) performance gain (timewise). for about 20 instances of the JVM (i know, that's a waste but more on that later..) i had about 47% gain in time. and i assume it will reduce drastically after that for just spawning that many JVM vs. doing actual work.

to optimize, what i'm going to do is first figure out how many nodes are running. then spawn that many JVMs and split the job between those VMs. and hope mosix sends out each VM to a seperate machine. this is not much different from running a rsh call (or ssh for the security minded folks) except, we don't have to figure out the load on each box we run the command on and we don't have to worry about collecting the output at the end of the run. if someone has a better idea about running java on mosix, i'd love to hear about it. just post your ideas as a reply to this message. i'm sure others will find it useful too. with the mailing list going away and no real documentation for mosix... bummer. also, has anyone tested it with apache? i don't know much about apache but know that it uses shared mem. so can the process forks of apache be migrated? + there's the issue of open sockets from a specific IP. how does that play into this. i've read stuff on web sites that say they're running apache on a mosix cluster. is it one of those 'ahh... i want a beowulf of these' nuts? or is that legit?

Re:how does mosix deal with dead cluster members?

[Halo5]

Nothing. The cluster will live on, because it is actually a peer-to-peer system (not client/server). This is in the docs somewhere.

However, if a node dies abruptly, the job may be lost (I'm not sure here, because it hasn't happened to me yet). Logically, the submitting node should see that the node has died and should re-submit to a good node. Anyone have a difinitive answer on this?

FAQ from Website

[lw54]

Before it gets slashdotted by everyone trying to figure out WTF OpenMosix is, here's the FAQ from their website.

Can I mix Mosix and openMosix nodes in the same cluster?

No. Just like the older Mosix, you should not mix nodes because the protocols are subject to un-announced changes from version to version. On top of that, every new version has bug fixes which warrant updating to the new kernels.

Whois the copyright holder of openMosix?

All the old Mosix code is copyright by Prof. Amnon Barak of Hebrew University of Jerusalem. All new code of openMosix is copyright 2002 by Moshe Bar, Tel Aviv.

How do I upgrade to openMosix?

openMosix maintains for now compatibility with the user-land tools of Mosix 1.5.2. I also have a port to openMosix of the user-land tools which will be released soon. To upgrade to openMosix, simply download the openMosix patch from www.openmosix.org and apply the patch with

patch -Np1 < openMosix1.5.2moshe

to a stock Linux kernel of 2.4.14 or 2.4.16 respectively. Make sure to get your old .config file (the .config file remains compatible) and recompile your kernel and modules. Then, reboot.

Is openMosix a fork of Mosix?

Right now, it is a pure fork. Eventually, it will become a separate clustering platform with its own distinct feature set and behaviour.

Re:MOSIX question

[Dr.Dubious DDQ]

From my brief experimentation with Mosix and a bit of reading, this sounds correct.

Basically, mosix is a very "chunky" sort of clustering - it works on the level of "whole processes". Because of this, you don't need to write your software to do the splitting and migrating yourself as you do with "less chunky" pvm and mpi. On the other hand, a process split off from a pvm program running can be handled by mosix like any other process and migrated to the cpu that mosix thinks can get the process finished fastest.

Mosix seems like an ideal way to 'lend' processing power to slower machines. This is what I was doing when I played with it previously - I had a K6/2-350 and a P-100 laptop with no L2 cache. I got Mosix set up on them both and used a command-line mp3 encoder as a benchmark. On the P-100, encoding speed was about 15% of realtime. On the K6/2, it was about 110%. Running Mosix between the two over 10Mb Ethernet, I could encode mp3 at about 85% - I suspect it'd have been significantly closer to 100% if I'd had 100Mb Ethernet at the time...

Hopefully OpenMosix will keep up with current kernel versions better. Better still, maybe they'll be able to get it merged into 2.5 at some point...