Slashdot Mirror
OpenMosix
Francesco Taurino writes "Moshe Bar has released a new Mosix system: openMosix.
From the site:
"For thousands of users Mosix has been a reliable, fast and cost efficient clustering platform. There are hundreds of Mosix installations in life sciences, finance, industry, high tech, research and government environments. The goal of openMosix is to give to these users a continued support and an up-to-date platform. openMosix is initially fully compatible with the last Mosix (1.5.2 for 2.4.13) kernel, but is now growing in its own direction.
If you would like to contribute to the openMosix project, drop a line to moshe@openmosix.org.""
OpenVMS has the most robust, and fastest clustering ability I have ever seen . I just cant wait until linux can do this.
This page left intentionally blank.
Umm.. just out of interest, and not that I suspect anything, but can I ask who your empleyer is? :-)
This is a good thing. Which is what me must all understand. It's good to see linux moving forward in such a good way. With open mosix linux and it's accompanying software is becoming more open which is a good thing. It's good to see this. Go linux!
From the Mosix home page:
Only Prof. Amnon Barak is authorized to represent MOSIX.
...???
What's the story here, Moshe?
Napster-to-go says "Fill and refill your compatible MP3 player", which is a lie. It's not MP3. It's WMA with DRM.
Yeah I wonder who your boss is cause your so full
of **it that I'm stil sitting here with an incredulous look on my face.
--B
Imagine a beowolf cluster of these!!
Oh yeah, this is mosix, bummer.
"And we have seen and do testify that the Father sent the Son to be the Savior of the World"
1 John 4:14
I just finished implementing my 6 node MOSIX cluster, and I ran across several bugs, and I couldn't find any place to report them. The MOSIX development list is closed subscription, and apparently the good Professor ignores his email.
I'm not clear about some things though... How is MOSIX currently licensed? Why are they being so closed about development?
I've had enough abrasive sigs. Kittens are cute and fuzzy.
MOSIX 1.5.7 for Linux 2.4.17 (K-MOSIX) is out, according to Freshmeat. Therefore "the last Mosix (1.5.2 for 2.4.13) kernel" seems incorrect.
I was thinking about building a mosix cluster, just to see how it works :)
Anyone have some good links for a howto? Or anyone want to share their experience?
What a stupid name. Makes me think of squirrels for some reason =P
of course it is not M$, since their PR people are waaaay smarter than who wrote this troll ;-)
"Contrary to popular belief, UNIX is user friendly. It just happens to be selective on who it makes friendship with"
and since information is a bit lacking at the link provided, here's a link to the regular mosix FAQ.
To-do List: Receive telemarketing call during a tornado warning. Check.
While we are on the subject
No, we aren't on a subject that has anything remotely to do with GUI's.
We are talking windows 3.0 quality here.
It's just not true. Windows 3.0 was more of a window manager than a gui, and it was the first version of windows that actually worked. KDE has been rated more simple than windows xp, and gnome is a lot nicer.
"And we have seen and do testify that the Father sent the Son to be the Savior of the World"
1 John 4:14
He works for Transmeta.
Does anyone else get a hard-on when they look at stuff like http://www.mosix.org/pics/index.html this?? OMG. I wish I had 40 boxes in my basement to do my evil bidding. Mwhahaha...
what a troll! Please, don't use the terms "windows 3.0" and "quality" in the same phrase, it's an oxymoron. Besides, it's about choice (as in freedom of). But you wouldn't grok that, would you?
Seriously, the most cost-effecient systems don't have a gui.
.. that the Mosix site is running on a Mosix cluster to withstand the slashdot affect
"Never let the truth get in the way of a good story..."
I guess I wanna know why there was a fork. I respect both the big Prof and Moshe from what I have read of theirs. Moshe says that Mosix is going in other directions, which sounds kinda...vague ;)
-- Who is the bigger fool? The fool or the fool who follows him? --
or at the very least bump HZ up to 1024 (from 100) so Mosix/Linux can be used for realtime computing/MIDI.
what happens if a node in a mosix cluster dies? i've hunted through the docs to no avail.
US Citizen living abroad? Register to vote!
Well, I wanted to read the FAQ to find out what Mosix was, exactly, but apparently you have to be an admin to get to the FAQ. That sucks.
Does the Mosix process migrations work with open sockets (i.e., will the party on the other side notice a disconnect when the Mosix process is migrated?)
I've read that Mosix cannot migrate Java applications due to their use of shared memory for JITting JVMs. What other restrictions are there for process migration?
Essentially,
Basically,
Arguably,
Actually,
In fact,
Exactly,
that's because most computational environments out there (intger math, ieeeeeee FP) can't deal with infinity. Thus, letting a program get away with a 0 div would be incredibly stupid.
;-).
If you want 0 divs in your life, program/run your apps in MATLAB
(Yeah im being anal retentive)
"Contrary to popular belief, UNIX is user friendly. It just happens to be selective on who it makes friendship with"
http://www.ai.mit.edu/lab/sysadmin/cluster.html
There are several limitations to what MOSIX can
currently do. Java native-threads is one thing,
since MOSIX cannot migrate apps using shared
memory all native-threads applications will stay
on the node they started on, green-thread
based application can migrate though the
internal threads aren't exposed to the OS so no
real parallelism is achieved. MOSIX also can't
migrate sockets so I/O bound problems also
stay at home. Mixed I/O and CPU jobs can
migrate for CPU cycles, but are brought back for
the I/O ops. In the limited testing to date,
processes that can't take advantage of MOSIX
don't seem to be hindered at all by it
The AC above, besides being a troll, is an uninformed one at that. The legendary goatse.cx is gone, dummy...
I am just wondering if applications need to be specifically written for MOSIX like they do for PVM?
:)
If not, I may just have to try this one
Mosix is completely worthless. They talk about its ability to use existing MPI or PVM applications. However, since applications which use shared memory, sockets, message, passing, message queues, etc (read: ANY FORM OF IPC) will not migrate, there's absolutely no point in using it.
It will work with existing MPI applications, but it won't migrate them. It will migrate other processes around them. Greeeeat... sounds really useful.
There's a special place in hell for morons like you.
Can I mix Mosix and openMosix nodes in the same cluster?
No. Just like the older Mosix, you should not mix nodes because the protocols are subject to un-announced changes from version to version. On top of that, every new version has bug fixes which warrant updating to the new kernels.
Whois the copyright holder of openMosix?
All the old Mosix code is copyright by Prof. Amnon Barak of Hebrew University of Jerusalem. All new code of openMosix is copyright 2002 by Moshe Bar, Tel Aviv.
How do I upgrade to openMosix?
openMosix maintains for now compatibility with the user-land tools of Mosix 1.5.2. I also have a port to openMosix of the user-land tools which will be released soon. To upgrade to openMosix, simply download the openMosix patch from www.openmosix.org and apply the patch with
patch -Np1 < openMosix1.5.2moshe
to a stock Linux kernel of 2.4.14 or 2.4.16 respectively. Make sure to get your old .config file (the .config file remains compatible) and recompile your kernel and modules. Then, reboot.
Is openMosix a fork of Mosix?
Right now, it is a pure fork. Eventually, it will become a separate clustering platform with its own distinct feature set and behaviour.
I really appreciate the work that Barak has done with and on Mosix. But I also find it kind of odd that Mosix could be the "property" of one individual. I would assume that it was developed with public research grants and while the author was employed at a university. Graduate students probably have also contributed, and there probably have been bug fixes as well. So, maybe it isn't bad if there is a GPL'ed distribution of Mosix after all. The GPL regulates issues of ownership rather well.
As for a user-space implementation of Mosix, I think that makes sense, although it has its drawbacks as well. One of the problems with user-space implementations is that they tend to be less than transparent in practice. It also strikes me as somewhat redundant, since Condor has already gone the user space route. A userland Mosix would only make sense if it were free and open source (as opposed to Condor).
Altogether, I hope people won't get too upset at each other over this. Mosix is great stuff and Barak and his university have been generous in making it available freely up to this point.
I read about the possibilty of migrating open sockets from one machine to another using MOSIX? How is work coming in this area? I would assume you would need a machine (such as a router) in front of the machines in which processes with open sockets can migrate (with the help of a new dynamic NAT entry).
not
...and recommend it!
MOSIX is great for general-purpose clustering. Processes are scattered across a cluster automatically without having to modify the programs. No API is needed other than usual Unix-level process use and it allows parallel execution of any program, although full use requires a parallel program design.
I just wish that it would go in the official linux kernel, something like
CONFIG_MOSIX=y
It's a great chance that Linux doesn't only play catch up with other flavors of Unix - it can take the leadership and give us the ability to create clusters using the tools in the standard distribution!
I had been using mosix for quite a while until recently. The 2.4.13+ext3+mosix release didn't even work properly with all of my hardware, oddly my networking refused to work at all; other kernels had no problems, a kernel built with the same .config worked fine on another box.
:) I do of course, assume that source IP is the biggest stumbling block in migrating network-bound applications.
Anyway, I had noticed that 2.4.13 was sticking around awhile.. meanwhile, the page was being updated.. adding some information about a user-space version and the 'disclaimer' on the download page.
Mosix sounds like a good thing, but in reality it isn't very suitable for many of my common tasks. The biggest problem is the lack of support for programs using shared memory. Apache, Mysql, these do not migrate. Also, programs using Xwindows will not migrate well.. as they are network bound and will migrate back to their home node once they need to report the the Xserver. Basically, don't install mosix on 10 of your home machines and expect any kind of performance increase.. besides, you could probably toss out your dual 1ghz PIII, replace it with a 400mhz celeron and not notice a difference.
It would be nice if there was a transparent pseudo layer for things like X.. where all of the machines have their IP for communicating via mosix.. but all 'outside' communication would be made via mosix though a 'public' ip. That public IP would then be used to connect to the Xserver; hence, apps network bound could migrate easily as they would still have the same IP and (spoofed) Mac address. Basically it would be building a NAT router into Mosix for the idea of being able to migrate network-bound applications. It sounds more complicated then it is, but less complicated then it is to impliment.. and I probably don't make any sense, but I know what I mean
Moshe,
Is there a direct correlation to your book not being released? My local book store said it was 'canceled'.
I for one, hope that the OpenMosix project moves forward, and makes quick and timely releases. I would like to see an openMosix for 2.4.17 and the upcoming 2.4.18! Great work guys!
thanks for the great explanation of MOSIX's limitations.
Balancing the Load (HotWired).
The big difference would be that the NAT server would have to communicate with the MOSIX nodes whenever a process migrates from one node to another.
Each node would have to have a range of ports to be used (since, for example, many programs may want to use the same TCP or UDP port to operate).
For you Mandrake users, I head a project to include LTSP and Mosix on a Mandrake configured kernel; to package and explain in very easy terms the whole process, and then eventually release a stripped-down Mdk, geared towards education (edu-tech is pretty much my field) ala K12 LTSP. We call it The Mandrake Mosix Terminal Server Project. Check it out and lend a hand if interested. Thanks.
put the what in the where?
C_T
Insert here.
Breakfast served all day!
You may not need a NAT router in front for MOSIX open socket process migration:
Assume that all nodes in the cluster are in the same subnet (a safe assumption). Also assume that all IP traffic in the subnet can be seen by all machines in the subnet (promiscuous mode on the NIC). Just before a process migrates it broadcasts to the subnet of MOSIX nodes that it will move the open socket from x.x.x.x:p to machine y.y.y.y. The old machine previously hosting the process will simply IGNORE/discard all IP packets dealing with x.x.x.x:p and the new machine (which the process migrated to) will intercept all packets that are destined for x.x.x.x:p. The new MOSIX node at y.y.y.y then sends out spoofed IP packets pretending to be x.x.x.x:p. In this way, the TCP connection will never be dropped and no one is the wiser.
This would require a fairly extensive rewrite of the MOSIX IP stack, however. But it would be self-sufficient.
The problem with your scheme is that the old machine's port will be tied up as long as the migrated process continues to run. Thus, if the migrated process was running on port 80 - the old machine could not create a (new) server to accept connections on port 80.
does quake run on mosix?
and if not...
what is it good for?
We run Mosix, and have had jobs fail. If a node goes down that is running a process two things can (and have) happened. The process dies, or it restarts on another node. What determins this? I have no idea. I can only speak from observation.
It is suggested in the documentation that you have a large swap space on your disk to handle nodes going down. Perhaps with a cached copy of a process it will live on.
In any case, its not scientific but I thought I'd throw that in.
What if each MOSIX mode had several IP addresses assigned to it? That way, if one IP/port pair were tied up - the MOSIX node still has other IP addresses available for connections at the same port number?
I found out the news yesterday, and I've already applied your patch + XFS. We've ran this on 1.5.2 and we haven't had too many problems with this setup.
Coding wise the conflicts seemed trivial (and many times redundant). To minimize potential conflicts we don't use MFS and we don't use the debugger.
The only difficulty came when we started using Mosix 1.5.2, we had some issues where we get intermittent periods of "Too many open files" when a node goes down. Somehow we've avoided them for the past month, we think this may have more to do with AutoFS.
I'm wondering if you would like the diff from this? I'm also interested in helping with the DSM development and socket migration. I may be slow on the uptake but where can I start and help out?
Ahh... perl...
This poster is a bot. I've only just started to read through slashdot.org forums and this is the third time in the past week I've seen this exact same word for word rant about Linux having a bad filesystem.
You've all just responded to some moron's play toy. He's probably getting off right now reading your replies.
Moshe Bar has released a new Mosix system: openMosix.
The word "bar" is Hebrew for "free"... Free as in speech, not beer, believe it or not!
i figured that's what this story needed
QED
BSD is for people who love UNIX. Linux is for those who hate Microsoft.
Since Mosix clusters usually are in a closed network, could there be a unique IP number for EVERY Process? If so, the IP address could be tied to the process, instead of tied to the machine
i think Mosix tries to get a cluster-wide process id, so if it's 16bit long, it could be in the class-B form [net.net.pid.pid], instead of [net.net.host.host]
when a process is migrated, it would send a broadcast message to change ARP tables, so the same IP address would be sent to a different MAC address
it would be better than setting every ethernet port in promiscous mode, since that woudn't work in a switched network (or in myrinet, i think)
-Kz-
As a MOSIX user, I think that the MOSIX-newbies reading /. may be getting the
wrong impression about this project.
For starters, MOSIX is MUCH easier to install than both PVM and MPI. In a
nutshell:
Step 1: Install userland programs
Step 2: Recompile kernel w/ MOSIX patch
Step 3: Setup LILO and reboot
Step 4: Repeat steps 2 & 3 for other nodes
NOTE: In my case, I only installed userland programs on the first node.
Although MOSIX is peer-to-peer, which is good, I'm using it with a master node.
Eventually, I plan to set up a second master node for redundancy.
NOTE #2: Most of the install is taken care of by using the install script.
As an example, it took me half a day to set up my first eight-node MOSIX
cluster, and 90% of that time was spent on the first node.
On User-MOSIX vs. Kernel-MOSIX: I think we may be missing an important point
here, as well:
It may be possible to incorporate U-MOSIX in places where K-MOSIX is not
possible (access to source code is required with K-MOSIX!). However, I do
think that a bit of salesmanship may be going into the description of U-MOSIX:
From the mosix.org website:
"U-MOSIX is better tuned for cluster and GRID computing, including the ability
to handle large number of short processes, run in heterogeneous clusters, with
different versions of Unix such as FreeBSD, Linux and Solaris"
How so? K-MOSIX is designed for hetero clustering. Large number of short
processes? The only way I can see this is with TCP/IP apps and socket
migration or with one helluva Myrinet connection between nodes. I can see the
advantage in *nix compatibility with userland MOSIX, but clustering
capabilities? I'll believe it when I see it. One of the principle advantages
of MOSIX that I have seen over Condor and the like is that MOSIX is
kernel-level. I've done my share of benchmarks, and nothing comes close.
The statements on the mosix.org website lead me to believe that U-MOSIX is
headed for commercial space. I hold no affiliation with either Professor
Barak or Moshe Bar, but I will say this: If Prof. Barak can produce
user-level code that outperforms K-MOSIX, then he deserves commercial
compensation and I will be the first in line to order.
I think that both gentlemen have done an outstanding job in producing a tool to
be used by researchers around the globe. My greatest hope is that this
development will spur competition (innovation is a better word, but WAY
overused!) between the two organizations.
Political Sidenote: I do feel that public educational funding should == open
source, but federal research funding is a different matter. This type of
funding is typically project-based, and IMHO the licensing of such should be
the decision of the government involved. I would, however, like to know the
specific problem that Professor Barak has with the GPL (assuming that the
rumors are true).
665: The mark on the forehead of Satan's slightly less evil brother, Stan.
That's a pretty good idea.
Moshe, are you reading this?
Think of the effect on switched networks. Switches have finite-sized ARP caches, so by artificially inflating them you increase the number of misses (where each miss equates to a broadcast since the switch doesn't know where to send the packet).
Not a serious problem, if your number of hosts (on your network, not just in your cluster) << number of MAC addresses supported by your switches (a few thousand, often). I suppose it does depend on how many "away-from-home" processes start per unit time.
Also, each ARP packet will itself be broadcast, causing more traffic up front. Again, this may or may not be significant. And each host has to process and cache the ARP.
If you are assuming that the MOSIX cluster will be on a dedicated network - probably reasonable - none of this is much of a problem.
"How can you claim that you are anti-crack, while still writing a window manager?" — Metacity README
Hmm... surely if the name "Mosix" is copyrighted then the name "OpenMosix" would infringe on that copyright.
The really sad part is that Alan was a huge Mosix supporter and contributed several patches for VM scheduling.
no, switches doesn't have ARP caches, they have MAC-Port tables. the ARP tables reside on the hosts.
the ARP-update packets (yet to be invented, i think) would include the new MAC address (or a special, host-bound IP address), that would be broadcast (bad, but only one packet), there wouldn't be a new ARP query.
in any case it woulbe FAR less traffic than the current moel of sending everything to the original node, just to be retransmitted. since in most scenarios every process would originate at the same node, it would easily saturate one leg of the switch.
a process-bound IP would allow network migration with just an extra broadcast packet for each migration (usually the migration needs to transfer a few megs per se)
-Kz-