Slashdot Mirror
K12LTSP + MOSIX Howto
Paul Nelson writes "Richard Camp posted a very complete, step by step guide to building a MOSIX cluster. "...The objective of this howto is to guide the reader on setting up a Mosix cluster with diskless nodes. The setup is based on K12ltsp Project. This should provide an easily scalable system."
No seriously, this stuff looks good.
smart posters that actually remember it's MOSIX not Beowulf?
--
If you moderate this, then your children will be next.
Why is this news? Yay. a new howto came out. That happens all the time. This would be a significant article on a high availability/high performance website, but on /.? Must be a slow news day.
Brian
hi there dontw e alerady have posicx? thewn whyh are we m,akeing a new poxix named moxix isnt' posox good enoujgh for evneryone? i like posix it si a goodl ibrary of fucntions and unix is cool mcuh bettre than windows at laest!! i wonder why buil gates didnt' maka widnows xp on unix cause thenm it woudl be more steable thanmit is now on vms cause unxi and pisix are betetr plateforms ofr opperaitng sytsmes than vmsa. dont'; you thinik so?
--frank
This is exactly where I feel Linux should be used. The idea of dumb terminals and a central server has proven to be the most cost effective way for companies to implement computer technology.
It's becoming clear that Intel/AMD etc are going to crush most other general purpose CPUs. Be it with SMP or SMT or both. With the increase in PCI bandwidth coming and the heralded 64bit chips intel will start to take over more and more server machines. Remember in the steel industry people scoffed at mini mills, kodak scoffed at digital cameras etc etc.
In the future most companies will have dumb terminals and a server room with racks of cheap intel boxes. The OS on the server will be fault tolerant to the max, oh I lost a node ahh well only 255 left. Uptimes measured in years. Hang on a sec that sounds like an IBM or SUN mainframe.
What is rapidly becoming apparent is that network speed is now more important than CPU/MEMORY speed.
This should provide an easily scalable system
Yeah, right. Like as if anybody who reads Slashdot is going to go "Cool! I'll go and build a Mosix cluster with diskless nodes now! I've always wanted an easily scalable system and just this looks like it might be it!".
Pushin' 'n dealin', shovin' 'n stealin'
The Mandrake Mosix Terminal Project is extremely similar and is based on the k12ltsp concept. Check it out if you can. K12ltsp is great for rolling-out massive amounts of LTSP servers quickly.
put the what in the where?
ClumpOS is a bootable CD with network drivers that is pre-setup with a custom kernel that contains MOSIX and MFS out of the box with no work required. You can download and burn ClumpOS and then boot it on your slave machines.
As far as building your MOSIX master goes, I prefer Debian with the prebuilt easy to deploy MOSIX packages and kernel patches. The links to find both are below:
Clump/OS: A CD-based mini distribution
MOSIX on Debian
MOSIX is a fun, extremely useful tool. Just remember when building your Debian kernel to make sure to turn ALL options on for MOSIX, this includes MFS. Otherwise, you will have weird problems with not being able to migrate processes to your cluster.
-Pat
I think this is where clustering should be done, for now at least, at the thread level. Most programs are multi-threaded. Most people don't want to rewrite programs to support MPI or PVM. Lots of projects that previously had to implement their own clustering protocols can just utilize Mosix instead. If I could talk my boss into it, I would put Linux/Mosix on every desktop at work and have a giant Mosix cluster. This is the future of computing.
"I assumed blithely that there were no elves out there in the darkness"
Using the Linux Terminal Server, this could be a good idea for schools. They usually have a bunch of computers with the same HD image. By using clusters, they would never need to upgrade the image on all computers, no need for expensive HDs and the sound level would probably fall quite a lot!
This is exactly where I feel Linux should be used. The idea of dumb terminals and a central server has proven to be the most cost effective way for companies to implement computer technology.
[...]
In the future most companies will have dumb terminals and a server room with racks of cheap intel boxes. The OS on the server will be fault tolerant to the max, oh I lost a node ahh well only 255 left.
I'm trying to figure out what the benefit of this is. You'd have to maintain the user clients - which will still break down - and the server nodes on top of this.
You get fault tolerance - but user terminals don't need uptimes of years with transparent failover. You get centralized administration - but there are many ways of making this happen with user workstations too (witness the NT systems here that re-image their own drives every week).
Performance will always be worse with a centralized solution than with user workstations, because you have no local disk for fast scratch space (used by many applications in the environments I've worked in).
If computers cost $10k apiece, I can see cost being an issue, but if the cost of hardware and maintenance for a user's machine is much, much less than the cost of the user sitting at the machine, I don't see any justification on the basis of cost either.
How is this supposed to be a "most cost-effective" solution, again?
[Disclaimer: I think dumb terminal systems are nifty; I just don't think they're useful under most business conditions.]
This idea is used at CERN. Many desktops belong to a cluster (managed with Condor), but only when not in active workstation use. Therefore full clustering effect only becomes at night, but then again the daytime desktop use is not slowed down by batch work.
--
If you moderate this, then your children will be next.
So it apears the one true god Microsoft is producing a new OS called IX, and when it comes out all of you will be shaking in your' boots and wanting to rip it off.
Microsft rules and everyone else drules
I wish you would mark me as a troll you pathetic slashdot zealots.
Having set this up myself, it seems the author has a few misconceptions about PXE. These seem to be common, as I get into heated discussions on IRC with people who have never done this themselves, but seem to think they know better than I do for some reason. I may have some minor errors in my description below, but I think it's mostly correct.
:)
First off, his cluster isn't really diskless, since he uses floppies.
PXE is an Intel specification, but it is open as far as I know. Intel provides binary only daemons for PXE for Linux. PXE is a way to get around the 640k limitation that is inherent when using the bootp(or dhcp)/tftp boot methods.
PXE is not something that is supported in the kernel as the author implies. PXE is a userspace daemon that allows the workstations to download the whole kernel and also it can present some pretty complicated menus to the user. It is one type of bootstrap, and it is pretty complicated to set up. The PXE daemon for Linux isn't documented very well either, and requires some strange configuration of itself, and also of the DHCP daemon on the server.
Basically, the way I understand it, the DHCP process begins normally from the workstation boot ROM, and the DHCP returns a specific value that tells the workstation information about PXE. The PXE client then connects to the PXE server, and the user is presented boot options, which can be complex.
I didn't use PXE in my final cluster though, due to the extra complication. What I found out was that the SYSLINUX people write something called PXELINUX. PXELINUX is misnamed because it does not use PXE, rather, it is a bootloader that loads over the normal BOOTP/TFTP method, which is loads simpler to set up and maintain. PXELINUX should be thought of as a replacement for PXE.
Without a boot loader, a lot of the docs say you can just send the kernel to the directly to the client. This would work, but iff your kernel is less than 640k, as tftp/bootp operate in real mode, and they have to download the whole thing before they begin booting. (BTW the docs on diskless setups in Linux are extremely out of date for the most part)
With a raw kernel setup, it's also impossible to pass the kernel any boot options. It's the same as if you dd the kernel to a floppy device.
I gained a lot of knowledge about diskless booting in modern Linux in my setup, if anyone wants me to write a book, I'm open to offers.
-Gigs
gigs(at)vt(dot)edu-cational
I've had enough abrasive sigs. Kittens are cute and fuzzy.
I would like to know if there are any really smart uses of this out there. Anyone using this at home for anything clever? Or at a small business/educational institution? I don't want to know about large companies doing 3D simulations with cluster.
Something usefull that can be used at home would be interesting...
I really don't give a fuck about MOSIX clusters
Also, be sure to support OpenMOSIX
Apparently MOSIX is going to go closed source, so test out OpenMOSIX if you can, the project is really taking off and has several contributers, but it needs your help in testing the kernels. OpenMOSIX is being sucessfully used in major installations now, so it should be fine for what you want to use it for, and also you won't be getting yourself going on a (soon to be) proprietary path.
I've had enough abrasive sigs. Kittens are cute and fuzzy.
Mosix migrates processes, not threads. There is quite a difference, which is why MOSIX is almost entirely useless.
2 hours have passed and no-one has posted the lame obvious un-joke. Have people in Slashdot finally grown a real, functional senses of humor?
Yep.
I got an old Pentium 100 I was using like a X-Terminal until its old disk died (well, it works for some 10 minutes then stops).
I just thought "Cool". Can I be that "anybody"?
Please, give up. No matter how money you have, you _will_ lose. Think of Linux like Wolverine, with infinite regeneration powers!
DISCLAIMER: Wolverine character is copyright (or trademark) by his legal owners.
This line from the faq nearly made me jump...
/usr/src/MOSIX-1.5.7/inst/add_kernel_to_grub
chmod goa+x
...for fear that the goatse guy has returned.
Is setting up clusters something "kernel-phobic" or "beginner users" should be attempting in the first place? Really, it is kind of funny to expect an article to be aimed at that audience.
Belief is the currency of delusion.
Could you pass on a little more info on the CERN clustering? Several of the people I sport work closely with CERN and have mentioned it but didn't really understand the details. URL?
Thanks!
I only know it from the user's point of view and can't tell much more. They also have dedicated clusters (i.e. no desktop usage). Both kinds of clusters run Linux but they probably have others as well (it's a huge organization, about 7 kpeople, so I don't know everything :-). Maybe the cerh.ch webpages and/or google will lead you further.
--
If you moderate this, then your children will be next.
http://www.clustermatic.org/
Similar to MOSIX - bproc and a suite of tools for getting a diskless cluster up and going quickly. Very cool - used for clusters of 128->1024 nodes currently.
And working on a Beowulf Cluster is sort of a painfull team love-hate relationship.
Here, I can build a cluster myself with 2 nodes, and add nodes as I go, whenever I prefer, easily, and with an installation scipt I can automatize...
Ok boys, all of you with old, unused Dual-PIII / 256 Mo / 10 Gigs, you can now help me build my own Supercomputer. Just you send them to me.
Also, I have that odd project of building a PS2 3d cluster, so please also send any spare PS2 you have 8)
It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker