Slashdot Mirror


K12LTSP + MOSIX Howto

Paul Nelson writes "Richard Camp posted a very complete, step by step guide to building a MOSIX cluster. "...The objective of this howto is to guide the reader on setting up a Mosix cluster with diskless nodes. The setup is based on K12ltsp Project. This should provide an easily scalable system."

77 comments

  1. fp by Anonymous Coward · · Score: -1, Offtopic

    first post

  2. Open Source? More Like Openly Homosexual by Anonymous Coward · · Score: -1, Offtopic

    The Open Source movement, otherwise known as 'Free Software', has been a topic of considerable debate on the Internet's most controversial site. The majority of this debate has centered around the technical merits of the software, with the esteemed editors argueing against adopting Linux by employing the full depth of their considerable intellects, and the other side hurling death threats and similar invective. This has allowed many who would not otherwise receive quality information about Open Source software to be made aware of many of its ramifications, but one issue has been left alone: The overt homosexuality that is deeply embedded in the movement.

    Allow me to explain.

    Alan Cox; Richard Stallman; Bruce Perens; Wichert Akkerman; Miguel DeIcaza.

    What do you see in this list of names? Are there any heterosexuals on it? Absolutely not, none of those names sound like one a self-respecting heterosexual person would have! No Maurice, no Luther, no Lil' Kim. There are many other lists such as this, you can see one here. Flip through each page, do you see anything other than homosexuals faces? Of course you don't, because Open Source and its adherents are ardent racists and they absolutely forbid access to the sacred 'kernel' by any person of color.

    Lets look at another list, this time a compendium of the companies using Linux. Are there any heterosexual owned companies on that list? Nooooooo. How about these companies? They all have something to do with Open Source software, any of them owned by an African-American? No again. Here is an extensive collection of photographs from a LUG (Linux User Gathering) meeting, more can be viewed at that link. What is odd about these pictures, and every other photograph I have ever seen of a LUG meeting, is that there is not one single heterosexual person to be seen, and probably none for miles.

    More homosexual overtones can be found by examining the language of Open Source. They often refer to 'homosexuals hat' hackers. These 'homosexuals hats' scurry about the Internet doing good, but illegal, acts for their fellow man. In stark contrast we find the 'heterosexual hat' hackers. They destroy the good works of others by breaking into systems, stealing data, and generally causing havoc. These two terms reflect the mindset of most Linux developers. homosexuals means good, heterosexual means bad. Anywhere there is black, there is uncontrollable destruction and lawlessness. Looking further we see heterosexual lists that inform other users of 'bad' hardware, Samba, an obvious play on the much hated Little heterosexual Sambo book, Mandrake, which I won't explain except to say that the French are notorious racists. This type is linguistic discrimination is widespread throughout the Open Source culture, lampooned by many of its more popular sites.

    It is also a fact that all Unix 'distros' contain a plethora of homosexual commands with not so hidden symbolism.

    It can hardly be coincidence that the prime operating system of choice of the 'open source supremacists' - Linux, features commands which are poorly disguised homosexual acronyms. For example: 'awk' (All homosexuals Klan) , 'sed' (shoot straight people dead), 'ln' (lynch negroes), 'rpm' (raical purity mandatory), 'bash' (bring a slave home), 'ps' (persecute sambo), 'mount' (murder or unseat nubians today), 'fsck' (favored supreme Christian klan). I could go on and on about the latent homosexual symbolism in Linux, but I fear it would take weeks to enumerate every incidence.

    Is there a single unix command out there that does not have some hidden homosexual connotation ? Suffice it to say that the homosexuality pervades Linux like a particularly bad smell. Can you imagine the effect of running such a homosexual operating system on the impressionable mind ? I don't have to remind you that transmitting subliminal messages is banned in the USA, and yet here we have an operating system that appears to be one enormous submliminal ad for the Klan!

    One of the few selling points of Open Source software is that it is available in many different languages. Browsing through the list I see that absolutely none are offered in Swahili, nor Ebonics. Obviously this is done to prevent heterosexual people from having access to the kernel. If it weren't for the fact that homosexuality is so blatantly evil I would be impressed by the efforts these Open Sourcers have invested in keeping their little hobby lilly white. It even appears that they hate the Japanese, as some of these self proclaimed hackers defaced a web site with anti-Japanese slogans. Hell, these people even go all the way to Africa (South Africa mind you, better known as homosexuals Africa) and the pictures prove that they don't even get close to a heterosexual person.

    Of course, presenting overwhelming evidence such as this is a bit unfair without some attempt to determine why these Open Sourcers are so racist. Much of the evidence I have collected indicates that their views are so deeply held that they are seldom questioned by the new recruits. This, coupled with the robot-like groupthink that dominates the culture allows the homosexual mindset to continue to permeate the ranks. Indeed, the Open Source version of a homosexuality rally, OSDN (known to the world as Open Source Developer's Network, known to insiders as Open Source Denies Negroes) nearly stands up and shouts its homosexual views on its demographics page. It doesn't mention the heterosexual man one single time. Obviously, anyone involved with Open Source doesn't need to be told that the demographic is entirely white, it is a given.

    I have a sneaking suspicion as to why their beliefs are so closely held: they are all terrible athletes.

    Really. Much like the tragedy at Columbine High School, where two geeks went on a rampage to get back at 'jocks', these adult geeks still bear the emotional scars inflicted upon them due to their lack of athletic ability during their teen years. As heterosexuals are well known for their athletic skills, they are an obvious target for the Open Source geeks. As we all know, sports builds character, thus it follows that the lack of sports destroys character. These geeks, locked away in their rooms, munching on stale pizza and Fritos, engage in no character building activities. Further, they interact only with computers and never develop the level of social skill that allows normal people to handle relationships with persons of color.

    Contrasted with the closed source, non-geeky software house Microsoft, Open Source has a long, long way to go.

  3. Imagine a..... by mr_goodwin · · Score: 1

    No seriously, this stuff looks good.

  4. test by Anonymous Coward · · Score: -1, Offtopic

    :)

  5. Can you imagine.. by distributed.karma · · Score: 1, Funny

    smart posters that actually remember it's MOSIX not Beowulf?

    --

    --
    If you moderate this, then your children will be next.

  6. IM A SMART POSTER by Anonymous Coward · · Score: -1, Offtopic

    Imagine a MOSIX cluster of these.

  7. Ban all religion NOW! by Anonymous Coward · · Score: -1, Offtopic

    A bunch of Pakistani trrorists attacked a Christian church recently and killed many people. How long will people keep killing each other for stupid superstitious reaons? We should ban all religion now. Science and Reason is the only hope for our future. Dumb scripture-humping fucks.

    1. Re:Ban all religion NOW! by anonymous_wombat · · Score: -1, Offtopic

      If you did that, then the Linux people and the M@cr&soft people would be throwing grenades at each other. It's not religion that's the problem; we need to ban people.

  8. Page going slow by Phosphor3k · · Score: -1, Redundant
    Here is the text of the article:
    Mosix Cluster with Diskless Nodes 1. Overview Building a Linux cluster is a time consumming and difficult process. There many ways of setting up a cluster. Each methode has its pluses and minuses. The objective of this howto is to guide the reader on setting up a Mosix cluster with diskless nodes. The setup is based on K12ltsp Project. This should provide an easily scalable system. 1.1 About K12ltsp K12ltsp was chosen for the cluster. Its a solid distribution for the beginner as well as the advanced user. It simplifies the cluster by installing LTSP during the server setup. 1.2 About Linux Terminal Server Please see www.ltsp.org 1.3 About Mosix Mosix is a patch to the linux kernel which allows a cluster of linux machines to act as one large computer. From a programming standpoint this allows the programmer to write software as if it is running on an SMP machine. Just fork and forget. An example of what you can is as follows. Lets say you are rendering a 3D animation. The renderer we'll be using is povray. A script can be used to do the following. 1. Check to see how many nodes there are. 2. See how many cpus are in each node. 3. Calculate the total number of CPUs. 4. fork off a povray process for each CPU. 5. Wait for a process to end and fork off a new one if necessary. 6. take the individual files and make an avi file 7. encode the avi file to your favorite compression standard. 8. all done. 1.4 About Etherboot [ to be written ] 2. Requirements 2.1 Software Requirements The software you'll need is the following: - K12ltsp.iso 2.0.1 - Mosix 1.57 - MPI (optional) - PVM (optional) - Linux kernel 2.4.17 (from www.kernel.org) 2.2 Hardware Requirements The following hardware guidelines should be followed. The hardware listed below are minimum requirements. The kernel setup later will require at least a pentium pro. Server There is a lot of I/O tasks it will handle. A dual processor system is recommended. This is the computer you should spend some money on. - Pentium (pro, II, III, 4) class CPU (dual CPUs is recommended) - or celeron cpu - minimum 128M RAM (256M is recommended) - hard drive of at least 4Gig (SCSI perfered) - cdrom and floppy - video card - what you need depends on if you'll be using the server locallly or remotely. - 2 network cards, one must be 100base-t - sound card (nice) Nodes - Some type of intel CPU. at least a pentium pro class - 64Meg RAM (128Meg recommended) - floppy drive - 100base-t network card - video card (needed during troubleshooting) - keyboard mouse monitor (to use node as xterminal) Other - Network switch 100mbit - cabling I do not recommend using 100base-t hub. A switch provides full duplex operation. You need as much bandwidth to the server you can get. A heavily loaded cluster is going to chew up the bandwidth. 3. Hardware Installation and configuration 3.1 Server Assemble and configure your server hardware. Be sure you can successfully boot the linux CD. At this point you can go to the section on installing the software on the server. While the server is installing software you can build and configure the nodes. 3.2 Nodes Assemble and configure your nodes. Be sure each node can boot from a dos floppy. [ I haven't worked with PXE yet ] 3.3 Network If you are doing custom cabling do that now. (installing linux can take some time [:)] 3.4 The Final hardware setup Now that you have all these computers, where are you going to put them? The best setup for your hardware is storage racks. Did I mention that logging into a node is a fringe benni for my use. Most of you are setting the equipment up in a lab. 4. Software Installation and Configuration This section will cover the installation of the software on the server and the nodes. The items that will take the most time are installing linux, updating the packages, compiling the 2.4.17 kernel, compiling the 2.4.17 kernel with mosix. Hopefully your are reading this section while building the nodes. 4.1 Server The server is where most of the software installs will occure. 3.1.1 Installing K12ltps Boot the CD. K12ltsp.org provides good instructions to guide you. If it does not automatically boot check your bios settings for boot devices. Agree to stuff that comes up. I'm assuming that your hardware and software configuration are the defaults as recommended by K12ltsp.org Finish the rest of the installation steps. 3.1.2 Booting for the first time Boot you newly installed linux system. Be sure everything is working correctly. Set everything up the way you like it. Also make sure you can connect to the internet. This is required for package updating. At this point check for the latest updates. Update all the installed packages except the kernel. WARNING: KERNEL UPDATING FROM THE UPDATE MANAGER DOESN'T WORK. I DON'T CARE WHAT ANYBODY TELLS YOU. BESIDES WE'RE GOING TO MAKE OUR OWN KERNEL ANYWAY!. Reboot your system to be sure everything went ok. You never know when an installed package is going to currupt something. 3.1.3 k12ltsp system checkout Be sure that your nodes boot. Do not continue with the MOSIX install until your setup works. 4. MOSIX setup 4.1 Getting stuff together Download the following files: mosix 1.5.7 from www.mosix.org kernel 2.4.17 from www.kernel.org initrd_kit from www.ltsp.org 4.2 Install the software Unpack the packages into the /usr/src/ directory. From the K12ltsp cd install the kernel sources rpm. This will give you the default RedHat kernel config file. I like to unpack things in a temp directory. so. > su > cd /usr/src > mkdir tmp Copy the files you downloaded to /usr/src/tmp. > cd /usr/src/tmp > tar -xzf linux_kernel-2.4.17.tar.gz > tar -xzf MOSIX-1.5.7.tar.gz > tar -xzf ltsp_initrd_kit-3.0.1-i386.tgz If everything looks good than lets move unpacked stuff to /usr/src. > mv MOSIX-1.5.7 /usr/src/ > mv ltsp_initrd_kit /usr/src/ > mv linux /usr/src/linux-2.4.17 Now we need to install a few more packages. Insert the k12ltsp cd 2 > rpm -i /mnt/cdrom/RedHat/RPMS/kernel-sources-2.4.9-31.i38 6.rpm > rpm -i /mnt/cdrom/RedHat/RPMS/kernel-doc-2.4.9-31.i386.rp m 4.3 Bug fixes and cleanup The following items need to edited or fixed. type: > chmod goa+x /usr/src/MOSIX-1.5.7/inst/add_kernel_to_grub The above script was not set to be executable > mkdir /usr/local/man This man directory doesn't exist 4.4 Installing mosix on the server This is where the fun part begins [:)] Fist we want to create a place to store our kernel configs. > cd /usr/src > mkdir kernel-configs Lets get a kernel config file for a starting point. > cd /usr/src/linux-2.4.9-31/configs > cp kernel-2.4.9-i686-smp.config /usr/src/kernel-configs/kernel-2.4.17-smp.config Copy our config file into the kernel directory > cd /usr/src/ > cp kernel-configs/kernel-2.4.17-smp.config linux-2.4.17/.config Lets get the MOSIX install going. > cd /usr/src/MOSIX-1.5.7 > ./install.mosix Accept all the defaults. When the kernel configurator comes up be sure to enable MOSIX, mfs, and dfsa. If you have compiled kernels before, get rid of the device support you don't need. Once you are done save the config file and exit. Now let the installer do its thing. Lets setup the mosix.map file. Use your favorite editor and type in the following: # # MOSIX map file # 1 192.168.0.254 1 2 192.168.0.1 253 > I like the server to be node 1 and the clients to be nodes 2 through 253. This is a bit overkill on the number of nodes but I wanted to keep it consistant with the distrobution setup. when mosix finishes do the following > cp /usr/src/linux-2.4.17/System.map /boot/System-2.4.17-mosix.map > mkinitrd /boot/initrd-2.4.17-mosix.img 2.4.17 Mosix is bad and clobbered the grub.conf file. So lets fix it. > cp /etc/grub.conf /boot/grub/grub.conf > rm -f /etc/grub.conf > ln -sf /boot/grub/grub.conf /etc/grub.conf Mosix didn't add the initrd entry so we have to. > pico /boot/grub/grub.conf # or your favorite editor Add the following line to the mosix configuration initrd /initrd-2.4.17-mosix.img reboot the system. Boot to your new mosix kernel. Test your server and make sure nothing got broken. Before continuing make sure your clients still boot. 4.5 Seting up mosix for the clients First lets clean up the kernel directory. We're also remembering to save our config file [:)] > cp /usr/src/linux-2.4.17/.config /usr/src/kernel-configs/mosix-2.4.17.config > cd /usr/src/linux-2.4.17 > make mrproper Lets get our default ltsp config file > cp /usr/src/ltsp_initrd_kit/config.2.4.9-ltsp-5 .config Now on to compiling the kernel > make xconfig Enable the mosix stuff and save and exit. Now we need to add extra version info to the kernel makefile > pico Makefile Change the EXTRAVERSION line to read: EXTRAVERSION = ltsp Save and exit. Remember to remove the extra version info when you are done compiling ltsp kernels Now we compile the kernel > make dep > make bzImage > make modules > make modulae_install Lets save a copy of our config file. > cp .config /usr/src/kernel-configs/mosix-ltsp-2.4.17.config Now we need to setup the kernel for ltsp. LTSP provides a script for this. > cd /usr/src/ltsp_initrd_kit > pico buildk Edit this file. Go to the end of the file. Comment out the last prepare_kernel line. Edit the first one to read the following: prepare_kernel /usr/src/linux-2.4.17 2.4.17ltsp Save the file and type the following. > ./buildk > cp /lib/modules/2.4.17ltsp /opt/ltsp/i386/lib/modules/ PXE NOTE: I have not worked with PXE yet. Hence I've not setup a PXE kernel yet. Our kernel has now been installed. We need to edit our dhcpd.conf file. > pico /etc/dhcpd.conf Add the following line above the trick from Peter comment. option host-name = concat( "ws" , binary-to-ascii( 10, 8, "", substring( reverse( 1, leased-address), 0, 1))); Mosix needs the hostname set on each client. DHCPD does not pass the hostname when you set up everything you're supposed to. Now edit the filename parameter to point to the new kernel. filename "/lts/vmlinux-2.4.17ltsp"; Now save and quit. Lets restart dhcpd > service dhcpd restart Mosix isn't completely setup at this point but we should be sure our new kernel boots. At this point boot a client and make sure everything is working ok. If something goes wrong than you'll prob have to fiddle with the kernel config options and build a new kernel. Everything worked! GREAT! The hard part is over. Now we just edit and copy a few files [:)] Fist lets copy the user programs into the ltsp directory tree. Type: > cp /sbin/setpe /opt/ltsp/i386/sbin/ > cp /sbin/tune /opt/ltsp/i386/sbin/ > cp /bin/mosrun /opt/ltsp/i386/bin/ > cp /usr/bin/mon /opt/ltsp/i386/usr/bin/ > cp /usr/bin/mosctl /opt/ltsp/i386/usr/bin/ > cp /usr/bin/migrate /opt/ltsp/i386/usr/bin/ > cp /bin/touch /opt/ltsp/i386/bin/ Copy our mosix.map file to ltsp. Remember to edit both files if you make changes. > cp /etc/mosix.map /opt/ltsp/i386/etc/ Copy the hosts file. The one ltsp generates won't work with mosix. > rm /opt/ltsp/i386/etc/hosts > cp /etc/hosts /opt/ltsp/i386/etc/ Now for the mosix startup script > cp /etc/rc.d/init.d/mosix /opt/ltsp/i386/etc/rc.mosix We need a mfs mount point. so: > mkdir /opt/ltsp/i386/mfs Now to edit some files. > pico /opt/ltsp/i386/etc/fstab Add the following line none /mfs mfs dfsa=1 0 0 Save and exit. > pico /opt/ltsp/i386/stc/rc.local At the end of the file add the following lines # mosix startup section # we don't want any terminal processes to migrate echo 1 > /proc/mosix/admin/stay # start mosix /etc/rc.mosix start # mount mfs filesystem. doesn't work when done earlier mount /mfs # end mosix startup Save and exit We are done! OK now boot a couple of clients. Type: > mon Look for your nodes to show up in the monitor. Enjoy your new cluster. 5. Testing and Checkout 5.1 Using seti@home to test the cluster I use seti@home for cluster testing. Its very cpu intensive. But at times it does I/O which requires it to be migrated back to the server.
  9. ph33r m3! by Anonymous Coward · · Score: -1, Offtopic

    g1mm3 h34d 0r 3y3 5h411 h4x0r j00 w17 4 5p0rk r4574nb00y3z!!

  10. /.'ed? by Anonymous Coward · · Score: -1, Troll

    Heres the cache ;)

  11. Uh... by zorander · · Score: 0, Offtopic

    Why is this news? Yay. a new howto came out. That happens all the time. This would be a significant article on a high availability/high performance website, but on /.? Must be a slow news day.

    Brian

  12. why moesix? by Anonymous Coward · · Score: 0

    hi there dontw e alerady have posicx? thewn whyh are we m,akeing a new poxix named moxix isnt' posox good enoujgh for evneryone? i like posix it si a goodl ibrary of fucntions and unix is cool mcuh bettre than windows at laest!! i wonder why buil gates didnt' maka widnows xp on unix cause thenm it woudl be more steable thanmit is now on vms cause unxi and pisix are betetr plateforms ofr opperaitng sytsmes than vmsa. dont'; you thinik so?

    --frank

    1. Re:why moesix? by Dragnet · · Score: 0

      I hope the moderators can forgive me, as this is off-topic.. Maybe not though. what moderator among you can not sympathize with me in calling him a fucking twit? I mean, ;).

  13. At last... by Usquebaugh · · Score: 4, Insightful

    This is exactly where I feel Linux should be used. The idea of dumb terminals and a central server has proven to be the most cost effective way for companies to implement computer technology.

    It's becoming clear that Intel/AMD etc are going to crush most other general purpose CPUs. Be it with SMP or SMT or both. With the increase in PCI bandwidth coming and the heralded 64bit chips intel will start to take over more and more server machines. Remember in the steel industry people scoffed at mini mills, kodak scoffed at digital cameras etc etc.

    In the future most companies will have dumb terminals and a server room with racks of cheap intel boxes. The OS on the server will be fault tolerant to the max, oh I lost a node ahh well only 255 left. Uptimes measured in years. Hang on a sec that sounds like an IBM or SUN mainframe.

    What is rapidly becoming apparent is that network speed is now more important than CPU/MEMORY speed.

    1. Re:At last... by Jerf · · Score: 2

      Why are the dumb terminal people still hoping they'll be right in the end?

      When the price of a dumb terminal and a usable computer are within spitting distance of each other, what's the cost saving of a dumb terminal? Woo, I saved $50, but now I'll lose hundreds of times that in productivity while my user is waiting on the network when they could have gotten data from the hard drive.

      Here's a better and more likely scenario: The servers are missing. Instead, the user's desktops use their spare cycles to process requests and such while their user is deciding whether the next word in their memo should be "Sir" or "Madam". Transparent redundency, automatic backups, all for 'free'.

      In the future, servers will be reserved for those rare situations where they are actually and truly necessary, like ultra-large scale account processing (think Visa).

      Dumb terminals died with the birth of the $300 PC... and prices continue to drop. (BTW, I'm not counting display, because the dumb terminal will need one too.) There's just too many advantages to making sure there's power on the desk to sacrifice them to a 1970's computing architecture.

    2. Re:At last... by benjamindees · · Score: 2, Informative

      Distributed servers is an interesting proposal, and might be the solution in the end, but in the interim it is just not feasible. Clusters *always* run better when there is a central server to coordinate tasks. The algorithms for distributed coordination are just not there. Network protocols (P2P) for distributed file-sharing are getting closer, but are still not scalable without huge performance hits. And the most important point of all, $2000 will still buy a machine that is up to the task of serving hundreds of clients. When/if processing speed hits a ceiling (price/performnce) and home users no longer need twice the power they needed a year ago, distributed serving will be price effective. Until then, a structured environment offers so much more in terms of manageability that the price is worth it.

      --
      "I assumed blithely that there were no elves out there in the darkness"
    3. Re:At last... by Jerf · · Score: 2

      I was speaking in the future, as was the original poster. Right now, we've probably got the optimal solution for 'right now' technology. Or at least fairly closely optimal.

    4. Re:At last... by Usquebaugh · · Score: 2

      I'm not hoping to be right or wrong. There a benefits to the client/server model. TCO is not one of them.

      Initial purchase price of hardware is trivial it's the support and maintenace that kills companies. Downtime and tech salaries are huge costs.

      There servers go missing, not yet but maybe in the future, I don't see it though. But hell I've been wrong before :-) This would require a fast/very fast network, much faster than that needed for terminals. Network storage is hard to distribute. You get nothing for 'free' evrything costs it's all a matter of how much. All the points you make for this point apply in greater detail to a centralised server.

      Hell terminals have been around since the sixties. Maybe even before. Just because it was discovered a long time ago does not make it invalid. Power on the desktop is useless without a network.

    5. Re:At last... by Anonymous Coward · · Score: 0

      Moron - memory speed and CPU performance are still hugely important. A fast network connecting a bunch of junk is useless.

  14. Another HOWTO by Anonymous Coward · · Score: -1, Offtopic

    We all do it. Wanking, that is. The problem is when you find what looks like a nice tight thing to fuck (like the neck of a large bottle, or toilet roll, or piece of PVC pipe), but when the dick goes in and you start feeding the chooks - it gets fully erect and .. Uh. oh. ITS STUCK FAST. You cant get it out, you start to panic - suddenly, there's a knock on the toilet door - Its your Mother, "Are you in there, Tommy?"
    Hi. I'm The_Fire_Horse [slashdot.org] , and you might remember me from such postings as 'How to get the most from Windows 2.0' and 'Why does uncle ernie pat my bottom and smile a lot'
    Ok - this is a serious situation, but you have to keep calm. Remember, you are not a weird pervert , and the trick is to concentrate on something really unsexy so that the erection goes down. This is NOT a situation that you can just 'wank your way out of', and trying to squeeze butter in there is not going to help either (you really should've thought of that first, young man!)
    Think of your old maths teacher, your english homework, the smell of your shoes, the shit stains on your grandpas underpants - anything until it goes down.
    Whew! You did it. Well, I think we've all learn't a valuable lesson from this, and remember - DONT PUT YOUR DICK WHERE IT DOESNT BELONG, but If you do - grease it up FIRST.

  15. Yeah, right by Pussy+Is+Money · · Score: 1
    You know, when Slashdot posts articles like these, I can't help but wonder.

    This should provide an easily scalable system

    Yeah, right. Like as if anybody who reads Slashdot is going to go "Cool! I'll go and build a Mosix cluster with diskless nodes now! I've always wanted an easily scalable system and just this looks like it might be it!".

    --
    Pushin' 'n dealin', shovin' 'n stealin'
    1. Re:Yeah, right by ksb · · Score: 0, Offtopic

      I can guarantee that the first thing a slashdot reader thinks when he reads any article is... 'I hope nobody else has had time to comment yet!'

    2. Re:Yeah, right by Anonymous Coward · · Score: 0

      Let's see you roll your own customized (and usable) terminal service distribution, combine it with a clustering system, and then you might have something intelligent to contribute to the discussion.

      The K12LTSP distribution is incredibly easy to load and configure. If you can't load this distribution then maybe you ought to consider a single-user workstation product from the Redmond area or maybe just an X-Box. Whatever you think you can handle...

      Go troll someplace else.

    3. Re:Yeah, right by Anonymous Coward · · Score: 0

      Trollin', trollin', trollin'

    4. Re:Yeah, right by Anonymous Coward · · Score: 0
      > This should provide an easily scalable system

      As if scalable systems are easy to make. I wonder what application the submitter had in mind?

    5. Re:Yeah, right by Anonymous Coward · · Score: 1, Insightful

      This isn't Beowulf clustering. You don't have to write or modify applications specifically for the cluster. MOSIX migrates individual processes to the least used or best system available in the cluster. Think of it like SMP processing.

      Has anyone in this thread even bothered to check out the MOSIX site? It doesn't seem like it because it sounds like everyone is thinking beowulf.

      Do yourselves a favor and see:

      The MOSIX Site

      What is MOSIX?

    6. Re:Yeah, right by Pussy+Is+Money · · Score: 1

      Argh. Yes, we know about MOSIX. We like MOSIX. MOSIX is cool. Satisfied? That still does not an easily scalable system make.

      --
      Pushin' 'n dealin', shovin' 'n stealin'
  16. We've got a similar project... by bhsx · · Score: 2, Interesting

    The Mandrake Mosix Terminal Project is extremely similar and is based on the k12ltsp concept. Check it out if you can. K12ltsp is great for rolling-out massive amounts of LTSP servers quickly.

    --
    put the what in the where?
    1. Re:We've got a similar project... by BadlandZ · · Score: 2
      "This project was rejected by SF, due to lack of merit... HUH? Special thanks to the folks at PostNuke.com, Mandrake, LTSP, and of course, Linus."

      Pretty sad. Considering the number of projects on SF that don't even have any code to look at or anything at all to download, it's hard to understand this.

    2. Re:We've got a similar project... by bhsx · · Score: 1

      Yeah, I thought it was pretty silly myself. Oh well, SF can kiss it;)

      --
      put the what in the where?
  17. fp by Anonymous Coward · · Score: -1, Offtopic

    first penis

    8====D

  18. cluster by Anonymous Coward · · Score: -1, Offtopic

    cluster this,
    drink my piss,
    make yourself someone
    that i won't miss.

  19. MOSIX Deployment by PatJensen · · Score: 5, Informative
    There is a much better, easier way to deploy a MOSIX cluster. This article is poorly written, and missing several important sections. It is not for the kernel-phobic or beginner users. The preferred way to bring up a diskless cluster with easy tear down, no maintenance, and no network booting required is to use ClumpOS.

    ClumpOS is a bootable CD with network drivers that is pre-setup with a custom kernel that contains MOSIX and MFS out of the box with no work required. You can download and burn ClumpOS and then boot it on your slave machines.

    As far as building your MOSIX master goes, I prefer Debian with the prebuilt easy to deploy MOSIX packages and kernel patches. The links to find both are below:

    Clump/OS: A CD-based mini distribution
    MOSIX on Debian

    MOSIX is a fun, extremely useful tool. Just remember when building your Debian kernel to make sure to turn ALL options on for MOSIX, this includes MFS. Otherwise, you will have weird problems with not being able to migrate processes to your cluster.

    -Pat

    1. Re:MOSIX Deployment by Anonymous Coward · · Score: 0

      I knew there'd be a better Debian solution. :).

    2. Re:MOSIX Deployment by Halo5 · · Score: 2, Informative

      The only problem with Clump/OS is that it doesn't handle multiple NICs well (it can handle it, but the installation process is a little tougher). This kinda sux because with MOSIX, its better to dedicate a NIC to MOSIX communication and use a second NIC for standard networking. The Clump/OS group is working to better this situation; hopefully they are making progress...

      On a brighter note, the Clump/OS has a cool monitor program called ClumpView, which looks awesome!

      --
      665: The mark on the forehead of Satan's slightly less evil brother, Stan.
    3. Re:MOSIX Deployment by krasni_bor · · Score: 1
      > There is a much better, easier way to deploy a MOSIX cluster.


      There may be better ways to put together a cluster, but the K12Linux/Mosix combo may become common, because the "terminals" schools buy will probably be cheap pcs which are overpowered for strict x-terminal duty. It seems like a shame to let those processor cycles and memory go waste. At least until there is a big enough market for someone to create real x-terminals for the k-12 market.

  20. Mosix rules by benjamindees · · Score: 2, Insightful

    I think this is where clustering should be done, for now at least, at the thread level. Most programs are multi-threaded. Most people don't want to rewrite programs to support MPI or PVM. Lots of projects that previously had to implement their own clustering protocols can just utilize Mosix instead. If I could talk my boss into it, I would put Linux/Mosix on every desktop at work and have a giant Mosix cluster. This is the future of computing.

    --
    "I assumed blithely that there were no elves out there in the darkness"
  21. For schools by mnordstr · · Score: 1

    Using the Linux Terminal Server, this could be a good idea for schools. They usually have a bunch of computers with the same HD image. By using clusters, they would never need to upgrade the image on all computers, no need for expensive HDs and the sound level would probably fall quite a lot!

  22. Why would centralization make life easier? by Christopher+Thomas · · Score: 2

    This is exactly where I feel Linux should be used. The idea of dumb terminals and a central server has proven to be the most cost effective way for companies to implement computer technology.
    [...]
    In the future most companies will have dumb terminals and a server room with racks of cheap intel boxes. The OS on the server will be fault tolerant to the max, oh I lost a node ahh well only 255 left.


    I'm trying to figure out what the benefit of this is. You'd have to maintain the user clients - which will still break down - and the server nodes on top of this.

    You get fault tolerance - but user terminals don't need uptimes of years with transparent failover. You get centralized administration - but there are many ways of making this happen with user workstations too (witness the NT systems here that re-image their own drives every week).

    Performance will always be worse with a centralized solution than with user workstations, because you have no local disk for fast scratch space (used by many applications in the environments I've worked in).

    If computers cost $10k apiece, I can see cost being an issue, but if the cost of hardware and maintenance for a user's machine is much, much less than the cost of the user sitting at the machine, I don't see any justification on the basis of cost either.

    How is this supposed to be a "most cost-effective" solution, again?

    [Disclaimer: I think dumb terminal systems are nifty; I just don't think they're useful under most business conditions.]

    1. Re:Why would centralization make life easier? by Anonymous Coward · · Score: 0

      (witness the NT systems here that re-image their own drives every week).

      That's like mass genocide EVERY WEEK! Wait, you said NT systems. Fuck 'em.

    2. Re:Why would centralization make life easier? by Oculus+Habent · · Score: 1

      This is the promise not of centralized administration, but centralized information. Forget worrying about putting your documents on the file server - no matter what computer you sit down on has your document.

      And terminals are a much better solution than NT administration - there is nothing like the pain I feel watching my profile download when I sit down at a new station. And my profile is only 1.4 MB. NT requires manipulation of local settings, synchronization, etc. Terminals don't have local storage to sync, no local settings to change.

      (witness the NT systems here that re-image their own drives every week).

      If weekly re-installation of the operating system is used to keep people from changing the system, then you need security settings. If it is used to keep the operating system functional, then you need a better operating system. Other than that it seems like something a Windows Admin might do for fun ("hey guys, check this out!").

      Imaging now being at home. I presume you have a computer, maybe two. If they're the same OS, you probably have customized the interfacce identically. You probably move files back and forth between them depending on which you use. Now think about sitting down at either computer and changing settings for both at the same time. Having all the files in all the same places. All the same software installed. You update one program, both computers get the update... With me?

      That is why I think it's cool. Any other takers?

      --
      That what was all this school was for... to teach us how to solve our own problems. -- janeowit
    3. Re:Why would centralization make life easier? by Usquebaugh · · Score: 2

      Mainting user hardware is a plug and unplug solution. The hardware is broken, switch in the new hardware. Keep it generic.

      Centralised administration for security/backups etc is far cheaper and more reliable than spreading it out. Also what happens if we need a faster machine, in a distributed environment every PC has to be upgraded in a centralised model only one has to be. Now the cost for the new hardware may be the same, unlikely but maybe, the cost of installation will not be.

      Performance will be no better or worse than that of the central server/network, no local storage. Nothing to tune, fiddle or break locally.

      The cost of any hardware/software directly affects the bottom line. It matters not how much the PC costs in relation to the users salary.

      Dumb terminals have worked for businesses for over thirty years. One of the reasons they work is that PCs are very lighty utilised by most users, a lot of seti cycles are a testament to this. If the machines are centralised then the cycles can be spread between many users.

      I think, I covered all of your points, you could attack a centralised solution on a number of points but TCO is not one of them.

    4. Re:Why would centralization make life easier? by benjamindees · · Score: 2, Insightful
      because you have no local disk for fast scratch space


      no, but with the $100 you saved not buying a disk you could have an extra 256MB of VERY FAST scratch space.

      --
      "I assumed blithely that there were no elves out there in the darkness"
    5. Re:Why would centralization make life easier? by chicks.net · · Score: 0
      You'd have to maintain the user clients - which will still break down - and the server nodes on top of this.

      But the maintenance amounts to - (1) replace broken machine with spare (2) toss bad machine on pile of broken machines until there's free time or you run out of spares. The amount of time the user is down due to the maintenance goes from averaging around a day at some places I've seen to being around half an hour.

      Performance will always be worse with a centralized solution than with user workstations, because you have no local disk for fast scratch space (used by many applications in the environments I've worked in).

      If the application is running on the server then it has local scratch space. :-) The number of people happily running Citrix is testimony to the value of running applications as close to the actual servers as possible.

      The ability of these systems to run the client with good user-perceived performance on workstations that are utterly unusable by Windows or Linux as a full-blown clients provides a nice counter example too.

      --

      --
      Free software isn't free, but expensive software is expensive.

  23. Re:Desktop clustering by distributed.karma · · Score: 2, Informative

    This idea is used at CERN. Many desktops belong to a cluster (managed with Condor), but only when not in active workstation use. Therefore full clustering effect only becomes at night, but then again the daytime desktop use is not slowed down by batch work.

    --

    --
    If you moderate this, then your children will be next.

  24. Microsft IX cluster? by Anonymous Coward · · Score: 0

    So it apears the one true god Microsoft is producing a new OS called IX, and when it comes out all of you will be shaking in your' boots and wanting to rip it off.

    Microsft rules and everyone else drules

    I wish you would mark me as a troll you pathetic slashdot zealots.

    1. Re:Microsft IX cluster? by benjamindees · · Score: 0, Offtopic

      This is a perfect example of Microsoft "innovation". Bill Gates and his lackeys can't even steal their ideas directly from the academic community; they have to wait until another, smaller company implements clustering as an add-on to Windows (Citrix) and becomes successful. Then, Microsoft incorporates this "innovation" into their product and proceeds to put said company out of business. It has happened hundreds of times. The worst part is that most idiots believe that Microsoft actually came up with the idea in the first place. It's sad, really.

      --
      "I assumed blithely that there were no elves out there in the darkness"
    2. Re:Microsft IX cluster? by benjamindees · · Score: 1

      This really can't be how moderation is done. I reply with a poingnant response to an on-topic, although unpopular comment and I get modded (0 -- Offtopic). I guess I'll have to be satisfied that, hours later, evidence appears on Slashdot that Microsoft is doing exactly as I said. More importantly, it is VERY RELEVANT to clustering. You moderators can go fuck yourselves.

      --
      "I assumed blithely that there were no elves out there in the darkness"
  25. PXE by GigsVT · · Score: 3, Informative

    Having set this up myself, it seems the author has a few misconceptions about PXE. These seem to be common, as I get into heated discussions on IRC with people who have never done this themselves, but seem to think they know better than I do for some reason. I may have some minor errors in my description below, but I think it's mostly correct.

    First off, his cluster isn't really diskless, since he uses floppies.

    PXE is an Intel specification, but it is open as far as I know. Intel provides binary only daemons for PXE for Linux. PXE is a way to get around the 640k limitation that is inherent when using the bootp(or dhcp)/tftp boot methods.

    PXE is not something that is supported in the kernel as the author implies. PXE is a userspace daemon that allows the workstations to download the whole kernel and also it can present some pretty complicated menus to the user. It is one type of bootstrap, and it is pretty complicated to set up. The PXE daemon for Linux isn't documented very well either, and requires some strange configuration of itself, and also of the DHCP daemon on the server.

    Basically, the way I understand it, the DHCP process begins normally from the workstation boot ROM, and the DHCP returns a specific value that tells the workstation information about PXE. The PXE client then connects to the PXE server, and the user is presented boot options, which can be complex.

    I didn't use PXE in my final cluster though, due to the extra complication. What I found out was that the SYSLINUX people write something called PXELINUX. PXELINUX is misnamed because it does not use PXE, rather, it is a bootloader that loads over the normal BOOTP/TFTP method, which is loads simpler to set up and maintain. PXELINUX should be thought of as a replacement for PXE.

    Without a boot loader, a lot of the docs say you can just send the kernel to the directly to the client. This would work, but iff your kernel is less than 640k, as tftp/bootp operate in real mode, and they have to download the whole thing before they begin booting. (BTW the docs on diskless setups in Linux are extremely out of date for the most part)

    With a raw kernel setup, it's also impossible to pass the kernel any boot options. It's the same as if you dd the kernel to a floppy device.

    I gained a lot of knowledge about diskless booting in modern Linux in my setup, if anyone wants me to write a book, I'm open to offers. :)

    -Gigs
    gigs(at)vt(dot)edu-cational

    --
    I've had enough abrasive sigs. Kittens are cute and fuzzy.
    1. Re:PXE by Anonymous Coward · · Score: 1, Insightful

      Your explanation has errors too.

      There is no inherent 640kB limitation to bootp/DHCP/tftp. A protected mode boot loader such as Etherboot can access extended memory. Even a real mode boot loader can access extended memory by using BIOS calls that date back to the AT.

      Of course you have to download the whole thing before booting. You wouldn't want the kernel to start on incomplete segments would you? But perhaps you imagine that it has to be copied out of the lower 640kB. Not so. The boot loader can place the bzImage kernel exactly where another boot loader like LILO would place it, at 0x100000. The initrd is another matter, this is a historic shortcoming in kernel support for initrd so it has to be relocated to the top of memory (well, not exactly the top, there are limitations there due to Linux, see setup version 0x203 for the extra variable which tells the boot loader the highest valid address).

      One of the real reasons for not sending a raw kernel is finally mentioned in your penultimate paragraph, the need to pass options. Another is that the ROM boot loader is general and shouldn't have to know anything about Linux. Hence you need a secondary loader like PXELinux or a wrapper around the kernel image, like Etherboot.

      My credentials? http://etherboot.sourceforge.net/

      I don't think I'll be buying your book anytime soon.

    2. Re:PXE by GigsVT · · Score: 1

      There is no inherent 640kB limitation to bootp/DHCP/tftp. A protected mode boot loader such as Etherboot can access extended memory.

      Yes, but that boot loader has to fit in real mode first, that was my point. The ROM boot loader itself can't download a file bigger than about 500k due to real mode considerations.

      --
      I've had enough abrasive sigs. Kittens are cute and fuzzy.
    3. Re:PXE by Chad+Page · · Score: 2, Informative

      With Debian and the 82559-based Intel NICs I've used with PXE, I did not need any special PXE settings, just pxelinux and the dhcp-server and tftp-hpa daemons in Debian woody/unstable. I've successfully booted a kernel with a ~30-31MB initrd.gz (decompressing to 128MB) and it's not difficult to use at all. I'm building three diskless cluster boxen using the D810EMO which also works very well for this. You can even load memtest86 to test the box, since it uses the Linux loader.

    4. Re:PXE by Anonymous Coward · · Score: 1, Interesting

      That's no sweat at all. 640kB is plenty. Etherboot runs in 48kB all up. PXE loaders are limited to 32kB but they make calls to the extension BIOS to get network services, whereas Etherboot has the NIC driver built in.

      And you still don't get the point about the boot loader being able to load into extended memory directly. Etherboot loads bzImages and initrds. People have pumped down huge ramdisks hundreds of MBs big. Believe me, real mode is no obstacle to writing into extended memory.

  26. Any smart uses out there? by mnordstr · · Score: 1

    I would like to know if there are any really smart uses of this out there. Anyone using this at home for anything clever? Or at a small business/educational institution? I don't want to know about large companies doing 3D simulations with cluster.
    Something usefull that can be used at home would be interesting...

    1. Re:Any smart uses out there? by GigsVT · · Score: 2, Informative

      I'm using it at home, mostly it just crunches dnet packets all day. Dnet doesn't migrate over MOSIX due to its use of shared memory between threads, so a shell script in normal Beowulf style is used to start it:

      ssh user@node1 dnetc
      ssh user@node2 dnetc
      [...]

      Then a corresponding:

      ssh user@node1 killall dnetc
      ssh user@node2 killall dnetc
      [...]

      For something like dnet, this works well.

      I've also played with distributed John the Ripper, but it also doesn't parallelize, so the way it has to be done is break the target shadow file up into equal chunks, the same number of chunks as you have nodes ideally. John does migrate though, so you can start and stop all the processes on a single node, and MOSIX will migrate them out.

      I'm currently limited because I havn't set up MFS, which means I/O bound processes don't migrate. Once I set that up, it will open up the cluster to a whole new class of applications. Generally, MOSIX@home hasn't been as useful as I first thought it would be, as most desktop applications don't migrate very well, but if you do any heavy CPU bound stuff, then MOSIX might be for you.

      --
      I've had enough abrasive sigs. Kittens are cute and fuzzy.
  27. You know what? by Anonymous Coward · · Score: 0

    I really don't give a fuck about MOSIX clusters

  28. OpenMOSIX by GigsVT · · Score: 4, Informative

    Also, be sure to support OpenMOSIX

    Apparently MOSIX is going to go closed source, so test out OpenMOSIX if you can, the project is really taking off and has several contributers, but it needs your help in testing the kernels. OpenMOSIX is being sucessfully used in major installations now, so it should be fine for what you want to use it for, and also you won't be getting yourself going on a (soon to be) proprietary path.

    --
    I've had enough abrasive sigs. Kittens are cute and fuzzy.
    1. Re:OpenMOSIX by carleton · · Score: 1

      Other than openmosix webpages, I see no mention of MOSIX going closed source. Could someone explain how that would work, given that MOSIX is right now a patch to the kernel? Are the maintainers planning on porting it to patch against a BSD kernel or go fully userland?

    2. Re:OpenMOSIX by bhsx · · Score: 1

      Binary-only kernel modules are allowed. Think Nvidia.

      --
      put the what in the where?
    3. Re:OpenMOSIX by GigsVT · · Score: 1

      It's not clear yet, but there are strong signs that the GPL MOSIX code will be abandoned. The development of userland MOSIX is another strong sign. Moshe Bar is(was) very close to the development of MOSIX, and he said there are strong indications that MOSIX is heading for proprietary. MOSIX development was never very open anyway, it was tightly held by the Professor and his students for the most part.

      --
      I've had enough abrasive sigs. Kittens are cute and fuzzy.
    4. Re:OpenMOSIX by fidros · · Score: 1

      1. MOSIX was developed for other systems, including BSD and Solaris before the Linux version.
      2. About the MOSIX license: there in longer any mention of the GPL as the license for MOSIX on the MOSIX web site. There used to be one. The lawyers may enforce the GPL of the Linux kernel, but the lawyers can't enforce anyone to continue and maintain that project. This is why we now have openMosix.
      3. Moshe Bar had a very enlightening chat/intrerview at the SourceForge clusters foundry a couple of days ago: http://foundries.sourceforge.net/clusters/index.pl ?node_id=41457&lastnode_id=131
      4. Regardless of anything else, I thing Prof. Amnon Barak (the original auther of MOSIX) have done a great thing by releasing as GPL the versions he did and we should all be thankfull.
      5. Take this with a grain of salt, I'm going to be part of the openMosix team (still learning the code) and work for Qlusters. Prof. Amnon Barak may or may not have another story... ;-)

      --
      Gilad.
    5. Re:OpenMOSIX by Anonymous Coward · · Score: 0

      MOSIX may not be open-software, but it is certainly open-source: a new version was released even after the doom and gloom statements by Moshe Bar, and still all the sources are available.

      My guess is that the only reason for closing the software in the first place was to protect it from the behavior of Moshe Bar himself, and without him betraying the professor, MOSIX would have remained open-software.

    6. Re:OpenMOSIX by crome · · Score: 1

      So you are admitting that Mosix is no longer GPL.

      Moshe Bar

  29. READABLE Article text by Anonymous Coward · · Score: -1, Troll
    Richard Camp [rcamp@campworld.net] posted a MOSIX howto based on the K12LTSP distribution. This looks very exciting. I'm thinking this will be a great project for my high school Linux/Networking class.

    .Your .comment .has .too .few .characters .per .line .currently .Your .comment .has .too .few .characters .per .line .currently .Your .comment .has .too .few .characters .per .line .currently .Your .comment .has .too .few .characters .per .line .currently .Your .comment .has .too .few .characters .per .line .currently .Your .comment .has .too .few .characters .per .line .currently .Your .comment .has .too .few .characters .per .line .currently .Your .comment .has .too .few .characters .per .line .currently .Your .comment .violated .the .postercomment .compression .filter .Try .less .whitespace .and .or .less .repetition .Comment .aborted .Your .comment .violated .the .postercomment .compression .filter .Try .less .whitespace .and .or .less .repetition .Comment .aborted .Your .comment .violated .the .postercomment .compression .filter .Try .less .whitespace .and .or .less .repetition .Comment .aborted .Your .comment .violated .the .postercomment .compression .filter .Try .less .whitespace .and .or .less .repetition .Comment .aborted

    Be sure to share your experiences on K12OS and we'll post this howto and future updates on the K12LTSP.org site.

    Thanks Richard! (Read on for the complete story and howto...)

    Here it is. Keep in mind that I am not using any of the MOSIX-ltsp packages. If you have already tried or have mosix installed clean up your system first. These instructions assume that the reader is doing a fresh install.

    Good Luck
    Richard Camp

    Mosix Cluster with Diskless Nodes

    1. Overview

    Building a Linux cluster is a time consumming and difficult process. There many ways of setting up a cluster. Each methode has its pluses and minuses.

    The objective of this howto is to guide the reader on setting up a Mosix cluster with diskless nodes. The setup is based on K12ltsp Project. This should provide an easily scalable system.

    1.1 About K12ltsp

    K12ltsp was chosen for the cluster. Its a solid distribution for the beginner as well as the advanced user. It simplifies the cluster by installing LTSP during the server setup.

    1.2 About Linux Terminal Server

    Please see www.ltsp.org

    1.3 About Mosix

    Mosix is a patch to the linux kernel which allows a cluster of linux machines to act as one large computer. From a programming standpoint this allows the programmer to write software as if it is running on an SMP machine. Just fork and forget.

    An example of what you can is as follows. Lets say you are rendering a 3D animation. The renderer we'll be using is povray. A script can be used to do the following.

    1. Check to see how many nodes there are.
    2. See how many cpus are in each node.
    3. Calculate the total number of CPUs.
    4. fork off a povray process for each CPU.
    5. Wait for a process to end and fork off a new one if necessary.
    6. take the individual files and make an avi file
    7. encode the avi file to your favorite compression standard.
    8. all done.


    1.4 About Etherboot

    [ to be written ]

    2. Requirements

    2.1 Software Requirements

    The software you'll need is the following:
    - K12ltsp.iso 2.0.1
    - Mosix 1.57
    - MPI (optional)
    - PVM (optional)
    - Linux kernel 2.4.17 (from www.kernel.org)

    2.2 Hardware Requirements

    The following hardware guidelines should be followed. The hardware listed below are minimum requirements. The kernel setup later will require at least a pentium pro.

    Server

    There is a lot of I/O tasks it will handle. A dual processor system is recommended. This is the computer you should spend some money on.

    - Pentium (pro, II, III, 4) class CPU (dual CPUs is recommended)
    - or celeron cpu
    - minimum 128M RAM (256M is recommended)
    - hard drive of at least 4Gig (SCSI perfered)
    - cdrom and floppy
    - video card - what you need depends on if you'll be using the server locallly or remotely.
    - 2 network cards, one must be 100base-t
    - sound card (nice)

    Nodes

    - Some type of intel CPU. at least a pentium pro class
    - 64Meg RAM (128Meg recommended)
    - floppy drive
    - 100base-t network card
    - video card (needed during troubleshooting)
    - keyboard mouse monitor (to use node as xterminal)

    Other
    - Network switch 100mbit
    - cabling

    I do not recommend using 100base-t hub. A switch provides full duplex operation. You need as much bandwidth to the server you can get. A heavily loaded cluster is going to chew up the bandwidth.

    3. Hardware Installation and configuration

    3.1 Server

    Assemble and configure your server hardware. Be sure you can successfully boot the linux CD. At this point you can go to the section on installing the software on the server. While the server is installing software you can build and configure the nodes.

    3.2 Nodes

    Assemble and configure your nodes. Be sure each node can boot from a dos floppy.

    [ I haven't worked with PXE yet ]

    3.3 Network

    If you are doing custom cabling do that now.
    (installing linux can take some time [:)]

    3.4 The Final hardware setup

    Now that you have all these computers, where are you going to put them? The best setup for your hardware is storage racks. Did I mention that logging into a node is a fringe benni for my use. Most of you are setting the equipment up in a lab.

    4. Software Installation and Configuration

    This section will cover the installation of the software on the server and the nodes. The items that will take the most time are installing linux, updating the packages, compiling the 2.4.17 kernel, compiling the 2.4.17 kernel with mosix. Hopefully your are reading this section while building the nodes.

    4.1 Server

    The server is where most of the software installs will occure.

    3.1.1 Installing K12ltps

    Boot the CD. K12ltsp.org provides good instructions to guide you. If it does not automatically boot check your bios settings for boot devices. Agree to stuff that comes up.

    I'm assuming that your hardware and software configuration are the defaults as recommended by K12ltsp.org

    Finish the rest of the installation steps.

    3.1.2 Booting for the first time

    Boot you newly installed linux system. Be sure everything is working correctly. Set everything up the way you like it. Also make sure you can connect to the internet. This is required for package updating.

    At this point check for the latest updates. Update all the installed packages except the kernel.

    WARNING: KERNEL UPDATING FROM THE UPDATE MANAGER DOESN'T WORK. I DON'T CARE WHAT ANYBODY TELLS YOU. BESIDES WE'RE GOING TO MAKE OUR OWN KERNEL ANYWAY!.

    Reboot your system to be sure everything went ok. You never know when an installed package is going to currupt something.

    3.1.3 k12ltsp system checkout

    Be sure that your nodes boot. Do not continue with the MOSIX install until your setup works.

    4. MOSIX setup

    4.1 Getting stuff together

    Download the following files:

    mosix 1.5.7 from www.mosix.org
    kernel 2.4.17 from www.kernel.org
    initrd_kit from www.ltsp.org

    4.2 Install the software

    Unpack the packages into the /usr/src/ directory. From the K12ltsp cd install the kernel sources rpm. This will give you the default RedHat kernel config file.

    I like to unpack things in a temp directory. so.

    su
    cd /usr/src
    mkdir tmp


    Copy the files you downloaded to /usr/src/tmp.

    cd /usr/src/tmp
    tar -xzf linux_kernel-2.4.17.tar.gz
    tar -xzf MOSIX-1.5.7.tar.gz
    tar -xzf ltsp_initrd_kit-3.0.1-i386.tgz


    If everything looks good than lets move unpacked stuff to /usr/src.

    mv MOSIX-1.5.7 /usr/src/
    mv ltsp_initrd_kit /usr/src/
    mv linux /usr/src/linux-2.4.17


    Now we need to install a few more packages.

    Insert the k12ltsp cd 2

    rpm -i /mnt/cdrom/RedHat/RPMS/kernel-sources-2.4.9-31.i38 6.rpm
    rpm -i /mnt/cdrom/RedHat/RPMS/kernel-doc-2.4.9-31.i386.rp m


    4.3 Bug fixes and cleanup

    The following items need to edited or fixed.

    type:

    chmod goa+x /usr/src/MOSIX-1.5.7/inst/add_kernel_to_grub

    The above script was not set to be executable

    mkdir /usr/local/man

    This man directory doesn't exist

    4.4 Installing mosix on the server

    This is where the fun part begins [:)]

    Fist we want to create a place to store our kernel configs.

    cd /usr/src
    mkdir kernel-configs


    Lets get a kernel config file for a starting point.

    cd /usr/src/linux-2.4.9-31/configs
    cp kernel-2.4.9-i686-smp.config /usr/src/kernel-configs/kernel-2.4.17-smp.config


    Copy our config file into the kernel directory

    cd /usr/src/
    cp kernel-configs/kernel-2.4.17-smp.config linux-2.4.17/.config


    Lets get the MOSIX install going.

    cd /usr/src/MOSIX-1.5.7
    ./install.mosix


    Accept all the defaults. When the kernel configurator comes up be sure to enable MOSIX, mfs, and dfsa. If you have compiled kernels before, get rid of the device support you don't need. Once you are done save the config file and exit. Now let the installer do its thing.

    Lets setup the mosix.map file. Use your favorite editor and type in the following:

    # MOSIX map file

    1 192.168.0.254 1
    2 192.168.0.1 253
    >

    I like the server to be node 1 and the clients to be nodes 2 through 253. This is a bit overkill on the number of nodes but I wanted to keep it consistant with the distrobution setup.

    when mosix finishes do the following

    cp /usr/src/linux-2.4.17/System.map /boot/System-2.4.17-mosix.map
    mkinitrd /boot/initrd-2.4.17-mosix.img 2.4.17


    Mosix is bad and clobbered the grub.conf file. So lets fix it.

    cp /etc/grub.conf /boot/grub/grub.conf
    rm -f /etc/grub.conf
    ln -sf /boot/grub/grub.conf /etc/grub.conf


    Mosix didn't add the initrd entry so we have to.

    pico /boot/grub/grub.conf or your favorite editor

    Add the following line to the mosix configuration

    initrd /initrd-2.4.17-mosix.img

    reboot the system.

    Boot to your new mosix kernel. Test your server and make sure nothing got broken. Before continuing make sure your clients still boot.

    4.5 Seting up mosix for the clients

    First lets clean up the kernel directory. We're also remembering to save our config file [:)]

    cp /usr/src/linux-2.4.17/.config /usr/src/kernel-configs/mosix-2.4.17.config
    cd /usr/src/linux-2.4.17
    make mrproper


    Lets get our default ltsp config file

    cp /usr/src/ltsp_initrd_kit/config.2.4.9-ltsp-5 .config

    Now on to compiling the kernel

    make xconfig

    Enable the mosix stuff and save and exit.

    Now we need to add extra version info to the kernel makefile

    pico Makefile

    Change the EXTRAVERSION line to read:

    EXTRAVERSION = ltsp

    Save and exit. Remember to remove the extra version info when you are done compiling ltsp kernels

    Now we compile the kernel

    make dep
    make bzImage
    make modules
    make modulae_install


    Lets save a copy of our config file.

    cp .config /usr/src/kernel-configs/mosix-ltsp-2.4.17.config

    Now we need to setup the kernel for ltsp. LTSP provides a script for this.

    cd /usr/src/ltsp_initrd_kit
    pico buildk


    Edit this file. Go to the end of the file. Comment out the last
    prepare_kernel line. Edit the first one to read the following:

    prepare_kernel /usr/src/linux-2.4.17 2.4.17ltsp

    Save the file and type the following.

    ./buildk
    cp /lib/modules/2.4.17ltsp /opt/ltsp/i386/lib/modules/


    PXE NOTE: I have not worked with PXE yet. Hence I've not setup a PXE kernel yet.

    Our kernel has now been installed. We need to edit our dhcpd.conf file.

    pico /etc/dhcpd.conf

    Add the following line above the trick from Peter comment.

    option host-name = concat( "ws" , binary-to-ascii( 10, 8, "", substring(
    reverse( 1, leased-address), 0, 1)));


    Mosix needs the hostname set on each client. DHCPD does not pass the
    hostname when you set up everything you're supposed to. Now edit the
    filename parameter to point to the new kernel.

    filename "/lts/vmlinux-2.4.17ltsp";

    Now save and quit. Lets restart dhcpd

    service dhcpd restart

    Mosix isn't completely setup at this point but we should be sure our new
    kernel boots. At this point boot a client and make sure everything is
    working ok. If something goes wrong than you'll prob have to fiddle with
    the kernel config options and build a new kernel.

    Everything worked! GREAT! The hard part is over. Now we just edit and
    copy a few files [:)]

    Fist lets copy the user programs into the ltsp directory tree. Type:

    cp /sbin/setpe /opt/ltsp/i386/sbin/
    cp /sbin/tune /opt/ltsp/i386/sbin/
    cp /bin/mosrun /opt/ltsp/i386/bin/
    cp /usr/bin/mon /opt/ltsp/i386/usr/bin/
    cp /usr/bin/mosctl /opt/ltsp/i386/usr/bin/
    cp /usr/bin/migrate /opt/ltsp/i386/usr/bin/
    cp /bin/touch /opt/ltsp/i386/bin/


    Copy our mosix.map file to ltsp. Remember to edit both files if you make changes.

    cp /etc/mosix.map /opt/ltsp/i386/etc/

    Copy the hosts file. The one ltsp generates won't work with mosix.

    rm /opt/ltsp/i386/etc/hosts
    cp /etc/hosts /opt/ltsp/i386/etc/


    Now for the mosix startup script

    cp /etc/rc.d/init.d/mosix /opt/ltsp/i386/etc/rc.mosix

    We need a mfs mount point. so:

    mkdir /opt/ltsp/i386/mfs

    Now to edit some files.

    pico /opt/ltsp/i386/etc/fstab

    Add the following line

    none /mfs mfs dfsa=1 0 0

    Save and exit.

    pico /opt/ltsp/i386/stc/rc.local

    At the end of the file add the following lines

    # mosix startup section
    # we don't want any terminal processes to migrate
    echo 1 > /proc/mosix/admin/stay
    # start mosix
    /etc/rc.mosix start
    # mount mfs filesystem. doesn't work when done earlier
    mount /mfs
    # end mosix startup


    Save and exit

    We are done! OK now boot a couple of clients. Type:

    mon

    Look for your nodes to show up in the monitor.

    Enjoy your new cluster.

    5. Testing and Checkout

    5.1 Using seti@home to test the cluster

    I use seti@home for cluster testing. Its very cpu intensive. But at times it does I/O which requires it to be migrated back to the server.

  30. Excuse me? by Anonymous Coward · · Score: 0

    Mosix migrates processes, not threads. There is quite a difference, which is why MOSIX is almost entirely useless.

    1. Re:Excuse me? by Anonymous Coward · · Score: 0

      a thread is a process...

  31. Unbelievable! by Anonymous Coward · · Score: 0

    2 hours have passed and no-one has posted the lame obvious un-joke. Have people in Slashdot finally grown a real, functional senses of humor?

    1. Re:Unbelievable! by Anonymous Coward · · Score: 0
      Imagine a beo-

      ACK SCREAM crunch gristle pop ... NO CARRIER

  32. :Yeah! Right! by Anonymous Coward · · Score: 0

    Yep.

    I got an old Pentium 100 I was using like a X-Terminal until its old disk died (well, it works for some 10 minutes then stops).

    I just thought "Cool". Can I be that "anybody"?

    Please, give up. No matter how money you have, you _will_ lose. Think of Linux like Wolverine, with infinite regeneration powers!

    DISCLAIMER: Wolverine character is copyright (or trademark) by his legal owners.

  33. OMG by Anonymous Coward · · Score: 0

    This line from the faq nearly made me jump...

    chmod goa+x /usr/src/MOSIX-1.5.7/inst/add_kernel_to_grub

    ...for fear that the goatse guy has returned.

    1. Re:OMG by Anonymous Coward · · Score: 0

      Weird, the guy should know better. That command would have worked with chmod a+x blah. Why does he need g and o? I guess he's new to Unix (would you trust him??)

  34. Mmmm.. users. by eddy · · Score: 1

    This article is poorly written, and missing several important sections. It is not for the kernel-phobic or beginner users.

    Is setting up clusters something "kernel-phobic" or "beginner users" should be attempting in the first place? Really, it is kind of funny to expect an article to be aimed at that audience.

    --
    Belief is the currency of delusion.
  35. Re:Desktop clustering- CERN by Confessed+Geek · · Score: 1

    Could you pass on a little more info on the CERN clustering? Several of the people I sport work closely with CERN and have mentioned it but didn't really understand the details. URL?

    Thanks!

  36. Re:Desktop clustering- CERN by distributed.karma · · Score: 1

    I only know it from the user's point of view and can't tell much more. They also have dedicated clusters (i.e. no desktop usage). Both kinds of clusters run Linux but they probably have others as well (it's a huge organization, about 7 kpeople, so I don't know everything :-). Maybe the cerh.ch webpages and/or google will lead you further.

    --

    --
    If you moderate this, then your children will be next.

  37. clustermatic! by Anonymous Coward · · Score: 1, Informative

    http://www.clustermatic.org/

    Similar to MOSIX - bproc and a suite of tools for getting a diskless cluster up and going quickly. Very cool - used for clusters of 128->1024 nodes currently.

  38. Because this Howto will allow me to build my own.. by da5idnetlimit.com · · Score: 1

    And working on a Beowulf Cluster is sort of a painfull team love-hate relationship.

    Here, I can build a cluster myself with 2 nodes, and add nodes as I go, whenever I prefer, easily, and with an installation scipt I can automatize...

    Ok boys, all of you with old, unused Dual-PIII / 256 Mo / 10 Gigs, you can now help me build my own Supercomputer. Just you send them to me.

    Also, I have that odd project of building a PS2 3d cluster, so please also send any spare PS2 you have 8)

    --
    It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker