Slashdot Mirror

← Back to Stories (view on slashdot.org)

Maintaining Large Linux Clusters

Posted by michael on from the sysadmin's-delight dept.

pompousjerk writes "A paper landed on arXiv.org on Friday titled Installing, Running and Maintaining Large Linux Clusters at CERN [PDF]. The paper discusses the management of the 1000+ Linux nodes, upgrading from Red Hat 6.1 to 7.3, securely installing over the network, and more. They're doing this in preparation for Large Hadron Collider-class computation."

7 of 134 comments (clear)

  1. Lucky bastards by Professor+D · · Score: 5, Interesting
    #include "back-in-my-day-rant"

    Damn. Back when I was on a high-energy experiment located in the middle-of-nowhere in Japan (subject of at least two slashdot articles), our japanese colleagues used to lease gaggles of Sun workstations at a yearly maintanence cost that exceeded the retail value of the machines themselves!!

    A few of us linux-fans used to grumble that we'd be better off buying dozens of cheap linux-boxes, but we weren't making the buying decisions. It seemed to us that the higher-ups didn't think cheap boxes with a free OS could compete on a performance basis with the Suns.

    As for me? I just installed CERNlib on my laptop and just laughed as it blew the suns away on a price/performance(+portability) basis

  2. Re:"But why?" asked Little Johnny. by Bob+Wehadababyitsabo · · Score: 5, Interesting
    Where I work, there is a 500 node Linux cluster for cladistic tree generation, which takes a lot of brute force and specialized tools to make happen. It is arguably much more complex then launching a rocket.

    Just because you don't need it, or can't envision needing it, doesn't mean nobody else needs that kind of power.

    Bob

    --
    fsck -u
  3. ClusterKnoppix - OpenMosix by Anonymous Coward · · Score: 5, Interesting

    I've been looking at ClusterKnoppix mentioned recently on slashdot. It has built in openmosix and also supports thin clients via a terminal service. Just pop it in, and instant cluster. In case you missed the article:

    ClusterKnoppix

  4. Single system image by Tester · · Score: 5, Informative

    Where I work, we are developping a clustering system using single system images.. Where all the OS is stored on a server and is NFS mounted by each node. Our current tests show that we can easily run 100 nodes on 100mbit ethernet from a single server... And the coolest thing is that the nodes mount the / of the server, so for "small clusters" (under 100 nodes), we have to do a software upgrade only once and all nodes and the server are upgraded... Btw, this whole thing can be done using an almost unmodified Gentoo Linux distribution.

    I'm hoping to convince my boss to let us publish detailed docs.. he thinks that if we do everyone will be able to use it and he will loose sales (we are in the hardware business..). Details at our homepage and about an older version (but with more details) at the place where we used to work.

  5. why such a huge cluster? by Anonymous Coward · · Score: 5, Interesting

    well, i recently interviewed at nvidia, and they have a 3,000+ cluster just for emulating the new graphics/io chips they're working on... they don't manufacture anything, the turn around time to manufacture a prototype for testing would take too long... so all they do is simulate the actual chips and then send the data off for fabrication once they're done. on a cluster of 3,000 machines, some jobs take all weekend, from what i understand.

    imagine if they just used one machine.

  6. Re:"But why?" asked Little Johnny. by vondo · · Score: 5, Interesting
    Disclaimer: IAAPP (I am a particle physicist).

    First, as another poster pointed out, these detectors produce a LOT of data. I'm on an experiment slated to take data at about the same time as the LHC experiments, with similar rate requirements.

    We plan to use a 2500 node cluster (of year 2007 CPUs) to filter our data in real time. The input rate into this cluster will be about 10 GB/s, output rate about 200 MB/s.

    But, each interaction is analyzed (usually) by just one computer. There are so many interactions, though, that you need massive clusters, but not much communication between nodes of the cluster.

    That's just for the data filter. You need even larger amounts of computing to analyze what comes out in that 200 MB/s and to simulate what happens in the experiment. Much larger amounts.

    Our experiment will ultimately require clusters this size at the laboratory and at something like a dozen other institutions.

  7. SystemImager-like update mechanism for non-Linux? by pschmied · · Score: 5, Interesting

    I'm surprised that nobody has mentioned SystemImager. If you haven't looked at it for maintaining large numbers of Linux boxes, scamper off and take a look now. It is worth your time.

    Now, that being said, I recently had the opportunity to evaluate using a number of OpenBSD boxes, but I couldn't find a utility for maintaining a bunch of the boxes in the same manner as SystemImager (i.e. Incrementally update servers from a golden master via rsync).

    So, has anyone run found anything that does what systemimager does, but that is cross-platform? Do any SystemImager developers out there want to comment on the potential difficulty in supporting other-than-Linux operating systems in SystemImager?

    SystemImager is one of the most useful tools I've ever seen, however, I believe that it would be an enterprise "killer app" if it could do MacOS X, *BSD, Windows etc.

    -Peter

    --
    . Penguins Surely Ca