Slashdot Mirror
10 Years of Beowulf Clustering
Quirk writes "Wired News has a blurb celebrating the 10th birthday of the Beowulf cluster. Attendees recalled the initial fear and loathing the Beowulf project had to overcome. The Beowulf project takes its name from an epic poem penned circa 1000 A.D."
...it feels like I've been reading Beowulf cluster jokes on Slashdot for longer than that....
Q: What does the "B." in Benoit B. Mandelbrot stand for? A: Benoit B. Mandelbrot
Ah, finally, a story where all those 'imagine a Beowulf cluster of...' comments actually would be ON TOPIC! Naturally that means there won't be that many, other than comments such as this one that is commenting on such comments...a meta-comment about Beowulf clusters. Speaking of which, can you imagine a meta-cluster...oh never mind...
In 1995, I put together an animation of a satellite my company was working on. I used POVray running on DOS, and wrote a little pair of programs that would hand off render-jobs to different computers. I used 16 computers (mostly P60's) lying around the office to render about 400 frames total. The whole job took about 35 hours of wall time, which was important because I had only three 1/2 days to tweak my small demo & make a final rendering.
.TGA files.
I didn't know network programming, so all communication was through read/writing a few networked control files. One acted as a semaphore - if you had sucessfully written your computer ID to it, you could modify the main to-do-list file. One specialized computer was assigned the task of copying the finshed files onto my new 810MB laptop's hard drive; otherwise the file server didn't have enough space for all the
It was a fun project & I've got it included on my resume. Today it sounds kindof trivial, so I've had to explain that general-purpose clustering tools weren't available then. I guess Beouwulf beat me to it by a year (and a zillion-fold on capability), so I was wrong. Information travelled so much slower those days...
HIV Crosses Species Barrier... into Muppets
The Beowulf poem is the oldest known epic in the Anglo-Saxon language (that's like, early english). It's about the life of a king of the "Geats" called Beowulf. It starts off as him as a young rash figher and follows through to his death after fighting a dragon.
Damn great story - there's probably loads of online texts (like this one?). The only surviving manuscript (possibly the only one ever written) is in the British Library. You can go there and see it.
A quick and sloppy google suggests that this is the first "imagine a..." comment (bottom of the page), though it's possible that the joke predates google search capability since Beowolf clusters and slashdot are both older than google. Still, the fact that it's not used as a joke, and the fact that it got a 1 rating (while, inexplicably, all those repeat jokes get modded to the stratosphere these days) lends an air of authenticity to the claim, lacadasical though the research may be.
grammar-lesson free since 1999. (rescinded - 2005)
Of 64 G5s
Or 128 Opterons
Between them only CAT5
Imagine all the boxen
Benching Quake FPS...
Imagine no shared memory
It isn't hard to do
Nothing to spinlock or thrash for
No cache coherence too
Imagine all the boxen
Crunching local tasks...
Imagine there's no mainframes
I wonder if you can
No need for Crays or S/390s
A cluster loosely bound
Imagine all the boxen
Sharing all the LAN...
You may say that I'm a uniprocessor
But I'm not the only one
I hope someday you'll join us
And we'll simulate nukes as one
I happen to know Dr. Sterling, I worked with him at CACR/Caltech. To answer your question, from what I heard and so on:
1. commodity hardware had some serious problems back then compared to today (so did big iron but that was always hush-hush and part of the mystique of running on big iron), and the software was a lot rougher, did less compared to the commercial Unixes, etc.
2. Running on big iron was, well, not only accepted, but sexy, it was the thing to do. to be able to say you are running your code on the new super fast SGI, Intel or IBM is much more uber (at the time anyway) than to say yeah we ran it on a bunch of desktop PCs in a cluster. Took awhile for the non-computer tinkerer scientists to accept the whole thing.
3. Even today, some jobs run better on a shared memory big iron machine than parallelized out on a
cluster using message passing. That was true then, also.
4. Scientists don't always think of a very scalable (i.e. increasingly faster potentially) thing like a cluster as good. Gone are the days where you can start a run and disappear to go mountain climbing or sailing for two weeks. At best a long simulation (or portion thereof) buys you a long weekend. The faster the number-crunching goes the more work you have to do, the more results are expected faster, etc, etc. A vicious circle really. If this concept shocks you, pretend grants, academia and all of it has no politics, only wonderful breakneck pursuit of fact and conquering new horizons...
That's just my take. Oh yeah, and highly unlikely to get funding or donations back then from the big companies of equipment to build a cheap alternative to their flagship HPC products... They didn't exactly encourage that sort of thing.