Slashdot Mirror
10 Years of Beowulf Clustering
Quirk writes "Wired News has a blurb celebrating the 10th birthday of the Beowulf cluster. Attendees recalled the initial fear and loathing the Beowulf project had to overcome. The Beowulf project takes its name from an epic poem penned circa 1000 A.D."
...it feels like I've been reading Beowulf cluster jokes on Slashdot for longer than that....
Q: What does the "B." in Benoit B. Mandelbrot stand for? A: Benoit B. Mandelbrot
Might as well have posted "Hey, practice your cliche overuse in here!"
Ah, finally, a story where all those 'imagine a Beowulf cluster of...' comments actually would be ON TOPIC! Naturally that means there won't be that many, other than comments such as this one that is commenting on such comments...a meta-comment about Beowulf clusters. Speaking of which, can you imagine a meta-cluster...oh never mind...
Imagine a Slashdot topic full of Beowulf clusters...
Vos teneo officium eram periculosus ut vos recipero is.
In 1995, I put together an animation of a satellite my company was working on. I used POVray running on DOS, and wrote a little pair of programs that would hand off render-jobs to different computers. I used 16 computers (mostly P60's) lying around the office to render about 400 frames total. The whole job took about 35 hours of wall time, which was important because I had only three 1/2 days to tweak my small demo & make a final rendering.
.TGA files.
I didn't know network programming, so all communication was through read/writing a few networked control files. One acted as a semaphore - if you had sucessfully written your computer ID to it, you could modify the main to-do-list file. One specialized computer was assigned the task of copying the finshed files onto my new 810MB laptop's hard drive; otherwise the file server didn't have enough space for all the
It was a fun project & I've got it included on my resume. Today it sounds kindof trivial, so I've had to explain that general-purpose clustering tools weren't available then. I guess Beouwulf beat me to it by a year (and a zillion-fold on capability), so I was wrong. Information travelled so much slower those days...
HIV Crosses Species Barrier... into Muppets
The Beowulf poem is the oldest known epic in the Anglo-Saxon language (that's like, early english). It's about the life of a king of the "Geats" called Beowulf. It starts off as him as a young rash figher and follows through to his death after fighting a dragon.
Damn great story - there's probably loads of online texts (like this one?). The only surviving manuscript (possibly the only one ever written) is in the British Library. You can go there and see it.
You might be interested in grid computing, in which a group of academics with heads too big for their common good decide not to build one fucking huge computer in one place, but instead spend all their grant money on fiber transceivers and other equipment that can transfer at a few dozen GBit between far less powerful clusters. Whenever you see a grid built with modern equipment (rather than one that strings together a few older machines), it means the people involved at some level were playing politics so that they could 'me too' their department into owning a piece of it.
I once watched some of this process in motion, which helped to smack down a far more sensical and quite impressive machine proposal, and found the whole thing to be entirely retarded.
Once this article gets duped a few times, we'll have a Beowulf cluster of stories.
Lawrence Person (lawrencepersonh@gmailh.com (remove all "h"s to mail)
http://www.lawrenceperson.com/
That's it. Every one of you boneheads is now my enemy.
A quick and sloppy google suggests that this is the first "imagine a..." comment (bottom of the page), though it's possible that the joke predates google search capability since Beowolf clusters and slashdot are both older than google. Still, the fact that it's not used as a joke, and the fact that it got a 1 rating (while, inexplicably, all those repeat jokes get modded to the stratosphere these days) lends an air of authenticity to the claim, lacadasical though the research may be.
grammar-lesson free since 1999. (rescinded - 2005)
Beowulf aims at minimizing computation time. One option for reducing the processing time of a program is to divide it into independent sub-tasks that can be processed by different CPUs. When the results of these sub-tasks are available, they can be returned to one of the processors for final processing. It is possible to use Ethernet transfers to extend this strategy across multiple computers. This is how Beowulf works: divide programs into many parts that are executed by many CPUs all of which transfer their data and instructions via Ethernet.
Danny.
I have written over 900 book reviews
I just imagined it...
Sincerely,
Pan Tarhei Hosé, PhD.
"Homo sum et cogito ergo odi profanum vulgus et libido."
Of 64 G5s
Or 128 Opterons
Between them only CAT5
Imagine all the boxen
Benching Quake FPS...
Imagine no shared memory
It isn't hard to do
Nothing to spinlock or thrash for
No cache coherence too
Imagine all the boxen
Crunching local tasks...
Imagine there's no mainframes
I wonder if you can
No need for Crays or S/390s
A cluster loosely bound
Imagine all the boxen
Sharing all the LAN...
You may say that I'm a uniprocessor
But I'm not the only one
I hope someday you'll join us
And we'll simulate nukes as one
I happen to know Dr. Sterling, I worked with him at CACR/Caltech. To answer your question, from what I heard and so on:
1. commodity hardware had some serious problems back then compared to today (so did big iron but that was always hush-hush and part of the mystique of running on big iron), and the software was a lot rougher, did less compared to the commercial Unixes, etc.
2. Running on big iron was, well, not only accepted, but sexy, it was the thing to do. to be able to say you are running your code on the new super fast SGI, Intel or IBM is much more uber (at the time anyway) than to say yeah we ran it on a bunch of desktop PCs in a cluster. Took awhile for the non-computer tinkerer scientists to accept the whole thing.
3. Even today, some jobs run better on a shared memory big iron machine than parallelized out on a
cluster using message passing. That was true then, also.
4. Scientists don't always think of a very scalable (i.e. increasingly faster potentially) thing like a cluster as good. Gone are the days where you can start a run and disappear to go mountain climbing or sailing for two weeks. At best a long simulation (or portion thereof) buys you a long weekend. The faster the number-crunching goes the more work you have to do, the more results are expected faster, etc, etc. A vicious circle really. If this concept shocks you, pretend grants, academia and all of it has no politics, only wonderful breakneck pursuit of fact and conquering new horizons...
That's just my take. Oh yeah, and highly unlikely to get funding or donations back then from the big companies of equipment to build a cheap alternative to their flagship HPC products... They didn't exactly encourage that sort of thing.