Wanted: Turn-Key 10-Node Beowulf Cluster

forgotten password writes: "I'd just started working on my morning M&Ms, when I was asked where my group can buy a good turn-key ~2CPUx10-node Beowulf cluster in two hours. I suspect the time frame is longer than that, although the window-of-opportunity for the money is apparently on the order of days, and a quote before the procurement meeting would help. Any ideas? Who's good? What it should cost? Thanks!" If you're quick, maybe you can become the world's newest manufacturer of custom beowulf clusters.

Comments

Atipa

[p14-lda]

Clemson University purchased a setup w/ 512 nodes from Atipa, they delivered it onsite. Can't beat that. Call 888-222-7822, and ask for Bret, tell him the PARL sent you

Re:Atipa

[Anonymous Coward]

Robert: Thanks buddy........send them over and we'll take care of them. Bret

Re:Atipa

[donglekey]

I went to a party at Comdex Chicago that was for Atipa employees, and listend to a talk from the CEO and they are ass-kickers. They have built clusters for Motorola (I think) and NASA. they certainly wouldn't be a bad choice.

Re:Atipa

[Dyelar]

Be warned, Atipa was recently bought by MicroTech computers, and they are sometimes known for not being very good about support, or providing decent equipment. I know that Microtech fired a whole bunch of the Atipa employees though. Before this though, Atipa was looking very good.

If you are interested, SGI can sell you turn key beowulf solutions also. You can also go to http://www.beowulf.org and they have a list of commercial companies that provide beowulf clustering solutions.

Re:Atipa

[Anonymous Coward]

Thank you for your kind words. We take this business very seriously and appreciate clients like yourself.

We do have installs in Motorola, as well as NASA, Los Alamos National Lab, Fermi National Lab, etc......Give me a call at 888-222-7822 if you need something. Bret

Re:Atipa

[Andrewkov]

I tried calling, I think his phone number is slashdotted..

Re:Atipa

[Anonymous Coward]

When working with ATIPA you must check every detail of the quote before you order. We had the experiance that we ordered a cluster from them, for a quote, and then afterwards discovered that they had left the Myrinet cables off of the quote so we had to invest more money (above the quote) to purchase the cables.

You must also make certain to check EVERYTHING that they ship you. It is our experiance that ATIPA will slip lower quality merchandise into your shipment, and you have to call them on it to have it replaced. One one of our orders they missed the quote in at least three points.

Finally we had the experiance that you must give them a deadline and have some financial requirement that they meet it. Even in the case of simple desktop PC's it will take them months before they finish the order and ship it, and by this time the parts have depreciated to 3/4 the initial value.

Oh! One more thing, don't expect any technical support. I called the tech guy (Rocky) repeatedly, and he never returned my call.

Has anyone else had these problems with ATIPA?

Re:Atipa

[Anonymous Coward]

Do not go there. The beowulf people at Atipa (which were from DCG Computers) are all gone.
Find a company that knows what they are doing not a box shop.

Re:Atipa

[TgrMan]

I don't know what you may have experienced with Atipa in the past but I can definitely assure you that things are on the right track now. With a team of engineers, of which I am one, who actually have engineering degrees from accredited schools, many things are in the works behind the scenes. Keep an eye on us over the next few months and I think you'll be pleasantly surprised. I am from one of the schools on the cutting edge of Beowulf which worked with Becker when he was at NASA's Goddard Space Flight Center and more recently all of the guys at Scyld, so I can assure you that Atipa and its partners are working harder than ever to become the premiere Beowulf provider.

Re:Atipa

[jmauro]

By far and away MicroTech is the sleaziest of all possible computer vendors. They only really have money, because for a while the state of Kansas dictacted that computers must be bought in state. Their PC's break after about 1.5 years and they try to wiggle out of complete support contracts, which would have 1.5 years left on them by saying stupid shit like memory is too expensive so we're not going to replace it (even though they are contractually obligated to do so). And they're laptop support is utter crap. You'll call about a failed laptop. Trace the problem down to bad memory (himem.sys won't even start because it can't access sections of memory), request to have it replaced, but they'll send it back saying Norton anti-virus fixed the problem. But the problem pops up in the returned box. Oh yea, and you can't find drivers for anything. Sorry to be so mean, but I've really had more than I can take of Microtech. Stay away from them like the plague.

Re:Atipa

[xmedar]

How about 10 nodes in 3U for Linux?

here

And why didI get the "Your comment violated the postercomment compression filter. " ???

SGI

[Anonymous Coward]

SGI is your best bet. They have been heavily into turnkey Beowulf clusters, installing some really big ones too. They have the experience and the technology.

planning

[bwhalen]

So it would appear that out of nowhere a multithreaded application appeared that needed this kind of horsepower??

Re:planning

[Anonymous Coward]

You are a stupid fucking a cunt.

Re:planning

[Anonymous Coward]

Actually, not multi-threaded, but using a message passing library. Even rarer!

Re:planning

[cweber]

Probably not. In science there's sometimes money that needs to be spent by a certain deadline. More often than not the principal investigator will then suddenly remember that proposal by some group member to spend big bucks on some obscure (for him/her anyway) technical thing and get on the bandwagon. At that point things need to move fast...

Been there, done that. Planning has not much to do with it, but getting a busy and distracted person's ear does.

Remember: Computing per se is not the primary focus of most compute power consumers.

Re:planning

[caseydk]

no, what it ACTUALLY is is a federal group of some sort...

the fiscal year ends in 10 days and any money not spent goes back to the treasury and scrutinized...

one of the contracts that i'm on right now is freezing as of 09/30 unless our client gets at least some of their funding soon...

the House seems to be busy doing other things right now instead of approving the budget... understandable though...

Re:planning

[Anonymous Coward]

Sometimes wants are there, but no money is there...then someone says hey we have money...only for 2 days though.

Western Scientific ...

[Bob(TM)]

... will sell you one.

Price depends on bells and whistles, but the 8 node, dual processor P-III system we got with SCI cards ran around $35K.

http://www.wsm.com

Re:Western Scientific ...

[Anonymous Coward]

damn you got ripped off

x4? xMore?

[DestroyahX]

I wonder what the limits and power of a 4-way quad Xeon Beowulf cluster would be like.

With RAM so cheap nowadays, one could have at least a capable cluster without monster chips. Coupled with Xeons, I fail to imagine how great the price:performance ratio is compared to Big Iron like IBM's Power3 and Power4 systems...

*drool*

Re:x4? xMore?

[Anonymous Coward]

Don't forget to include replacement parts and maintenance to the cost of ownership.

Big Iron hardware tends to be of a higher quality than your average PC stuff.

Re:x4? xMore?

[DestroyahX]

Commodity parts. Big Iron peice are more expensive than PC pieces.

I still think that the $:performance ratio is very exciting on the PC side.

Don't forget Linux, which is free/

Re:x4? xMore?

[markhahn]

they suck, basically. a 4-way Xeon is sitting
on a miserable 800 MB/s (450 sustained) bus,
so unless your code is totally cache-friendly,
the CPUs will starve. and if your code is so
cache-friendly, you should probably be looking
at different hardware in the first place.

Re:x4? xMore?

[DestroyahX]

they dont make 400mhz mobos for xeons? if nto now, they will for intel's upcoming 2.2ghz xeons.

Re:x4? xMore?

[choprboy]

As someone who actually owns a quad Xeon (Intel Sitka 4x400MHz 1M cache) and is building 2 more for a cluster... the answer, as always, is "depends". It all depends on what the intended use is. For a embarrassingly simple parallel processing job (aka running 20 seti@home jobs) the price/performance ratio can be quite poor. Prices are definately down, but the motherboards and RAM are still fairly expensive (typically you need EDO ECC DIMMs with high end server boards, not the cheap SDRAM). You can pick up several 1GHz barebones Athlons for the same price and run the data serially thru each at a faster pace.

On the other hand, if you have a true multi-threaded, highly integrated task that requires high inter-process communication, separate boxes are a poor choice. Something like a large relational database or multi-dimensional vibration calculation wherein each calculation requires knowledge of it's neighbors motions, is far superior on a multi-CPU box. Unless you implement an expensive Dolphin/Myrinet network, the process communication alone, be it over ether, SCSI, or FC, kills a multi-box solution. Not to mention the fact quad Xeon boxes typically take 4-8GB of RAM so everything is always local.

So they want a cluster in 2 hours to do .... ?

[Rosco P. Coltrane]

You can't buy (or make) a x node cluster and expect to run Quake x times faster : parallel processing machines require parallel programming. This is usually means that you want a cluster to solve a particular problem, that the problem is specified, the parallel application is properly designed, architectured and implemented.

I have the feeling that friend forgotten_password's group have no clue about Beowulf clusters.

Re:So they want a cluster in 2 hours to do .... ?

[bbay]

Surely we can give the fellow the benefit of the doubt, and assume that he knows at least this much.

Re:So they want a cluster in 2 hours to do .... ?

[Diclophis]

Not together true... http://www.mosix.org/ is a kernel patch that will allow seamless clustering...

Re:So they want a cluster in 2 hours to do .... ?

[bort13]

so you're saying that just because they might not need it is a good reason for not buying such a cool toy?

for shame!

Re:So they want a cluster in 2 hours to do .... ?

[xcjohn]

*or* they know exactly what they're doing and you're bein an idiot and assuming the guys a fsckin moron. People dont go out and buy 10 node clusters of 2 way machines on a whim.

Re:So they want a cluster in 2 hours to do .... ?

[Anonymous Coward]

Hey, you folks have obviously never worked in
a University environment before. It often works
this way. They've probably wanted the cluster for
forever, but couldn't get it approved. Now a grant
is running out with money left over: spend it or
lose it. So suddenly a muckety-muck is running
down the hallway, huffing hard, and telling them
to find something to spend this wad on NOW. Since
none of the researchers can buy their specialized
toys fast enough, the computer guys are eventually
asked. I've bought a RAID server and a high-end color laserjet this way. Because I know how to order stuff NOW.

Re:So they want a cluster in 2 hours to do .... ?

[carlos_benj]

remove your left eye to email me.

Gee, I know he was asking for a quote, but if the price is that steep just for the RFP I'm afraid we'll have to pass on the consulting fees....

(I normally don't respond to .sigs, but given the topic....)

Re:So they want a cluster in 2 hours to do .... ?

[pa-guy]

Hey man, do you work at the U of A? That sounds like the shit I do all the time....

10,000 G

[teknopurge]

dont pay more then 10k. there was an article on slashdot a while back about building cheap beowolf clusters. too lazy to search for it.

-teknopurge

Re:10,000 G

[bteeter]

Errr.

No, I don't think so. The article you are refering to was actually about building a server with 1 TB of storage for under $10,000. I don't think it had anything to do with clustering. See it here:

http://slashdot.org/article.pl?sid=01/07/19/155421 6&mode=thread

That was over 2 months ago, so the price is probably even less than $10K now...

Take care,

Brian
--
AssortedInteret.com
Featuring 100% Linux Based Web Hosting
Try us Risk Free with our 30-Day Money Back Guarantee
http://www.assortedinternet.com/hosting/

I dont think you can ...

[Tensor]

I never saw a place that sold ready made beowulfs ... i think you need to "assemble them" yourself. And if u can manage that in two hours you should come work for us ! Check out http://www.beowulf.org/

Re:I dont think you can ...

[Anonymous Coward]

Not so. There is a company that specializes in turn-key Beowulf systems www.plogic.com

If I remember my Beowulf correctly...

[Slashdolt]

It should cost an arm, but not necessarily a leg.

Re:If I remember my Beowulf correctly...

[Anonymous Coward]

beowulf clusters are cheap no? Whasn't the first built from cast off computers? Isn't that the whole point?

Isnt a turn-key beowulf cluster somehow contradictory?

Re:If I remember my Beowulf correctly...

[nobody69]

Bad Slashdolt, bad!

Beowulf Site (www.beowulf-underground.org)

[p14-lda]

Check out www.beowulf-underground.org That is the place for everything beowulf. It is run by the guys in the Parallel Architecture Research Lab at Clemson University.

this month's Linux Journal...

[Kraken137]

on page 7, has an ad for Aspen Systems, advertising out-of-the-box beowulf clusters. I've never used them myself, but they have an impressive list of clients in the ad, including Los Alamos, MIT, NIST, NOAA, and Sandia.

www.aspsys.com

Scratch That

[Tensor]

These two places make and sell Clusters

http://www.par-tec.com/

http://www.aspsys.com/

I have no knowledge about how good these are.

BTW, you DO know that you need a special type of paralell programming to really make use of all the added processors rite ? its not like your regular apps will run faster.

Good luck
T

GHS Intelligent Integrated Systems

[Anonymous Coward]

GHS Intelligent Integrated Systems sell turn-key beowulf clusters at a very reasonable rate. Contact David Grant

From Linux Journal

[kperrier]

A quick search of the ads in my linux journal:

• http://www.microway.com/
• http://www.aspsys.com/
• http://www.swt.com/

I hope this helps!

Kent

Vendors

[PenguinX]

Penguin Computing ships beowulf clusters

IBMdoes a lot of linux stuff, they even have beowulf traning classes - I imagine that they have some turnkey solution.

Compaq sells 'em. too.

In other words, almost any company that sells Linux servers sells beowulf clusters o' servers as well. And if you want training, quite a few of them out there have classes for it too :)

Re:Vendors

[Anonymous Coward]

Compaq has good maintanace services if your talking about servers but there price can be high

IBM welll maintanance... if you got AS400.. it will run linux though.

With Pinguin Computing i don't have any experience

Re:Vendors

[shogun]

Penguin Computing [penguincomputing.com] ships beowulf clusters

This ask slashdot might not of been neccesary if the original questioner wasn't filtering slashdot's banner ads. :)

Yikes

[Anonymous Coward]

My high school myth classes have come back to haunt me.

Re:Yikes

[Anonymous Coward]

>Hey, like in that story, Grendel eats their goats first, right?

If there's one thing I've learned reading and posting to slashdot, it's never to mention goats...

*shudder*

Maybe something here ...

[Midnight Thunder]

I have never set one up, but maybe visiting the Beowulf might be a good starting point. Other links include the Beowulf Clusters page at Yahoo and the Oak Ridge Extreme Linux Page.

If you don't find any answers to your quest then you could always buy 10 dual-processor machines, configure one and then copy its HD image to the other 9 ( I have never tried this ).

OUCH

[Tensor]

1. Microway's Dual 1GHz Pentium III Beowulf Cluster
Package Pricing (Including Server):
8 Processors: $ 8,625
16 Processors: $16,325
32 Processors: $31,725
64 Processors: $62,525

dammm this is like a 1k per proc ! i am sure you can build it cheaper

Re:OUCH

[Anonymous Coward]

Actually 1k per CPU is what we ended up paying for our cluster.. (Mind you it's not beowolf, but same basic hardway..) Dual Athlon 1U, external SCSI.

Re:OUCH

[BrentN]

Not if the price includes the networking costs. If you're using "real" interconnects (i.e. not 100BT), the networking costs can be as much as a quarter of the total cost. Also, if you buy rackmount enclosures, they are typically more expensive than if you use minitower cases. But even so, when we built our 168 node Pentium cluster, we paid $900/node exclusive of networking costs.

One Word...

[Anonymous Coward]

One word:
MOSIX

It's a breeze to set up and administer. Cool stuff

Linux Networx

[Snodgrass]

We build beowulf clusters and have (if I do say so myself) the most advanced cluster management utilities in the industry.

Check us out.

I'm not a salesman, but we've got some people that would be more than happy to talk to you. :)

Re:Linux Networx

[Anonymous Coward]

salesperson
thankyou.

Re:Linux Networx

[Snodgrass]

I am a man and will to refer to myself as such.

However, if you inquire at the web site a salesperson will contact you. :)

Re:Linux Networx

[Anonymous Coward]

>> However, if you inquire at the web site a
>> salesperson will contact you. :)

NOT!

You get (well, I've got) an automatic replay saying that "a salesperson will contact you within X days". 3 Months and a couple of attempts later... not a single word from LinuxNetworx.

Re:Linux Networx

[Snodgrass]

3 months?? Hmmm...somebody's slacking then. I don't know what else to tell you except you can try the 800 number...I'll kick up some dust and see where the ball was dropped.

By the tone of your question

[alen]

I bet you either work for the government or do IT consulting for them. Is it for some kind of scientific work or to help kill of the terrorists?

Re:By the tone of your question

[psychalgia]

interesting thought, im right behind you. Perhaps an encryption cracking machine? hmmm....

Re:By the tone of your question

[Anonymous Coward]

"They need it for theire cow: bovinne moooo"

Re:By the tone of your question

[Anonymous Coward]

Most likely, nothing so special....
Gentlemen (and ladies), it is near the end of FY 01 and in the government, that is when the bean-counters find all the unspent funds and say, "USE IT OR LOSE IT!"

If you have ever worked in the government, you know the story:

DON'T SPEND!
DON'T SPEND!
DON'T SPEND!
DON'T SPEND!
DON'T SPEND!
September 15ish
SPEND WHAT YOU GOT AND THIS OTHER $20K!
OH, IF YOU DON'T SPEND IT TODAY YOU WILL LOSE IT!

In the words of an old drill sergeant:
"Ask me how I know!"

Re:By the tone of your question

[alen]

Actually at my last posting in SETAF the planning started 10 months out. Every one put their dream sheet in for unfunded projects months out and hoped for the best. That's how they found the money to dig up the streets and lay fiber.

Re:By the tone of your question

[psychalgia]

heh, well, its that way EVERYwhere else too, not just gvmt.

Can you imagine...

[Anonymous Coward]

...a Grendel un-cluster of these?

No need to do research ....

[Anonymous Coward]

just imagine one.

Don't forget Scyld.

[pi_rules]

Check out Scyld [scyld.com]. If I'm not mistaken Donald Becker (one of the founders of Beowulf) is the head of the company ... or at least has something to do with it.

Re:Don't forget Scyld.

[Anonymous Coward]

Donald Becker ? Isn't he the guy who wrote the linux kernel source code for my shitty rtl8139 chip (nic) and my shitty via rhine chipset (on my dlink nic)...

Man... that code is so fucking buggy and each time he touches it (every new kernel release has a modded rtl8139 driver), it fx*cks up even more. And now I find out that he's the founder of beowulf... Now I understand why he keeps messing up the code... he has too many damn things to take care of... !!!!!!!!! ARGHHHHHHHHH

Re:Don't forget Scyld.

[becker]

The rtl8139 driver code keeps churning exactly because I don't have control over the changes. Look at who is making the modifications -- it's not me.

Many of the changes do come from my driver releases, but are incompletely or incorrectly copied over the driver in the kernel.

The was all part of the kernel development fall-out about two years ago. I only wanted to release working drivers into the kernel, while Linus wanted all development moved into the regular kernel releases. Linus just started accepting patches from anyone, which resulted in much lower quality drivers.

Re:Don't forget Scyld.

[jfunk]

Hmmm, I have 2 8139s and a Rhine and I have never had a single problem with them other than the fact that I get about twice the performance out of the Rhine...

I also upgrade my kernel regularly.

Are you sure that your cards or MB chipset aren't bad or anything?

Also remember that DB wrote many rock-solid drivers for high-end cards like 3Com, etc. I have some of those, too.

Besides, when you buy cheap-ass $5 NICs and they don't work properly, you should blame the cards first.

Re:Don't forget Scyld.

[evil_one]

Donald isn't the maintainer anymore. Starting with (I think?) 2.2.19 it has a different maintainer, and although the drivers rock at 100mbps, there are issues with 10 mbps. Consider replacing them with Becker's drivers, which you can find at scylid.

Re:Don't forget Scyld.

[Anonymous Coward]

becker dude - most of us appreciate the work you put into it. your 3com drivers are especially rock solid and i dont touch networking hardware that doesnt have a becker driver.

Applications?

[jnik]

As other have pointed out, getting the cluster and getting it running isn't the hard part. Parallelizing your application, choosing a parallel library/utility, and coding to it will take a lot of time. A beowulf cluster is basically just a big bag of Linux boxen until you code specifically to it.

Of course, if the issue is just spending the funding fast, go right ahead. Don't expect results for a long time, though.

Just wondering...

[teaserX]

Would 2 Quad boxes be better than 4 duals? I don't think they're *much* more expensive (per slot)and administering two boxes might be more practical than four. The performance gained by eliminating communications overhead may make for a desirable price/performance. I don't know; I was just thinkin'...

Re:Just wondering...

[zaius]

Two quad boxes would theoretically outperform 4 duals, because you have more bandwidth between the processors. Of course, if it's not a parallel task in the first place, extra processors won't give you anything.

You're wrong about the price though. Two quad boxes would be _much_ more expensive than four duals, because you can no longer use your traditional PIII/Athlon processors. You have to up it to an Alpha or Xeon. Both those processors and their respective motherboards cost 1.5-3x the price of PIII's, for the equivalent number of processors.

The point of Beowulf clusters is to be able to put bunches of computers that are not very powerful in and of themselves together to make something really powerful, so using quad processor boxes kinda defeats the purpose. The ideal price/performance balance is two processors per box, and this is easy to see by looking at the commercially available solutions.

Re:Just wondering...

[RollingThunder]

Probably depends where most of your collisions take place, but I do know that when you have multiple Pentiums (not sure on Xeon based), that when one CPU is accessing the memory or I/O, the other has to sit and wait. That's one of the big benefits of a multiple Athlon system - both CPU's can access memory simultaneously.

We bought one of the APPRO 1U dual TBirds, and this thing -screams-. It also howls, but that's the four big blower fans. :)

APPRO is at http://www.appro.com, and Anandtech had a writeup on the server.

Re:Just wondering...

[j.e.hahn]

(Note: all of the below refers to servers from the Dell product line. It's what I use, because it's what my company buys. Other vendors are just as suitable, for the most part)

Hmm... I'd say no. We have a cluster of 8 quads, plus a bunch of smaller, older duals. They're beautiful clusters, but I'd rather have dual processor machines. Here's why:

1) A quad processor box is 4U of rackspace. I've never seen one smaller. They may be larger still.
2) I can buy a 1U dual processor box. This means 2x the CPU per Unit of rack. Hence greater compute density.
3) A quad processor box has greater bus contention than a dual.
4) If you lose a node, you lose a larger chunk of processing power, and it costs more to replace.

In my mind the ideal node size, at least with commodity hardware, is dual. (If you start talking really custom gear, then it's another story entirely...) However, YMMV.

Linux Labs

[jjo]

Linux Labs will happily sell you clusters, either in standard or custom configurations.

My rough estimate from reading their website is that a 10-node 1.33-GHz Athlon cluster from them would price out at something like $16-17k.

Missing Critical Linux

[Outland Traveller]

I've run into guys from an company called mission-critical linux www.missioncriticallinux.com at the local LUG meetings. I know that they do custom clusters. Perhaps they can help?

See Commercial sites at bottom of www.beowulf.org

[bwhalen]

As stated previously look there...

Re:VANILLA ICE

[Anonymous Coward]

I find your assessment of Vanilla Ice's demise a bit hard to swallow ...

... not

What do you run on the darn thing ?

[beanerspace]

I'm glad someone is asking about the cost. But I think I've got the hardware thing worked out.

I've got a bunch of old Pentium I's and II's in a basement of a charity I do some volunteer work for. Can get a hold of a router. I've got all the nerd-boys a bit pumped to beowulf a cluster this winter when it's too cold to paintball.

So my question is, what to run on the blasted thing once you get it up ? Is there anything open source out there worth looking into ? Or am I just going to have to buy an application. If so, which one, how much ?

Re:What do you run on the darn thing ?

[psychalgia]

wow, you sound like my brother... are you adopted?! Beowulfs in lieu of paintball, and in charge of pumping up the nerds...like looking into a nerd-mirror.

Re:What do you run on the darn thing ?

[Anonymous Coward]

If you don't know what you need one for, then why do you need one?

Anyway, if ray tracing is your bit, look up PVM PovRay

Re:What do you run on the darn thing ?

[weakethics]

What a question!!
Hell, I don't need beer, but what's life without some fun?

Re:What do you run on the darn thing ?

[cr0sh]

Most of the apps for Beowulf clusters tend to be scientifically oriented apps. It really comes down to there being only certain kinds of problems that can be broken down into parallel processes that can be executed concurently, then the answers "recombined" at the end to get the result.

IIRC, Mosix allows for using the machine as one large machine, essentially allowing each process (or groups of processes) a single CPU, but funneling the result back to the main controller, so to the user, it looks like one large machine. This is different from a standard parallel processing Beowulf cluster, which behave like classical parallel processing machines.

For that domain (parallel processing), aside from coding your own stuff (hard to due, even if you are a master coder, from what I understand - due to having to understand what classes of problems can be broken down into parallel tasks, and then actually applying that knowledge to real tasks), there is one application that would prove to be "fun" - Ray Tracing.

Fortunately, POVRay has a paralleled version available, for Beowulf clusters. I don't have a link, but I know it exists (heck, you can probably get it at the POVRay site).

One other disclaimer: Everything I have said should be interpreted as "coming out my ass", simply because I have no experience at building or using Beowulf clusters or any other parallel processing architectures. Most of what I know about them have come from /. discussions, various web sites of Beowulf cluster projects, books, and magazine articles (as well as one funky book I picked up at the library detailing programming in Fortran for a Connection Machine - eeepp!). The POVRay stuff is true, though...

Re:What do you run on the darn thing ?

[tolldog]

As much fun as rendering seems over a large network, if you are really doing any major rendering and you don't have Cray-linked SGI's .. you are better off treating it as a NOW (network of workstations).
The POVRay rendering dropped off at 8 procs or so, if i can remember correctly.

We render a hand full of frames per box and do this on several boxes. This keeps the network overhead between machines low.

Sometimes when people talk about beowulf, what they really mean or really want is just a bunch of managed computers running tasks. If the task can be broken down at submission time, like most renders, then take advantage of a NOW.

Re:What do you run on the darn thing ?

[asland]

search for mpipov. Installed it on the sun cluster at my school the other day.

Re:What do you run on the darn thing ?

[ryanvm]

One other disclaimer: Everything I have said should be interpreted as "coming out my ass"...

Heh - I think with Slashdot, this is usually an assumed competence. But we appreciate you candidness anyway.

Re:What do you run on the darn thing ?

[duffbeer703]

No shit sherlock.

The guy knows what a Beowulf does, he needs somebody to BUILD one for him.

Goddamn karma whore.

Re:What do you run on the darn thing ?

[cr0sh]

Good points.

In a way, from what I understand, this is kinda what Mosix does for you, automatically, and and thus different from a true Beowulfed parallel processing app.

I guess it also depends on if you want to learn parallel processing techniques, or if you are trying to get a job done.

The drop off I would imagine is due to slow interconnects among the nodes - I am sure there would be better gains the higher speed your interconnects were.

Anybody know how the POVRay for Beowulf machines really works? Does it break the viewport into n sections, handing each section to a different node, or does it process multiple rays (ie, a node per ray), or what? Anybody got a link on this?

Re:What do you run on the darn thing ?

[SubtleNuance]

If you just want to tinker, and ball-busting computing power is unnecessary, you really shouldnt be building a Beowulf cluster.
Have a look at the info contained at MOSIX.org they provide kernel patches && apps to build clusters also, but these clusters allow -depending on your configuration (how/who/what of each computer in the cluster) - processes to migrate from 'more idle nodes'.

From what i understand, you could setup your 'head or workstation' machine to let procs dribble off onto other nodes.

have a look - much more usefull to geek exploration than beowulf... even if beowulf gets 'all the /. mentions'

Re:What do you run on the darn thing ?

[beanerspace]

..., then why do you need one?

Because we can !

Re:What do you run on the darn thing ?

[Anonymous Coward]

read his parent, you fucking retard.

Re:What do you run on the darn thing ?

[Anonymous Coward]

too bad that slashdot's shitty message board software and slower-than-shit bandwidth make it impossible to read entire threads, much less find the parent

Rendering, maybe...?

[Anonymous Coward]

If you're looking to use Renderman out of the box, take a look at:

http://www.terrasoftsolutions.com/products/briQ/ vi deo.shtml

Each node is about as big as a 5.25" drive (called Briqs), and you can cluster towers of eight together. They're cheap, too.

Presumably, you can cluster these for different purposes, too.

Aspen Systems

[iso]

There's an ad in the latest Embedded Linux Journal for a company called Aspen Systems (1-800-992-9242) advertising Beowulf clusters. Thtat's about the sum total of all I know about them ;).

- j

Cheaper solution

[NumberSyx]

Better and cheaper idea, do it yourself. Instead of buying 10 dual proccessor systems at $2000+ each, goto Fryes and get 30 Emachines or whatever they are selling for $299, add a couple hundred dollars for 30 decent network cards, one monitor for the control node (barrow a couple more to do the installs) a few hours to install and configure RedHat 7.1, which comes with the clustering software and you'd be done by morning. You will probably get better performance at half to two thirds the cost. This is what clustering is all about, turning cheap off the shelf systems into a super computer.

Re:Cheaper solution

[dsb3]

> This is what clustering is all about, turning cheap off the shelf systems into a super computer.

No, this is not what clustering is about. Beowulf, specifically, is about gaining raw performance without concern for stability or fault tolerance/redundancy.

There are many other uses for and of clustering; not all of them are concerned with speed (your super computer quotient, if you will), less of them are concerned with saving money.

Granted, the original post specifically mentioned Beowulf, but from that one should not assume that clustering == Beowulf.

Oh, and why on earth do you need a monitor to do the install??

Re:Cheaper solution

[markhahn]

beowulf isn't just clustering, it's compute-intensive clustering. that means that run-of-the-mill 100bT cards will likley not be enough - think Myri2000, SCI, Quadrics, gigabit.
whitebox machines also tend to be truely horribly configured - you probably don't want UMA video stealing dram bandwidth, for instance. and even for a smallish cluster, it's smart to get better power supplies and fans, simply because the noname stuff has short expected lifespans.

Re:Cheaper solution

[huckda]

Oh, and why on earth do you need a monitor to do the install??

I'll tell ya why you need a monitor to do the install...

so that when the first machine is up and running you can download ROBOCODE and do some javabot simulations on your new cluster and see how many battles you can simulate in 1 hr. =)

JOY JOY HAPPY HAPPY JOY JOY!!!

Re:Cheaper solution

[i-sob]

The $299 is more than likely the price AFTER a few hundred dollar rebate that requires a long term contract with an ISP (CompuServe, MSN, or AOL most likey) ...now what would this company that obviously has serious work to do on a Beowulf cluster want with 30 four-year subscriptions to MSN? :)

Re:Cheaper solution

[Jeff Knox]

Not true. You can get eMachines at fry's for aroud 299 bucks (or the 3 I got for 375, with a 75 dollar rebate, NO ISP contract, plain old rebate) without any form of ISP contract. Add a 80 buck 17inch monitor and you have a nice system. I bought one for my girlfriend. And then a few more.

Re:Cheaper solution

[milkmandan9]

Forget the eMachines.

I've just finished building a (really small) 4-node cluster for some research I'm doing.

Each node is a Duron 750 w/256MB RAM and a 100Base net card, power supply and floppy drive. They're vertically stacked on some 2' threaded rod from McMaster Carr.

But get this: I built the whole thing for $170 per node . That's everything, shipped to my door.

They load the kernel off the floppy and grab the NFS root from a fileserver I've got set up, so no need for hard drives. Granted, there's a bit more network and memory overhead (both /var and /tmp sit in memory), but I can buy more nodes from the money that I've saved on hard drives.

Re:Cheaper solution

[i-sob]

I'm impressed, and I stand corrected.

Re:Cheaper solution

[Cytotoxic]

milkmandan9's $170 per node cluster is worthy of a feature article all to itself. Somebody run over to his place and make him write it up!

Re:Cheaper solution

[tmuka]

I too would appreciate pictures of this creation. I am intrigued by your approach.

Re:Cheaper solution

[ErikZ]

I don't understand how you stacked motherboards on a rod. Got any pics?

fast and cheap

[AeiwiMaster]

How to make a Beowulf cluster in 2 hours for free.

1) Find a single machine to use as cluster server.
2) Install some linux (I prefers debian.)
3) Set it up as a diskless client server.
Debian have some diskless packages, but I prefers
to use cluster-nfs form clusternfs.sourgeforge.net
it makes it easier to maintain the system.

you can find a howto for setting up a mosix cluster
at the cluster-nfs site dropping the mosix stuff and installing
mpi and pvm should do it.

4) To use your groups computers as diskless nodes
go to http://www.rom-o-magic.net make some boot floppies
and use them to boot against your server.

5) Have fun with your new Beowulf cluster.

Whenever you need a Beowulf cluster just boot up
via the floppys.
In this setup it is only one the server you need to maintain.

Knud

Re:fast and cheap

[AeiwiMaster]

Sorry
The links is Cluster-nfs and rom-o-matic

Knud

Re:fast and cheap

[chriss]

it's http://http://www.rom-o-matic.net/

chriss

I wish

[Anonymous Coward]

I wish people would stop calling Beowulf a cluster, it's not a cluster technology!

We've bought from two places

[BrentN]

We bought a 168-node Pentium cluster from Atipa, and we're negotiating for a 1024-node (yes, that's right) Athlon cluster from Linux Networx.

Re:We've bought from two places

[Anonymous Coward]

This is Atipa. Can we discuss your 1024 node cluster?

http://www.ljworld.com/section/business/story/59 06 4

Please contact me at 888-222-7822 x3100

Linux Journal Ads for Clusters

[Duke]

RTFLJ :)
From the Oct. 2001 Linux Journal
http://www.microway.com - Alpha, Athlon & Pentium
http://www.aspsys.com - Athlon and Pentium
These ads mention clusters specifically.

Linux Networx

[Anonymous Coward]

They've been VERY responsive in my discussions.

www.linuxnetworx.com

Some vendors..

[pdiaz]

..listed here:

http://planetcluster.org/links.php?op=viewslink& si d=20

Hope that helps

APPRO AMD Servers

[Krieger]

You could purchase 10 APPRO AMD MP servers. These machines are based off of the Tyan boards so have all the built-in high end features you could wish for, while still being relatively cheap and easily networkable given the builtin NICs. Plus SCSI and potential for RAID if you feel you need it. and all in 1U.

http://www.appro.com/

or for the AMD systems specifically http://www.appro.com/1124.html

Should give you enough information to put together a 10U Beowulf cluster...

Re:APPRO AMD Servers

[tomblackwell]

We had bad luck with Appro servers in the past (non-Beowulf, though). They specified high-end processor cards and shipped very low-end cards. The Appro chassis became something of a joke around our office.

Who is this for?

[PhipleTroenix]

On Friday the government authorizes $20,000,000,000.00 for defense. The following Thursday a request goes out to procure Beowulf cluster in two hours. Let me guess this is to heat your dorm room right?

Your tax dollars at work.

Re:Who is this for?

[alen]

I totally agree. I spent 8 years in the army and the last four years at this link . October 1 is the beggining of the fiscal year for the US Government. That means every last penny from FY 2001 funds has to be spent or "lost".

I remember some cool shopping trips with my US Government VISA card in September. Basically we throw stuff in the cart just to spend the money. Whether you need it or not important.

Re:Who is this for?

[demigod]

You don't have to heat the dorm room to much.

http://www.terrasoftsolutions.com/products/briQ/po wer.shtml

Don't know that it's a beowulf cluster though.

Re:Who is this for?

[Anonymous Coward]

Does it bother anyone that he's doing this with a US GOVERNMENT (== "My Tax Dollars" for us US citizens) credit card? Maybe we should save this money for something...maybe disaster relief funds?

Re:Who is this for?

[Anonymous Coward]

So tell me, what did you buy with my money?

Re:Who is this for?

[alen]

Scanner. Some software. Don't really remember. It's the way things work. If you don't spend your budget, it gets cut next year. And nobody wants their budget cut. So you find a way to spend the money. I didn't like it, but I didn't contrlo the money.

My first job out of the army was consultant for a gov't agency. They didn't have the same mentality. Maybe that's why we had a surplus.

Beowulf Cluster

[Anonymous Coward]

a ~2CPUx10-node Beowulf cluster. Wow. Just imagine a beowulf cluster of these

Re:Another good turd today

[Anonymous Coward]

You're drunk already? On a Thursday afternoon?!

can you imagine...

[Anonymous Coward]

...a beowulf cluster of these?

Bret

[Anonymous Coward]

Look at the brain on Bret!

Re:Bret

[Anonymous Coward]

Brad.

It's look at the brain on Brad....

RTS+FPS Sim

[Da VinMan]

Not encryption... modeling of the ground actions to be take in Afghanistan or other areas. Probably need to know just how hard it's going to be. Couple the micro-management concerns of a first person shooter with the strategic elements of a real-time strategy with the parallel concerns of mutiple agents in the field and you could probably simulate a proper battlefield in short order.

Hmm...

My $0.02 worth of a guess. :)

Re:RTS+FPS Sim

[Anonymous Coward]

Um, the people who would be modeling that already have computers that would make a 10-node Beowulf look like a cheap digital wristwatch from the 80's. This is probably a government person who found some end-of-year cash and needs to have the hardware delivered by cob Sep 28. Bt, dt.

Yeehawww!

[Anonymous Coward]

Imagine a Beowolf Cluster of THESE!!!

Scyld

[4of12]

Haven't dealt with them directly, but I believe they have Don Becker, one of the Beowulf pioneers.

I think they have developed a system to help provide a single system image, along the lines of MOSIX, but not MOSIX, IIRC. This can help managing such a cluster which could otherwise be like managing 10 separate machines - a hassle.

Plug for Microway computers

[Anonymous Coward]

I'll put in my plug for microway computers www.microway.com They've given me great service ... damnit, processor doesn't work. Replacement was in the mail the same day. Quick turns, pretty good at doing beowulf stuff. Also my favorite supplier of alpha powered stuff.

RLX Technologies

[buffy]

Check out http://www.rlxtechnologies.com/. We've got Scyld stuff running very well. Processors are not very beefy, but we can get 24 of 'em in three rack units (or 336 in a 42U rack), and have some very nice tools to manage the whole shebang. Very little power required, easy to wire.

Just to say it out loud...I work there, so it's a blatant self-promotion, I guess. But it is a bad ass little product that packs a punch.

Wow!

[quintessent]

Just imagine...

a Beow...

oh wait. Never mind.

Imagine....

[Anonymous Coward]

hey, somebody had to say it! :-)

Re:VANILLA ICE

[robvasquez]

He works at the orange julius near my apartment, and does Malda's MOM

Re:Could you imagine...

[Lxy]

Hey moderators: I know you guys look for these posts just to mark them -1, but occasionally these are on-topic posts. This is one of those times. "Imagine... a beowulf cluster of these" is a staple here on /. and needs to be recognized every once in awhile!

uh

[Anonymous Coward]

the person posing this question to slashdot is obviously an illegal linux hacker that wants a fast beowulf cluster to illegally crack encryption. slashdot should not condone illegal activities by evil linux hackers. seriously, what else would an evil hacker need a beowulf cluster for? only thing i can imagine is so that some fucking nerd can get a boner over his FPS in quake.

What is turn-key?

[Spiff28]

Honest to goodness curious question that I'm sure the slashdot crowd could answer: what exactly does "turn-key" mean here?

Re:What is turn-key?

[Juju]

That all you have to do, is turn the key so that it starts. Think like a car not like a PC where you need to install stuff.

In other words, no installation, everything is configured to work out of the box...
Must be a very big box though... ;o)

Re:What is turn-key?

[Weasel Boy]

Their tech steps away from the keyboard, and you log in.

Re:What is turn-key?

[bn557]

I think it'd take 2 boxes, cuz you'd need to keep the power cables and ethernet cables separate(and those DAMMED user manuals)

...now turn to page 557 of the idiots guide to configuring a beowulf cluster...

Buying stuff NOW

[robvasquez]

I call PC Mall. Order by 10pm, get it by 10am the next day!

Amazing, if you think about it.

Buy it in two days

[tie_guy_matt]

Ah yes, you must work for the government.
Yes with the fiscal year ending we all have lots
of money to spend this year or we'll loose it
next year. Don't you love working for the government?
Sometimes I think the only job in America that
is better than working for the government is being a domestic cat.

LOL [NT]

[Juju]

I said [NO TEXT] ;o)

Western Scientific

[dieman]

Has nice setups. www.wsm.com

More like 3-4 grand for 1TB fileserver

[Jeff Knox]

You can easily build a 1TB fileserver (P3 1GHZ, or Athlon, 1gig of ram, etc etc) for 3-4 grand, using quailty IBM drives. None of that Maxtor garbage. Its probably closer to 4 grand with the ibm drivers, and closer to 3 with Maxtor drives.

Re:More like 3-4 grand for 1TB fileserver

[Anonymous Coward]

IBM drives = WD w/ new labels.

just make sure you have AMAZING backup solutions.

Other vendors

[Kismet]

Try Aspen Systems or Einux Network Solutions

Not sure how "out of the box" the Einux ones are, but you can get a 10 node dual-Athlon setup for around $25k.

Smaller companies

[Felidae]

You might want to check around to see what companies locally might make them. IBM clusters are avaliable through a number of smaller companies, most will ship them just about anywhere in the country, but you can save a ton on freight be finding one local. There is one in Houston, TX that I know of if you are in the Texas area, that is PCPC, Inc. (www.pcpcdirect.com) that custom configures and builds IBM clusters

YASOB: Yet Another Supplier Of Beowolves

[nels_tomlinson]

Microway also sells these toys. If you want Alpha, look here. If your budget runs more to Athlons, look here. Unfortunately, you will have to choose between 8 and 16 CPUs on the low end here; they don't have a 10 CPU Athlon cluster.

You should think a bit about whether the extra abilities of the Alpha boxes are worth the extra bucks for your application. One thing which I think that I remember about the Alphas is that they use a crossbar switch to link the several processors on a motherboard to memory, et cetera. This should give better throughput. They also have huge caches which should help with big matrices. I think that if you have lots of little problems which should be run in parallel, more nodes with lower price and capability per node might be the way to go.

I remember back in the days of the XT, Microway used to sell math coprocessor and video boards for PCs which cost more than the box you hooked them to, along with high-grade compilers which would put that hardware to work. They were once the place to get hardware and software for doing seroius number-crunching on a PC.

Re:YASOB: Yet Another Supplier Of Beowolves

[jaxon6]

i used to work there, so i know a bit about these guys. they don't have limits on what you get. they don't seel '8-node clusters' or '16-node clusters' only. if you want 10, you get 10. wanna add a cd-rw? sound card? kick-ass geforce3(they were fun to setup) i don't remember the days of the xt, so i can't comment on that, but they do a good job of linux clusters.

MicroWay (www.microway.com)

[Anonymous Coward]

I haven't used any of their stuff myself, but have heard good things about them from several different places.

They have Intel, AMD and Alpha clusters available, as I understand it, more or less out of the box, up to 64-way.

QLITech Linux Computers

[James Hetfield]

Looks like QLITech, (Made famous by buying out Tuxtops this spring) is offering clustering boxes these days as well.
1U/2U AMD/Intel CPUS, and Gigabit or Dolphin Interconnects. Cool Stuff.

I've dealt with these guys for servers and a Laptop, pretty good pricing, and damn good service.

http://www.qlitech.net

Re:Cheaper solution: WOW!

[Anonymous Coward]

WOW! Pictures! WOW!

Great job! How about some pictures of this 'El Chipo deluxe'

750MHz Athlons...fast, and good floating point... How much cache memory and the FSB bus speed?

YellowDogLinux G4 Cluster

[Magus]

b riQ/</A>

This is really cool, 8 node cluster size of a regular Tower. Each node is the size of a CD-rom drive. And I believe that they will soon be putting out Black Lab linux which looks to be a Cluster software package. If these wern't so expensive, I would get one now.

Re:YellowDogLinux G4 Cluster

[Knobby]

You can buy the BriQ's from Total Impact, but they're not cheap as you pointed out.

I'm glad to see that TerraSoft is working with TotalImpact. They were selling rack mounted clusters comprised of g3's ripped from iMacs.. This is definitely a step in the right direction!

wow!

[Anonymous Coward]

Man, just imagine a -- oh wait -- never mind.

Did you try ebay?

[Anonymous Coward]

Well, didja?

Cray sells Beowulf clusters

[totallygeek]

Here is an article in which Cray builds clusters. Cray is most known for supercomputing.

off topic: instability, windows and science

[nels_tomlinson]

Your sig line mentioned that someone should do a paper on windows instability. Here is one of them. It's the third in a series.

One portion which shocked me was:

Our final piece of analysis concerns operating system crashes. Occasionally, during our UNIX study, tests resulted in OS crashes. During this Windows NT study, the operating system remained solid and did not crash as a result of testing. We should note, however, that an early version of the fuzz tool for Windows NT did result in occasional OS crashes. The tool contained a bug that generated mouse events only in the top left corner of the screen. For some reason, these events would occasionally crash Windows NT 4.0, although not in a repeatable fashion.

They crashed a unix os? Wow! That doesn't match up with my limited experience. The only way I've ever done that was by trying to do stupid things as root, like running mindi with a buggy kernel. I wouldn't have thought that this would be a problem for a normal user.

Here is something which didn't surprise me at all:

Our 1995 study found that applications based on open source had better reliability than those of the commercial vendors. Following that study, we noted a subsequent overall improvement in software reliability (by our measure). But, as long as vendors and, more importantly, purchasers value features over reliability, our hope for more reliable applications remains muted.

Mine, too.

Re:off topic: instability, windows and science

[Anonymous Coward]

Well as someone at uwisc, its more the difference between API and OS design...

For instance his fuzz tester does things like slam system calls with random values, does wierd things to files, moves the mouse around a lot, clicks the buttons, hits keys, yadda yadda...

But he doesn't actually try to DO anything :) [both he and his wife do have Solaris boxen on their desk if I recall correctly...]

a bit of overkill perhaps...

[neurovish]

...but, as one other post pointed out (only one?) Cray has mentioned plans of selling beowulf clusters utilizing alpha processors. If I had the money to throw out my car window, I'd go for one of these.

From the Scyld website

[evil_one]

Scyld Vendors Scyld Beowulf Professional Product Scyld has partnered with industry leaders for them to provide Enterprise Level systems which consist of pre-integrated, supported hardware systems loaded with the Professional Scyld Beowulf:

• Compaq
• PSSC Labs
• Penguin Computing
• eLinux
• Aspen Systems
• Custom Fit, Inc.
• Atipa

yelldoglinux.com?

[Anonymous Coward]

Tried yellowdoglinux.com they suposedly sell moderatly priced systems witch could be noded indefinatly, for about 2k a node.

A suggestion to save cash on power consumption...

[SpamapS]

If this is going to be a sustained computing effort, you can save a lot of cash on power consumption by going with the RLX servers. RLX Servers They use Transmeta's Crusoe processor, so they actually can pack 336 of these in a single rack! Definitely worth a look.

get it all from the web...

[RadioheadKid]

First you could got here
And then here

How hard can it be, it's not like you're launching rockets...Wait a second, 2 days to buy, quick access to cash, damn he probably is launching rockets!

www,openclustergroup.org

[Glasswire]

download, install.

re: turn-key beowulf cluster...

[christopherwg]

any particular reason to specify a beowulf cluster?
and such a specific request for number of nodes too...
i didn't have the time(or patience) to read _all_
the replies, but i didn't see any mention of
project appleseed anywhere. goto:

http://exodus.physics.ucla.edu/appleseed/applese ed .html

and add this cluster configuration to your comparison matrix.

your .sig

[Anonymous Coward]

I'm the one seeking up behind you with the chainsaw and an invisibilty cloak.


seeking or sneaking? Did I miss a joke somewhere?

Re:your .sig

[choprboy]

No, you didn't miss anything... Apparently I'm missing my dictionary though. In case anyone wondered about the origin, I used to play way to much xEvil.

Spelling police!

[Anonymous Coward]

One entry found for accurate

Main Entry: accurate
Pronunciation: 'a-ky&-r&t, 'a-k(&-)r&t
Function: adjective
Etymology: Latin accuratus, from past participle of accurare to take care of, from ad- + cura care
Date: 1596
1 : free from error especially as the result of care
2 : conforming exactly to truth or to a standard : EXACT
3 : able to give an accurate result
synonym see CORRECT
- accurately /'a-ky&-r&t-lE, 'a-k(&-)r&t-, 'a-k(y)&rt-/ adverb
- accurateness /-ky&-r&t-n&s, -k(&-)r&t-n&s/ noun

www.webster.com is your friend. :)

Re:Spelling police!

[BardicStorm]

One entry found for inaccurate.

Main Entry: inaccurate
Pronunciation: -'a-ky&-r&t, -k(&-)r&t
Function: adjective
Date: 1738
: not accurate : FAULTY
- inaccurately /-ky&-r&t-lE, -k(&-)r&t-, -ky&rt-/ adverb

www.m-w.com is your friend. :)

Re:Spelling police!

[bn557]

copy and paste... what I did for my sig. I don't fix other peoples typoEs. I'm knot a secratarie

Re:Cheaper solution: WOW!

[milkmandan9]

Pics of the $170/node cluster threaded together.

FSB is 133, if I'm not mistaken. Cache is ... whatever is on the Durons. Sorry I don't know off the top of my head.

Been There Done That

[elinenbe]

This summer I was employed at a finite element analysis company in Philadelphia, and I designed, budgeted, and built a 10 processor AMD System (Octavian) based on gigabit ethernet, 1.2 Gig MP AMD chips on Tyan mobo's with half a gig of RAM per node, and it did not cost more then $13000 plus some setup time. (This was with a gigabit switch, etc AND the current cost with dropping prices would be less then $10000) The computer was designed to run LS-DYNA (a Livermore software finite analysis program) and it has not let them down.

Here are some benchmarks:

Octavian Benchmark runs
as of 8/17/01

Problem description:
Acetabular cup with a spherical metal ball compressing the liner into the
shell.
The effect of holes used for screw fixation to the bone is included
Total Mesh Statistics
45814 Nodes
37696 8-noded solid elements
2 contact pairs
Dynamic Relaxation Solution:

Execution Statistics
2 Processors : 62 Minutes
6 Processors: 34 minutes
10 Processors: 24 minutes

Analysis of the execution throughput indicicates a linearly increasing speed
(1/wall clock time)

And here is a review of the results:

the cluster is performing BEAUTIFULLY, and we have been crunching problems on it pretty much nonstop since it was brought online. It has really saved our butts, as we could never have met some key project deadlines without the speed. I've included some bechmarking stats below FYI for a contact problem that took only 24 minutes to run on octavian using 10 cpus. The comparable time on our $50K dual CPU octane workstation is 3 hours and 26 minutes, which translates into a speed up of 8.6 times for about 1/5 the cost.

So, What I am trying to say is build the thing yourself. You will know much more about the system, and you will be able to install any software you want to without having to deal with "customer service". Also, it will save you a bundle as a turnkey solution is nearly 3 times the price. (Even if this cluster was built with Myrinet it would still be far less then any of the pre-built solutions) Lastly, design the cluster for what you need. If your problems involves lots of RAM, then spen money there. If CPU is the bottleneck spent money there. If communication is the bottleneck....

Best of Luck,
Eric

Terra Soft-pre configured Black Lab Linux clusters

[Douglas]

Complete 8 CPU cluster, $17319.00 + Shipping

THE briQ Cluster

8x 5.25" slit case
8x PowerPC 7400 (G4) 500 MHz
8x 1MB L2 Cache at 250 MHz
8x 512 MB SDRAM, 3.3V (up to 8 GB per cluster)
8x 2.5" IDE 40 GB, internal to each briQ
4x 3.5" Ultra100 IDE 130GB, in alternate slots
with 4 briQ CPUs
300 Watt power supply
Internal 8 port 10/100 Ethernet switch with
Gigabit Ethernet upliink.

Installed Software:
YDL 2.0 or Black Lab Linux

PowerPC 7400 (G4) 500 MHz
1 MB L2 Cache per CPU at 1/2 processor speed

Northbridge - IBM CPC710-100+
100 MHz 64-bit 60x System Bus
Southbridge - Winbond W83C553

2x168 Pin DIMMs, up to 1Gb of SDRAM
Open Firmware Boot ROM/BIOS, supports remote boot
up to 40GB 2.5" IDE internal to the briQ case
up to 2 3.5" IDE external to the briQ case
16550C UART, DB-9 connector
10/100Base TX Ethernet, Dual 10/100 optional
Power Consumption: 20 to 40 Watts per CPU node
64-bit, 66mhz PCI, Custom 66MHz PCI connector
Programmable Vacuum Florescent Display (VFD)
2 lines X 20 characters Two push button switches
bi-color LED

Sure, I can sell you one in less than an hour, but

[(H)elix1]

but it may take a while before it gets shipped (grin)

rm -rf /bin/ladin

Hello? (Totally OT - Mod me down, I need the fun!)

[cr0sh]

Hello?

I wasn't replying to the article, but to an individual who was asking what he can use a Beowulf cluster for, as he obviously didn't know.

I am the last person needing to whore, since I am capped, and been that way for months...

Re:Cheaper solution - add shared storage

[paled]

does anyone know of an affordable shared storage solution for this?
I know that SCSI can operate on a shared bus for 2 nodes, but I'd really like to test out Oracle 9i Real Application Cluster (RAC) (formally Parallel Server) on a 4 node setup on Linux - but would need a shared storage device for log files, data files, control files.

FC would be a little pricey.
NetApp filers are outta my league.
thanks.

Well..

[mindstrm]

Buy ten machines, and build it yourslef? I mean.. that's all a beowulf is... it was a project to work on using off the shelf hardware for parallel processing.

DO you have some app that needs it? I mean, you can't just run anything....
Did some department just come up with some app already designed with the PVM libraries or something?

Hmmmph..

[Da VinMan]

Yeah, you're most likely right of course. But, my idea is much more fun.

Think about it though, how likely is the military to be in possession of simulation software like I mentioned with full-out graphics capabilities that actually runs on those spiffy supercomputers they own? Maybe very likely. If not, then a Beowulf cluster is the perfect starting point along with existing code perhaps from the likes of Loki.

Hmm....

;+)

What's it for?

[Mister Transistor]

If you are looking for a huge webserver or FTP server or such, then maybe a Content-addresable switch is the thing for you, and you really don't NEED a "beowulf" cluster to achieve your goal.

I recently set up a "web farm" of 10 Linux boxes, using off the shelf Compaq DL580 servers (running SuSE linux, pretty much straight out of the box). The magic component that made it a "cluster" was the Content-addressable switch which sent incoming web traffic in a load-balanced fashion to the 10 stock web servers.

Finally, the serial consoles of all 10 were run to a terminal server hooked to a stock DL360 running Red Hat for access to the consoles of the 10 servers, if necessary.

I can have that up for you in 3 days from receipt of the hardware :)

/. helps war effort!

[gsfprez]

or was it so obvoius that he's asking this for those reasons?

I'm not complaming - i'm just finally glad to see /. as something USEFUL.

fight on!

Scyld Computing Corporation

[danFL-NERaves]

They gots the mad scylds.. Scyld is a highly regarded vendor in the Beowulf field. They host the beowulf maillist (beowulf.org), offer technical certifications and work with the open source community. They are all that and a can of Moxy.

Scyld is a software company so they don't sell directly. To get a turnkey system they have partnered with hardware vendors who will sell a pre-integrated cluster along with hardware support and Scyld support for the integrated Beowulf Professional Edition software. Visit their site at:

Scyld Computing Corporation

Or go directly to their partners:

Compaq

PSSC Labs

Penguin Computing

eLinux

Aspen Systems

Custom Fit, Inc.

Atipa

Dan

PPC would use far less energy

[Anonymous Coward]

The Power3 and Power 4 processors use less power to run than a P3 or P4 chip. Remember you are using an entire cluster of these computers and someone has to pay the electric bill. And I hope you have good ventilation or air conditioning.

+2 Funny on the MQR standard

[MarkusQ]

Who modded this down?!?

It's a joke for gosh sake; he was playing off the first person's Grendel reference by alluding to The Three Billy Goats Gruff!

-- MarkusQ

Turn key is great but,

[levinas]

What I would rather see is a software rather then a hardware solution to beowulf. Right now to make use of a cluster one needs to have the know how to compile software to work in that environment this limits the usage of clusters to people in the know for example universitys. It would be better if more work was done in the creation of a linux distro and some generic applications(no doubt one already exists).

btw, just IMHO but isn't the whole point of creating a cluster to make use of your old hardware that is gathering dust rather then having to buy a mainframe. Why buy a dozen ghz pentiums when you can get a thousand 486's from a scrap dealer?

thanks

[Anonymous Coward]

Mr. Becker: thanks for the ethernet drivers. Obviously not everyone apreciates the hard work you did.

My $0.02

[bogdant]

I do think that getting a Beowulf in two hours is very wise. This is not like getting a new PC. You gotta consider the application(s) that you want to run on that cluster, and what kind of network infrastructure you need, and a zillion other things.

Anyways, AFAIK GSH sells PC & SUN clusters. You might wanna check them out.

You can also visit http://www.parallelcrunchers.net for more infos related to Beowulfs.

This is technically off-topic

[biglig2]

But am I the only person here to think "working on my morning M&Ms?"

Call Dell

[Anonymous Coward]

Many people do not know this but Dell sells very nice turn-key clusters. They are working with a company called Paralogic These guys know what the are doing.

Take a look at this

[Anonymous Coward]

If you have a serious interest in Beowulf then you may want to read this link

why not a Mac?

[mr.ska]

I've seen a few stories about researchers looking for a cheap, quick, and powerful cluster for their research purposes who decided to go with Apple hardware. One group got an 8-node iMac cluster up and running in less than a day (which I can't find the link for.. sorry).

A quick search of Apple's site actually mentions clusters [apple.com]. Perhaps 10 dual G4's would suit your needs?

A couple more vendors

[Anonymous Coward]

These guys seem professional.

www.linuxnetworx.com
www.plogic.com

I want more pics! And a write up!!!!

[ShaneTheGeologist]

Please!

It's gotta be said.

[Penguinoflight]

Imagine a Beowolf Cluster of these!!

Re:Maybe something here ... [disk images]

[Anonymous Coward]

i did this, it worked. i even used hard drives of different sizes. i was very suprised. i wanted to make two test machines, one for sendmail, the other for qmail.

Beowulf

[PegQuin]

Call Eric over at eLinux.