Where to Spend $1M on a Cluster?

Natchswing asks: "My university has been given a $757,825 NSF grant to build, 'A 256 node (128 pair) Beowulf parallel computing cluster ... to improve the realism of gravity-wave modeling by permitting treatment of the three dimensional problem and multiple wave interactions.' They want to pay a company to just show up and drop off a functional cluster rather than build it themselves. Since word has leaked out regarding the purchase intent, every computer manufacturer under the sun (including Apollo himself) has called up trying to sell their cluster. Since I'm no cluster expert, I'm writing Slashdot. If you had $0.7 mil to buy a pre-built cluster who would you go with and why?"

Comments

Competitive Bidding

[duffbeer703]

Have the companies submit bids... then compare them and make a decision.

This isn't rocket science.

RFP is the answer

[hectorh]

I think that you should look at your intended application.

- How much disk space are you going to need in total?
- How much disk space are you going to need per node?
- How much RAM is each node going to need?
- Is your application going to benefit from a low-latency or a high-bandwith connection between nodes?
- What about cpu? which cpu family will provide the best bang/$ for your calculations? PPC or X86? x86-64 maybe?

Once you know what you need, put it together in an RFP and send it out to every company that shows up under a google search for "beowulf cluster"

Review the responses and pick the best.

Since you are asking this question here, I'm going to refrain from suggesting the better option which is to build your own.

Hector

Negotiate a success story

[elliotj]

Whoever you chose to go with (I'm partial to Apple, but that's just me - and just because they have sexy hardware), see if you can get them to give you either more for your money, or free implementation/consulting help, or something like that in exchange for using your implementation as a success story. I think Virginia Tech got a bunch of free stuff from Apple when they decided to build their supercomputer.

All these vendors want to be able to talk about their work. Letting them use you for marketing may help you get more for your money.

cluster experience

[Robbat2]

First of all, you really should put out an RFP for your cluster.

We've got a 128 node (1 cpu per node) cluster from Atipa http://www.atipa.com/ that cost CDN$ 0.25M.
128 P4 Xeon, 1GB RAM, 120Gb IDE, Gigabit Ethernet.
I'd expect you to get a lot more for your USD$ .75M, like maybe doubling your size and getting AMD64 nodes. Look at your primary problemset first, see if it's IO-intensive or CPU-intensive to figure out what you want in the way of disk/networking.

The only thing I don't like about it is Atipa's configuration of Redhat8 (they didn't offer anything newer at the time). Look for something newer there.

Atipa is one of the suppliers for SGI-branded clusters as well.

I'd really like a cluster from http://adelielinux.com/en/, but I wasn't aware of them at the time we did our RFP and cluster purchase.

Warning !

[dargaud]

Commercial clusters, hah ! My university did exactly that and they've had only problems. There was specialised hardware in it. It was never well supported by the Linux they installed on it, which was impossible to upgrade or change according to the admin who kept loosing hair on it. In other words that system never worked properly.

When my research group decided to build one, I was incharge, opted for OpenMosix and after a tweaking period worked really well. Now with the various bootable CDs with OpenMosix (PlumpOS, BCCD, Quantian, ClusterKNOPPIX...), tests and upgrades are done by just pressing reset !

Of course with clusters your mileage may vary.