Recommendations On Supercomputing Hardware?

dameon asks: "I have been asked by my supervisor to select a replacement for our current SGI Onyx2 space heater. The current setup contains 24-195 Mhz IP27 processors, 12GB main memory, and around 140 GB of total storage space. We use it to run a bunch of CFD (computational fluid dynamics) code. Currently the demand on our system is so much that the jobs are backing up. So, they came to me with two quotes and said: "Which one is better?" I have had limited experience in the field of powerhouse number-crunchers. The two quotes I have received are from HP and SGI. SGI's quote is for: an Origin 3400 with 12 GB Memory, 24-400MHz/8MB R12K's, and 1/2 TB of storage space. HP is offering 3 9000 series N-4000's adding up to about the same specs in total, with the exception of the processors. Hp is offering 550 MHz PA8600's (1.5MB) processors in their setup (it also has more storage space setup with a hyperfabric configuration). All of the software we use will run on both platforms. So, I would like to put this to the Slashdot community: Which one is better?"

"The HP system is freaky expensive, but is the extra 150 MHz/processor worth the extra money? What else do I need to take into consideration? SGI's processors (while slower) have more cache. Overall, what do I need to look out for when spending this much money? What is the best deal? Am I missing another possible solution altogether? And yes, I already suggested a cluster of linux boxes similar to the one at Los Alamos, but the apps we use have no Linux support."

Comments

Re:I'd go with the Origin

[BluBrick]

Thanks for the info.

Yes, SAM is nice, and yes, most things on solaris do take days of manual deciphering to accomplish. However, I have heard (totally unsubstantiated, mind you) that, with its reliance on database configs rather than text files, AIX is quite difficult to admin w/o the aid of smitty or Xsmit(smitX?). That's my main beef. Working in a highly heterogeneous environment, I cannot afford to get too comfortable with "flavour specific" tools, lest I forget how to do it all at the command line.

That's why I like to install the GNU utils on every machine I admin.

Unix ain't Unix, but GNU *IS* Unix... sorry, RMS!

(Hey, I like that. Meet my new .sig :)

NEC SX serie

[Pierre Phaneuf]

Have you considered a NEC SX-5 or one of their variant? Even smaller models will shove an Origin right into the ground, and many commercial applications like ABAQUS are available for these.

NEC is involved in some Linux strategy, even at the level of its supercomputers. For example, the SX-5 can use an Intel-based Linux machine as a support system.

--
Pierre Phaneuf

End Result NOT Clock Speed Or Cache

[ONOIML8]

Maybe I'm way off base here, I usually am. But I think with this kind of hardware it's the same as with PC and Mac hardware, you can't judge a devices ability by the measure of its clock speed, buss speed, or cache size.

All of those things factor into how many instructions per second it can handle but you can't judge a device by that either. Little things like getting the data in and out need to be considered too.

The software you run might run better with one hardware setup than another too.

I don't think that you're going to know which one is better until you try each computer for yourself. Crunch the same numbers on each and see for yourself which one you're more happy with.

You wouldn't buy a car without a test drive, you wouldn't buy a house without a walkthru. Don't buy a computer until you've cunched some data. With the kind of money your company will be spending for the darn thing you're going to have to live with what you get for a while, be damn sure you're happy with it's performance.

Other quotes!

[Ripper]

I'd also recommend you get quotes from other vendors as well. I know IBM makes some serious hardware (SP2, the computer which beat Kasparov in chess...) for uses like this. Also, they announced a new model yesterday (S85), but it might not be suited for your needs. Go check out their site (www.rs6000.ibm.com) and contact your local IBM salesveasel. Also Compaq's Unixes might be good to check out. Someone mentioned Sun, but AFAIK they're more suited for ISP/ASP use, correct me if I've missed something?

Re:Other quotes!

[d^2b]

As a confirmed alpha bigot, I can confirm that I was pleasantly surprised by the IBM SP nodes. BUT, only the performance on your code matters. I still think for me that an alpha linux cluster is the most cost effective way to go; with the SP you are really paying for 1) the high speed interconnect 2) The fact that the software works nicely together. Other than that, it is just another cluster. If you haven't yet figured out the difference between clusters, uniform access shared memory machines, and non-uniform shared memory machines I recommend reading the first few chapters of "In search of clusters" by Pfister

Re:...cant you borrow them?

[Tower]

I'd definitely check out other vendors as well - IBM has a benchmark center for the AS/400 - I'm sure there is a similar deal for the RS/6k line (an SP would be really fun). See what all of the companies (HP, SGI, IBM) will do for you in terms of benchtesting *your* stuff on their systems. I would think they would be more than happy to help.

#include<std_disclaim.h> (Big Blue)
--

There ARE no others to quote!

[Bingo Foo]

SGI and HP may be the only vendors with shared memory machines this big. Certainly the IBM's are all distributed memory, except for their continuation of the Sequent line that they bought. Still, for interprocessor communication and memory accesses, nobody can touch SGI's NUMA on massive shared memory machines, whether your code is threaded or MPI.

I think you'll find that the 400MHz R12k's (of which we have 8 in an Origin 2000) kick serious ass. The huge cache and fat pipelines beat 800MHz Pentiums by a factor of at least four in our cursory benchmarks on some of our non-parallel codes.

Bingo Foo

---

HP thoughts...

[yakfacts]

I am working for a small company with needs for reasonable computing power and we just purchased a top-of-the-line HP workstation.

I am more of a hardware guy. This machine is very well built; not like the cheap 712/60's I've used in the past. The electrical and mechanical construction of this computer reminds me of the big iron machines of the past.

Plus, it's faster than hell.

So, from a hardware standpoint I love HP; it is sweet. But as others have remarked, HPUX is a pain to admin--at least for a HPUX newbie. I have admin'd Linux and BSD boxen for seven years, but HPUX is a new animal. If it ever gets even a tiny bit confused, it throws it's hands up in the air and vomits. This annoys me.

As far as SGI systems go, they have never struck me as very well built, even though I like them. A few years ago the University I was attending (getting a MS in EE) purchased what was then the fastest graphics computer in the world, a many (128?) processor SGI/Cray in nine racks. The dust level in our machine room (which was pretty high by computer room standards--it was installed by Univac for the 1108) caused it to fail within a month. Even with the dust problems fixed (the FS guys took the entire thing apart and vacuumed each board--plus replaced a lot) the machine has been a constant battle, and Irix issues cause it to reboot frequently.

An 1986 vintage HP9000 series 800 mini (the fastest box in '86) has been running nonstop in the same room for going on fifteen years with almost no downtime. Yeah, it's old and is more dust-tolerant, but it also seems better built.

Clearly I'm not an expert, but I would vote for the HP hardware over the SGI. Too bad you can't run your apps on the HP box with Linux installed...

We like SGI's (and Suns)

[jayteedee]

We do some heavy CFD work at my employeer too (Orbital Science Corporation). In the past, we have stuck with the SGI's after getting samples from each manufacturer and reviewing all of the offerings. SGI was bested only by the DEC Alpha machines (nothing like raw clock cycles), but the future (2 years ago) looked a little bleak for the Alphas, so we went with SGI's. They have held up well, and bely their meager clock cycles. SGI systems are always tuned to work as a system, and with CFD you are going to need it. High speed access to memory, cache, etc. Given the specs you mentioned, I would think the performance would be a wash (my 'educated' guess), so I would definately go with the lower price. I can't imagine you could go wrong. The newer PC on my desk is technically spec'd out to be twice as fast on a Mhz basis, but the SGI will absolutely clobber the PC on large computationally heavy Matlab scripts run on both machines. We are starting to comtemplate our next upgrade, and are leaning towards the UltraSparc III's. We have some beefy II's around here and they are doing real well. I'd at least look at Sun's offerings.