Slashdot Mirror
SGI NUMAflex Linux System On Display @ SC2002
jarrod.smith writes "
According to SGI will unveil
its Intel® Itanium® 2
NUMAflex shared-memory supercomputer architecture (which runs Linux as its OS) at Supercomputing 2002 which runs this week in Baltimore, MD.
The link at SGI says the system will be on display at the show. The exhibit floor opens this evening. Unfortunately I did not go this year. Can those lucky enough to be at the meeting scope it out and post comments?"
Additionally, it offers unparalleled scalability in the line of Linux supercomputing. This is a system built to grow with a business, although your business better be pretty much grown already to back the check you'd need to fill out to buy it.
My conclusion: it's an excellent largish solution for academia seeking a more stable environment than can be achieved with Beowulf clustering and excellent pricewise solution for businesses seeking to expand without sinking a lot of money into unnecessary costs.
(which runs Linux as its OS)
WRONG! It runs linux as it's kernel.
actually, OSCAR is a GNU/Linux fork.
After all, most GNU/Hippies are grouchy and live in a garbage can.
I've been hanging out at SC2002 all day, and I can tell you that nearly every booth on the show floor is showcasing Linux. Of course all the Linux cluster vendors have it, but so does sgi, Sun, IBM, Intel, AMD, HP, Compaq (separate booths - guess the merger isn't *really* done yet), and all the smaller vendors, to say nothing of all the research labs, etc.
Large linux systems and clusters are really all the rage right now in SC circles. I think the only booths I saw here not using Linux were the Apple booth (though they did have one gorgeous brand-new G4 running Xfree and twm, the sick bastards!) and the Japanese manufacturers NEC and Fujitsu (off in their own worlds, as always).
Linux isn't a big surprise to the SC set, though - this is a group that's used to UNIX. Hell, Microsoft doesn't even have a booth here, and they were at the last LinuxWorld conference.
-Isaac
I am not a lawyer, and this is not legal advice. For Entertainment Purposes Only.
...okay so Linux is being applied to all these terrific projects of scale both large and small. Is it because it's an open system with seemingly hyperactive development or is it because it's simply better than anything else out there?
Linux is being used because there's no x86/Itanium port of Irix. SGI use Irix, which as of 6.5 is a superb Unix implementation, on their MIPS hardware. IBM use Linux because of all the software that's available for it, but Linux runs within a virtual machine on top of their proprietary zOS.
XFS has already made it into Linux, maybe some other Irix stuff like GRIO will be next.
You pay a lot of money to get a very large computer that can do very large tasks very fast.
Wasting 20-40% of the resources of your $2k desktop on your OS's feature bloat may not be too bad, but wasting 20-40% of the resources of your $5 mil supercomputer is a lot of money.
Or put another way, Linux is used in supercomputers because it can be set up to do exactly what you want it to, and ONLY that - which for most HPC applications is compiling and running custom code to solve Big Problems.
You're not going to use a 512 processor supercomputer to Save Christmas by being able to get those pictures off your digital camera without spending 3 hours trying to download the drivers.
paintball
http://www.microsoft.com/windows2000/hpc/
Cornell has some windows clusters that they seem to like ok.
http://www.tc.cornell.edu/
My opinions are my own, and do not necessarily represent those of my employer.
The current generation of SGI NUMAflex based machines use a mesh of full duplex 3.2 GByte/sec interconnects. That's 25.6 Gbit/sec.
:)
That's way more than 3 times. Plus the latency is several orders of magnitude less.
The tradeoff is cost. A fully populated rack (32 Itanium2 CPUs or 128 MIPS R1x000 CPUs) starts at $1M can can easily run upwards of $4M. If your task is CPU bound, then a homebrew cluster will be almost as good. If your task is I/O bound, you can't beat the Origin. Until the Cray X1 ships, anyway.
Also keep in mind that while an Origin system can be partitioned, they are typically run as one single image system. The beasts easily expand from 2 CPUs up to 512 (even 1024 with special support from SGI). The cross-system memory latency increases with the larger configurations, but the net cross-section bandwidth/thruput increases linearlly with the CPU count. Very efficent design.
Pretty sweet machine. Again, until the Cray X1 ships!
1. Raw MHz means nothing. SGI's MIPS-based machines perform excellently, at the top of the Spec2000 benchmarks, and extensively blow away both x86 and Itanium I. There isn't enough data yet to draw any conclusions about Itanium II. I would take an Irix-based Origin system any day over the completely unproven IA64.
Huh? There's data out there, let's look at some of the SPEC results:
CPU specint specfp
600MHz R14k 483 495
800MHz Itanium 314 645
1GHz Itanium II 807 1356
So the MIPS CPU does pretty well considering its low clockspeed, but Itanium II has a much higher peak perf. When you combine that with the monstrous bandwidth of an O3K machine, you get something pretty powerful.
2. Having developed for NUMA architectures, I am confused as to why this machine is designed the way it is. Unless they've done extensive modifications to the kernel, and especially the brain-damaged Linux thread libraries, you're going to end up with what are supposed to be threads of the same process running with different memory access properties.
Linux 2.4 with the O(1) scheduler is pretty good at SMP thread scheduling, but since the latencies accross NUMAflex are so low, process and thread placement aren't as important as they would be on, say, an IBM NUMA box, which is much more 'non-uniform' wrt memory latency. The 2.5 kernel should be even more scalable, as it'll probably include a NUMA-aware scheduler.
3. Even more confusing, what little press there is on this machine claims constant data access to anywhere in the combined memory space. NUMA by definition is non-uniform memory access. What's with that?
I imagine they're referring to the fact that you don't have to use a message passing API to do your app development, i.e. all the memory of the machine is in one big address space. Of course accessing different nodes will result in varying access times, but that's obvious.