Slashdot Mirror
IBM unveils 64-way NUMA server; Promises Linux support
I just found this article at Info World which talks about IBM releasing a 64-way NUMA-Q server. The interesting part is that IBM promise to release a version of Linux optimized for
NUMA servers. What do you think about it?
Well, this isn't really an "IBM" system. It's a Sequent system which was far along in the design process when IBM bought them a year or so ago. There seems to have been a tremendous brain drain since the purchase (*), so this machine may be born as a legacy system.
(*) According to one of those drained brains, IBM didn't seem to have a clue what to do with them. Lacking any top-down direction, they tried to launch some bottom-up initiatives, which IBM management squashed.
- Old Man of the Mountain ---- "I want to disturb my neighbor"
Going for the geek familes market, Rowenta unwrapped two new toasters -- a 4 toast QUAD-BURNER and a entry level 2-way energy saving version. "We hope to give all the toaster lovers out there more satisfying experience, from bachelors to large grandma and grandpa and lot of grandchildren type families" said John Williams, who manages the R&D at Rowenta. Company officials also announced their intent to deliver a version of Linux optimized for QUAD-BURNER. "It sure makes sense to use open source OS to make open faced sandwiches," John remarked. /.!)
Rowenta is based in Standfort Offenbach in Germany.
J.
(mention Linux and get posted on
NUMA is not really a clustering method. It's a way of addressing some of the drawbacks of large-scale SMP. You have quads (4 CPUs), and each quad has memory, and cache. Memory access is non-uniform, because special techniques are used when memory is accessed between quads. Think of it as a distributed shared memory computer, all in one computer.
Each quad has connections to other quads and IO buses.
Because the quads are actually separate, you can subdivide your machine and run different operating systems on different quads, yet they can share (at bus speeds) data between them, such as a fibre-channel array.
And Dynix is a good UNIX, too. It has it's problems (like it's low on the port list of just about everything) but it runs all the GNU software I've tried on it and is very reliable.
"NUMA" stands for "Non-Uniform Memory Architecture". It's one approach to dealing with system memory on a machine with a large number of processors.
The idea is that each processor module has its own dedicated RAM, which can be accessed both locally and remotely by other machines across the network. System memory as a whole is the aggregate of the local memory banks on all of the processor modules. While this is all in one physical address space, access time will vary depending on whether you're accessing a local or non-local bank. Hence, "Non-Uniform".
There are undoubtedly extensions to NUMA that do more complicated things with system memory; this is just the version that I was told about at university.
We already have 7, count'em SEVEN FIRST POSTS! I wonder if IBM's including a 64-way First Post server with their NUMA boxes...
--Joe--
Program Intellivision!
I may be misunderstanding but, from what I understand NUMA is not strictly for multiple boxen. I was more under the impression it was the middle step between SMP and MPP clusters...
An actual NUMA-Q server is a (up to) 4-way Xeon box in a 4X rack cabinet. NUMA is software that lets a bunch of those share RAM and processor time. Sequent (IBM) recently overcame the old 64 processor limit on their NUMA implementation.
Maxed out, with the enterprise cabinet, 4GB of RAM and 100GB of storage, you're looking at hundreds of thousands of dollars.
Sequent's web site seems to be down right now.. (cough). Heh.
And, as to why you might want one, we average over 400 processes at any given time with a load avg of around, uhm, zero, on our production box. These things can take a LOT of abuse.
--
blue
i browse at -1 because they're funnier than you are.
Assuming that they've got this even close to right, managing a 64-CPU NUMA-Q system should be no more complicated than managing a 1 CPU NUMA-Q system. Until the sysadmin tools for Beowulfs get a hell of a lot more sophisticated, managing a 64-CPU Beowulf is going to be much more complicated than a 1-CPU Beowulf.
Again, assuming that they've gotten things right, programmers should be able to continue working with a model they know and have experience with. Applications have to be re-written (and frequently re-designed) from scratch to run on a Beowulf, and programmers need to use a totally different mindset. In theory, any application that works on an SMP should "just work" on a NUMA machine - possibly with a recompile. To really get peak performance, applications may well need some tuning, but that's certainly easier than rewriting.
I'll bet that when this machine ships, Oracle (just to pick a big example) will already be running on it. When will Oracle ship a Beowulf-aware version?
- Old Man of the Mountain ---- "I want to disturb my neighbor"
"it" being porting linux to a NUMA-Q.
(NUMA is a method of sharing CPU and RAM access across mutliple boxen)
NUMA-Q's are x86 boxen. They have some really, really, really cool features. I do wonder if IBM plans to write drivers for the FibreChannel SCSI adaptors and etc that come standard with most NUMAs.
OTOH, there is noooo reason not to use dynix on a NUMA. It's included with the (MASSIVE) cost of the box, it's based on BSD, it's a nice OS with tons of kick-ass features, and it's symbiotically enmeshed into these servers.
Hey, I wonder if IBM is actually gonna write a NUMA layer for linux? I mean, if they don't then all you end up with a buncha 4-way rack mounted linux boxes.. for $365,000 apiece.
One other thing, Sequent (now IBM) has the absolute best support I have ever seen. I have sent email to their web site about completely esoteric crap and had them call me back and get a dialogue open with the developer of whatever I was having trouble with. If you've got the cash and don't wanna deal with Solaris, DYNIX is the way to go.
--
blue, who is wearing his Sequent hat today.
i browse at -1 because they're funnier than you are.