Slashdot Mirror
SGI Introduces World's Densest Server
Twirlip of the Mists writes "Today SGI announced the Origin 3900 server, the world's densest computer. How dense? How about 16 MIPS R14000A processors and 32 GB of RAM in a 4-rack-unit 'superbrick,' for a grand total of 128 processors and 256 GB of RAM in a single rack. That makes the new machine the densest single-system-image computer in the world; it's even denser than most blade systems. Just for fun, the server also includes a whole bunch of 64-bit, 133 MHz PCI-X slots (from 11 up to hundreds and hundreds, depending on configuration). There's coverage of the announcement on ZDNet, CNET, and InfoWorld, as well as on SGI's own site."
I meant to mention this in my submission, but it slipped my mind. The R14000A only consumes 17 watts of power. Four of them, plus the Bedrock memory controller chip, plus up to 8 GB of RAM, fit on a board inside a 1 RU clearance. Four of them, plus some nifty backplane hardware, fit into a "superbrick," meaning sixteen processors in 4 RU.
As far as heat loading goes, the "superbrick" is basically one big wind tunnel, with giant fans on the front and ventilation out the back. It pumps a lot of heat into the room, but the temperature in and around the CPUs is really pretty low. I think it peaks around 35 C.
I write in my journal
(I'm answering these questions off-the-cuff, so if I mistype any details, sorry.)
If you know what a first-generation C-brick looks like, imagine squeezing that board into a one-rack-unit form factor and stacking four of them together.
Each superbrick includes four boards, spaced one unit apart, with four R14Ks, the Bedrock, and some RAM. The boards are connected with an internal eight-port crossbar router, making the superbrick a self-contained 16-processor unit. Externally, the superbrick connects to the base I/O brick via XIO+; the base I/O brick contains stuff like the system disk and the first 11 PCI-X slots.
I'm not positive how the superbricks are configured. Theoretically, you can partially populate them in one-node increments (meaning 4 CPUs and some RAM), but SGI may or may not sell them that way for manufacturing and QA reasons.
I believe the CPUs come with 8 MB of s-cache each.
The CPU-to-CPU and CPU-to-RAM bandwidths vary depending on the topology you're crossing, but I believe the minimum is 1.6 GB/s unidirectional, or 3.2 GB/s bidirectional. Intra-node bandwidths are somewhat higher, I believe.
No, the CPUs are regular single-core MIPS R14000As. They're tiny chips that don't consume much power, so you can really squeeze 'em in there.
Keep an eye on techpubs.sgi.com, because SGI will be releasing the developer and owner docs for the new system there shortly. (By "shortly" I mean as soon as a few hours or as long as a few weeks, depending on when the docs get released.) You'll find all the technical data you want when those docs go up.
I write in my journal
Sun fits 106 processors into a rack. They were previously the record holder. The Origin 3900 is considerably denser than the Sun Fire 15K, both in terms of processor count and PCI-X slot count-- though not at the same time, of course.
I compared the density of SGI's system to blade systems because those are widely considered to be the densest computers in the world, with something like 90 or 100 individual one-processor computers per rack. This system is not only dense in terms of pure processor count that most-- not all, but most-- blade servers, but it's also got all the advantages of a single system image for HPC applications.
I write in my journal
From their website: "The RLX System 300ex chassis holds 24 ServerBlades in 3U and supports the new ServerBlade 1200i." -- and it's even based on Linus's Transmeta chipset!
Not sure how Sun's server can top this... somebody help me out here.
Can you imagine a Beowulf cluster of SGI servers SHOVED UP YOUR ASS?!!!
thank you.
The difference? The SGI is a single system image, i.e one single computer, that can be used as a server. With the RLX solution, you need to configure it as a cluster, with all it's inherent troubles, to be able to do roughly the same thing, and still not have shared memory etc.
Actually, the spec sheet indicates that it is 8.9kW per rack (2.2kW for Drive arrays). That is on the high side, but liveable. (6kW is the max for "standard" cooling-- you can accommodate up to 10kW with a high delta-T cooling system. Water cooling comes into play after that.)
The value of shrinking it down is (as you allude to) not a real-estate issue, but more about the computing efficiencies of a denser package.
The HP blades (6U) are about 35kW nameplate per rack, with a real load of about 10-11kW. The energy savings of SGI might actually give it some value in comparison!
Check out Nvidia's data centers. Beware... windows media format warning.
Notice how many times the word linux is used...
That's a cluster, not a single-image supercomputer. Read again the coments to this article on Slashdot, there are many explanations why a cluster, no matter how many CPUs you throw at it, will never be able to solve entire classes of problems fast enough; to do that, you need a single-image computer, like the SGI stuff.
I can't believe this got moderated as "insightful." Crap like Indys and O2s is what put SGI in a bad place to begin with. SGI always had fantastic graphics technology and a kick-ass operating system. When they tried to sell low-end workstations-- Indys and O2s running IRIX, and all the stupid stuff with Intel machines running NT and Linux-- their net revenues went into the toilet.
Not quite true. After all, in 1994 an Indy had better price/performance than a comparable Pentium system... and a Pentium couldn't touch a fully loaded Indy. With better marketing, SGI could have dominated the high end 2D and low end 3D space, driving out Apple and Intergraph, and continued to hold high-end 3D. I agree that NT was a colossal mistake for them, and they aren't recovered from that mistake next.
It's when SGI de-focuses to talk about stuff like PCs with fancy cases or video servers or data mining software that they start to lose their way.
SGI servers are fantastic for large databases, the features that make them great for rendering and number crunching (high memory bandwidth, very fast disk I/O, single system image) can easily be applied to databases. The Origins should be wiping the floor with Sun's Fire range. It's a marketing failure, not a technology failure.
This isn't SGI finding a new reason to exist. This is SGI going back to what has always been one of its best reasons to exist. Over time, SGI's technical lead in graphics has diminished, fueled primarily by (believe it or not) home computer games. But even now, nobody can touch SGI for high-performance scalable servers like the 3900.
It has diminished true, but it still exists. There isn't a PC that can touch the Fuel workstation, for example.
SGI's MIPS chips are engineered to generate magnitudes less heat than Intel or AMD chips. Itanium cores throw 130W or so, whereas SGI engineers all of its cores to fall between 15 - 20W. And if there's one thing SGI knows, it's how to engineer a case. I have an Origin 200 sitting at home, and that thing has enormous heat sinks on each of the CPUs along with three industrial sized fans pushing air through.
e;
If that's true, then a 128-processor system would require a minimum of either 32 or 64 GB of RAM, depending on whether you can put 256 MB on a node board.
You can put up to 32GB on a node board/CX brick, a max of 8 CX bricks per rack makes for 256GB per rack, up to 1TB in a 512cpu system (4 racks). Not sure what the minimum is.