Slashdot Mirror
SGI Introduces World's Densest Server
Twirlip of the Mists writes "Today SGI announced the Origin 3900 server, the world's densest computer. How dense? How about 16 MIPS R14000A processors and 32 GB of RAM in a 4-rack-unit 'superbrick,' for a grand total of 128 processors and 256 GB of RAM in a single rack. That makes the new machine the densest single-system-image computer in the world; it's even denser than most blade systems. Just for fun, the server also includes a whole bunch of 64-bit, 133 MHz PCI-X slots (from 11 up to hundreds and hundreds, depending on configuration). There's coverage of the announcement on ZDNet, CNET, and InfoWorld, as well as on SGI's own site."
As someone who has worked with blades, my first question is what they do about heat... Sure the CPU's may run a little cooler, but at that density, what keeps it from melting???
An ounce of perception is worth a pound of obscure
These servers are pointless in most datacenters. In order to fill one rack with this much horsepower, you would need at least two empty racks next to it to compensate for the power draw and (much) increased cooling needs. I would argue that the target market for this equipment is government labs, research institutes and universities--not usually starved for floor space.
each processor consumes a reasonable amount of elecricity. why have these never been used in anything other than sgi boxen, and cobalt raqs?... neat processors along with arms of course. too bad the world is stuck in the "my processor is faster than thou" mind set. i had thought some years ago that apple would have been well off buying sgi since they have similar markets at the low end of sgi and at the high end of apple.
Obviously, that should be 64 gigabytes of RAM, not 64 megs.
Interesting thing about this system will be, rather than the maximum RAM capacity, the minimum RAM required. The original Origin 3000 required some minimal amount of RAM-- 256 or 512 MB or something-- for every four processors. I'm not sure if this new model has the same requirement, but I'd imagine that it does. (It's an architectural thing. Every node board has to have some RAM on it, because that node board may be nominated at boot time to act as the boot master, among other reasons.)
If that's true, then a 128-processor system would require a minimum of either 32 or 64 GB of RAM, depending on whether you can put 256 MB on a node board.
I write in my journal
www.clustercompute.com
well, on a per mips basis maybe, but then again I could use faster cpu's today.
MP3 Search Engine
There are 128 cpu intel/amd solutions that fit in a single rack. I know of at least 3 companies that produce them and they are cheap.
There are a few blade systems that can squeeze 128 or more processors into a rack, but those are blade systems, not single-system-image compute servers. You can't use a blade server to do the job of an Origin 3900. (Of course, the converse is also true; you wouldn't buy an Origin 3900 to do something you could do with a blade server instead.)
SGI tends to produce exactly what the customer wants. It's just that their customer is more often than not the federal government, or a very large corporation. It's not well-known-- in fact, for a time it was classified-- but SGI designed, manufactured, and sold an entire line of what were basically DSP coprocessor units specifically for Lockheed's satellite division. Called the "tensor processing unit," each one was basically an expansion module for the Origin 2000. SGI built it just like a commercial product, complete with documentation and everything, and manufactured them in large quantities. It's just that you couldn't buy them unless you were Lockheed.
It's only when SGI tries to branch out that they do poorly. I don't know WTF they were thinking when they decided to try selling inexpensive (relative to other SGI products) workstations running NT or Linux. That was just insane. But as SGI strips more and more of that BS away, they get closer and closer to being a sound company again.
I write in my journal
I'd worry about the bus chipset heating up more than the processors.
It does. The Bedrock chip is both considerably larger and considerably hotter than the R14000A is. (Bedrock is the memory controller, node crossbar, and "bus" arbitrator.)
As to your other comment, SGI got a lot for their money when they bought Cray back in the mid 90's. They took a lot of good Cray technology-- like crossbar-based NUMA system design principles-- and incorporated them into their large server systems. I believe SGI was the first company-- other than Cray itself-- to break the one-hundred CPU barrier on a single system image. (The T3 series was a monster, but I don't recall exactly how many CPUs you could cram into one.)
I think it was Seymour himself who once said, "A supercomputer is a device for turning compute-bound problems into I/O bound problems."
I write in my journal
I see no mention of benchmarks, despite comparing their system to the industry leading IBM p690.
This is the same level of performance that has resulted in SGI's ass being handed to them on a plate for the last several years. YAWN.
Today, SGI poo-poos faster clock rate CPUs because they do not have one. They say their Itanium2 based system will be out in early 2003. At that time, we can expect SGI marketing to change their tune.
Few computing problems scale to large numbers of lower performance CPUs efficiently (queuing, resource contention, scheduling issues, etc). For most problems, a smaller number of faster CPUs will get the job done more quickly.
I was hoping for more.
Close, but no kewpie doll. A superbrick hold 16 processors (not 64; I think that was a typo on your part), and connects externally via NUMAlink to other superbricks. But, if I remember my numbers right, the maximum memory latency across the longest multi-router NUMAlink hop in a 128-processor Origin 3000-series system is less than the normal processor-to-processor latency in the Sun Fire 15K. NUMAlink is incredibly fast. The ratio of local memory latency to remote memory latency is something 1:1.5, as opposed to about 1:10 in IBM's and Sun's big systems.
I write in my journal
Sure, if you buy a ton of second-hand peecees and glue them together in a Beowulf, you have lots and lots of flops (= CPU power).
;-)
But the flops are not everything. The problem with clusters is the network latency when the nodes talk to each other. That latency is small for your average network application, but immense for a supercomputer trying to make all its CPUs talk together. This is why there are entire classes of problems that cannot be solved properly on clusters (non-parallelizable problems).
As opposed to that, an SGI supercomputer has the inter-CPU latency orders of magnitude lower. Same GFlops per total (same CPU power), but certain problems are solved orders of magnitude faster.
That's the power of latency.
They're not out of the woods by any means.
History speaks pretty clearly about what happens to companies that marginalize their business into making 1-offs for infinite-budget DoD contracts and agencies. Eventually, projects get cancelled, line items in budgets get axed, and whole departments are re-orged into something different.
Cray, anyone ? Cray-Research basically went under when the Cray-3 contract was axed. They were counting on that single-machine to keep the afloat. They futzed around with GaAs custom process and never got it qutie working right, and then the cold war ended and with it the justification for subsidizing a maker of 1-off supercomputers.
(Incidentally, the purchase of Cray is what really broke SGI's back. 50% more employees, 2% more market cap, and the O2k/O3k technology came from stanford, not Cray) SGI bought itself into the supercomputing space with the cray acquisition, but their sales reps didn't know what to sell... T3, vector, or Origin. It bled the company pretty badly.
Nobody argues that right now, there are some things for which there simply isn't any other rational choice besides SGI. In the early 90s, that was "anything with video, at all". Look how that market has all but vanished for them.
The problem is the number of markets for which SGI is the only choice is shrinking and will continue to shrink. Only the institutions that need to be 1-3 years ahead of the curve will pay the huge markup for it. The big advantage of the O3k system is, as you ponit out, the single-system image. But this is only really advantageous for lazy programmers, and when you're talking 3m for a machine to do scientific or simulatino work, i suspect a lot of the code running on these is very custom, and NOT done by lazy programmers. So the brilliant thinking SGI has put into the hardware can sometimes be beaten by domain-specific software. Eg, lets say that MOSIX and 10Gig ethernet advances to the point that you can build a 1024p 512 node cluster, where the backbone (10Gb ethernet) is constructed in the same hypercube fabric as the numalink cables, and MOSIX can with software emulate the memory/process/thread migration that O3k is doing now....
then will 2.9m for a machine still seem justified ?... a 512 node wintel cluster is cheaper than 2.9m if the node cost is under about 5500. How many x86 boxes do you know of that cost 5500.. even with 2 procs, a few gb of ram, and 4 or 5 10GB ethernet controllers (so that each node is n-way connected in the same hypercube fabric that O2k and o3k provide)
My opinions are my own, and do not necessarily represent those of my employer.
There are entire classes of problems which cannot be solved fast enough on clusters, but only on single-image systems. Anything that cannot be made into a parallel algorithm falls into that category.
With networked clusters you're always going to have latencies, orders of magnitude higher than with single-image supercomputers.
Sure, perhaps in 10 or 15 years, we're going to have network latencies as small as those of a PCI bus, but i'm not really talking about future that far. Until then, clusters will be slow for certain problems. Deal with it.
It seems that massively parallel computing has gone the way of the Dinosaur what with the advent of more powerful CPUs. But I read that Danny Hillis of MIT and Thinking Machines fame had built a supercomputer called the Connection Machine which housed 65,536 procs each of which lived on the same wafer with dynamic ram and were arranged in a 16-dimensional hypercube array. I don't think the old beastie had nearly as much ram as the new SGI (of course, this machine was 80's vintage). But depending on the physical size of the old box, could this have not been the world's densest computer ever?
Quod scripsi, scripsi.
I see lots of errors and misunderstandings here. Apparently people have a hard time understanding tech. that is not in thier PC.
NUMAflex is the coolest concept in systems architecture today. I'm eager to see some trickle down into lower-end markets.
Read this before you post:
John Mashey's excellent NUMAflex paper.
RLX Technologies has a server based on Transmeta Crusoe chip and it can hold 24 CPUs in 3U space, giving 336 processors per rack (and 336GB of RAM and 27TB of HDD :)
See promo here..
- Raynet --> .