New SGI Altix 3000

dlloyd writes "SGI has just publicly announced the Altix 3000 series of computers that can scale from 4 to hundreds of processors, with up to 64 processors per single system image. Processors each come in a C brick that has 4 CPUs. I/O is done though IX and PX bricks (12 PCI slots per brick, IX bricks have a base I/O controler and two ultra 160 disks inside), just like on the Origin 3900 series. Anything more than 8 CPUs (2 C bricks) is connected by R bricks, which route the NumaLink packets between nodes. The NumaLink network is good for an aggregate 6.4 gigabytes/sec to *each* node. That scales as you add more C and R bricks. Basically, you can think of this as SGI's origin 3000 series, except that it runs Linux and has Itanium2 processors. The performance and scalability is like nothing that has ever run Linux and is *far* ahead of the competition. For those of you who wonder why anyone would need a 64 processor Linux machine, many scientific and technical customers prefer running their code on large, single system image machines. Large single system image machines are also less labor intensive to maintain and admin, plus they work much better on code that needs to share memory and pass messages between threads (even myrinet and mpi is glacial compared to the SGI numalink network and running code multithreaded)."

Comments

SGI is still in business?

[yppiz]

What is keeping SGI afloat? Service contracts on existing machines?

--Pat / zippy@cs.brandeis.edu

Re:SGI is still in business?

[yppiz]

Thanks for the suggestion. So I did take a look, and the answer to the question "how are they staying afloat?" is "they aren't. They are steadily hemmorhaging money."

I posted my question because, as early as 1996 or 1997, it was clear that commodity machines were going to kill SGI. I'm just amazed that they're still alive.

Here's their income statement, balance sheet, and cash flow statement. As of today, For three out of the previous four quarters, they had sales growth (sic) of -20% or worse. The three analysts who cover this stock have a hold rating, which in analyst-speak means sell.

--Pat / zippy@cs.brandeis.edu

Re:SGI is still in business?

[kelzer]

Hmmmm, you seem a little touchy about the subject, and you posted as an AC. One might be inclined to think you had an interest in SGI.

Perhaps you've purchased a LOT of SGI stock in the last year, MR. ROBERT R. BISHOP! Ha! Thought you could sneak in an anonymous post, but you underestimated the analytical and investigative abilities of us Slashdotters.

Nice try.

So, got any old O2's laying around gathering dust, that you don't need anymore?

Why Linux?

[HeelToe]

I still don't understand why SGI has foregone such a great OS as IRIX. Why go with Linux? Just trendy, or does it really offer advantages for scientific computing?

Re:Why Linux?

[larien]

Yup, IRIX was good, but maintaining a full OS takes a lot of money. This way, they can piggy-back on investments made by other people & companies while still having a modern OS. They've already integrated XFS into linux, and it wouldn't surprise me to see other SGI/IRIX technologies coming into linux in the same way. Similarly, IBM have migrated JFS into linux.

The older the boy...

[RebelTycoon]

IX, C and R bricks

The more expensive his LEGO gets...

Re:The older the boy...

[fgodfrey]

...and you thought you were joking. "Lego Hardware" was an internal code name for this machine, and the Origin systems at one point. Rumor was that they wanted to use it publicly but LEGO(tm) objected. Not sure if the rumor had any truth to it, though, since the name was dropped before I started working for SGI/Cray.

SGI is finally making some new products

[ozzee]

Scientific computing has allways been SGI's niche. They unfortunately stumbled around the time that Belluzzo took the helm and wasted the entire internet bubble recovering from the mess that caused.

It's great to see that they're finally back and doing some really serious new stuff.

It's a shame though that they won't be running the AMD 64 bit chips, although, I'll be someone is looking into that.

Congrats SGI !

Don't forget

[PD]

These machines support 512 GB of RAM in one chunk. A Linux cluster might outperform this thing, but you'll need to chunk your data up to fit into the individual nodes' memory. Sometimes this can be a pain in the neck to do, hence the market for something like this.

Re:Isn't the issue in this area $/MIPS?

[XaXXon]

Yeah, you're wrong. This isn't a beowulf, it's a multiprocessor box. It runs standard software. That means you don't have to re-write everything to support clustered solutions.

Lots of people don't understand that a 1024-processor beowulf won't run battlefield 1942 (if you've ever played it, you understand what I'm talking about), because it's not like a 2-processor workstation box. You have to write your software so that discrete pieces can be offloaded to other nodes and have the results posted back. A beowulf cluster is similar to SETI@Home or whatever distributed computing project you like. Though the interconnects are faster, the general idea behind how the software works is similar.

With this SGI system, it's like a 2-processor workstation on steroids. You can run standard multi-threaded code on it and actually use 1024 processors (and could possibly run battlefield 1942).

Looks nice too...

[Midnight Thunder]

One thing that I have always liked about SGI systems, is that not only do I get a high performance system, but I also get something that looks good design wise. Other companies, such as IBM give me the feeling that I am buying, in equivalent terms an F1 car with the body of a Lada. If I pay top of the line prices, I also like to have something nice to show off.

Altix?

[Gudlyf]

With all those bricks invovled, maybe they should call it the SGI Tetrix.

Huh? it uses Itanium 2

[green pizza]

SGI's Altix machines use Itanium 2 CPUs (up to 150 watts per CPU). They have **VERY** advanced cooling subsystems. This is not the MIPS/IRIX Origin series, this is the Itanium2/Linux series.

Re:Isn't the issue in this area $/MIPS?

[Zathrus]

No, because $/MIPS is a misrepresentation. Heck, MIPS alone is meaningless, because all it does is take a theoretical maximum of CPU speed. MIPS doesn't take into account anything beyond CPU speed - like memory speed, backplanes, drive arrays, etc.

If you have heavily interrelated datasets, like in just about any thermal dynamics/plasma/weather problem, then there is so much interdependancy between adjacent "cells" that each work unit needs information from adjacent work units constantly. Spread that system out on a cluster solution and you're DOA because your communications between boxes are horrendously slow, with latencies measured in milliseconds instead of nanoseconds. So while you may have some absurd number of MIPS, the reality is that the CPUs are sitting idle 90% of the time waiting for data from some other CPU/memory block.

Take all those CPUs, all that memory, put them in a single box and do the backplanes and memory interfaces right (this is where the cost comes in by the way) and your latency becomes reasonable and you actually get all those MIPS.

It boils down to what the problem set is. If you need an obscene amount of transactions or have a highly interdependant problem set then you're better off with a single large box. If you can break up the problem set and minimize interactions then clustering is your friend.

There's also the issue of maintainance, and while the hardware costs may be lower for a large cluster, the time spent fixing the hundreds of boxes may kill you. Have a single box that's designed for redundancy and you'll pay a fortune for the support contract, but you won't spend an appreciable amount of your time on hardware support on the rare occasions it actually needs something.

MPI?

[Alex Belits]

even myrinet and mpi is glacial compared to the SGI numalink network and running code multithreaded

Don't mix shitty parallel computation libraries and actual performance. Multithreaded applications without MPI are, of course, faster than anything with MPI, however it says absolutely nothing about:

1. Multithreades vs. multiple processes.
2. Myrinet
3. Network programming
4. Clustering
5. NUMA implementations that reduce everything to SMP with a cache that gets flushed a lot

Re:Itaniums?

[jbischof]

Itaniums are marketed toward high-end servers only. Generally not available to, and too expensive for, the general public (if that is what you mean by regular consumer).

In Intel's mind, the Itanium doesn't compete with the Opteron. Opteron will be at Xeon's throat, trying to tear up some of the 95% market share that xeon has in corporate and other mid-range servers.

You guys are missing the main point!

[LinuxParanoid]

You guys are all missing the main point!

SGI is the first billion-dollar systems vendor to move their totally high-end million dollar hardware to run Linux, and not just to run Linux poorly, but instead their mega-boxes *require* Linux to performe excellently (unlike, say IBM "Linux/390" mainframes where Linux is not really the native OS supporting all the hardware features and is mostly a curiousity or very expensive Apache server.)

The other vendors, Sun, HP, DEC, IBM have not been nearly as aggressive and are depending on their own UNIXes to remain on their high-end boxes.

SGI is depending on Linux and has tweaked it enough to run huge, 64-way complex NUMA systems. This is a major infrastructure bet on Linux, and (assuming this is a shipping, working product) a huge mark of progress for Linux that it can, today, support this sort of high-end scalable hardware.

We all knew it *could*, in theory, but SGI has invested in making sure that *it does*!

This marks a major shift of SGI to an Intel/Linux pure play. It's not just a bunch of low-end Linux server boxes (which they've done before, and Sun/HP/IBM also do), or boxes that you can run either Linux or some proprietary UNIX. It's a full-scale massive 64-way NUMA SMP server that is optimized to run Linux.

Hats off to SGI, I say.

(I wish they had better business prospects but its hard to do that with a niche sort of product like high-end SMP/NUMA technical computing. We'll see if they can push it into a broader customer base with sufficient application support.)

I wonder how Oracle would do on this sort of puppy?

--LP

Re:Correction

[fgodfrey]

I think you're confusing NUMA with message passing. NUMA stands for Non-Uniform Memory Access (actually, this machine is cc-NUMA - Cache Coherent NUMA, but I digress). NUMA means that when I do a standard memory reference it will go faster or slower depending on where in memory that reference goes. This is accomplished by having a group of processors and a group of RAM DIMMs tied to each other with a memory controller that is also a router. If you want someone else's memory, you go over the router to the other memory controller and it returns the answer. That takes longer than your local memory (longer vs. shorter is not uniform), hence the machine is NUMA. IIRC, this machine is NUMA after 2 processors up to the max system size. Despite running multiple Linux kernels, all the memory is visible to all the processors even outside your own kernel. It seems they've picked 64p as the maximum useful size for a single kernel.

What is your quad P4? That's SMP - symetric multiprocessing. Symetric means that all memory accesses take the "same" amount of time since there is only one pool of memory for all the processor and no processor is closer to it than any other. SMP systems larger than around 32 processors are rare since your single memory subsystem needs to feed *all* theprocessors.

So what is a Beowulf cluster then? A typical Beowulf cluster (well, just a cluster in general) is a group of nodes which can't directly address each other's memory and hence have to send a message to the other guy to read/write his memory. Cards like Myrinet exist to try to get some form of shared memory between the nodes in the cluster to varying success. Compared with this, they are low bandwidth and high latency. (Of course compared with a Cray X1, this machine is low bandwidth, but I'm biased :)

There have been a variety of NUMA machines released over the years. Highlights other than this thing include the Thinking Machines boxes, the Cray T3D and T3E, the SGI Origin series, and the Cray X1.