Slashdot Mirror
Specs On New SGI Onyx And Origin
An anonymous reader wrote in to tell us that
SGI has announced their latest and greatest MIPS-based computers, the Onyx and Origin 3000 line. Up to 1 TB RAM and 512 processors, all on a single system (not a cluster).
Beyond Boxes has a nice summary, too. This is definitely a great system for anyone who wants to
have their computer be the size of several refrigerators ;)
So the rest of the industry is playing "catchup" to SGI ?! I don't really think there's a huge market for large-scale multiprocessor machines when equivalents can be built up easily from cheap hardware and fast network infrastructure.
Actually, they can't be.
This is not a cluster - it's a multiprocessing supercomputer designed as a single unit. The internal busses have far, far greater bandwidth than even the expensive networks in a high-end cluster.
It does have competition - the Sun Starfire. But that's about it.
Clusters are definitely useful, and give you by far the best bang-for-the-buck on problems with relatively light communications load, but problems with a heavy communications load are best run on machines with high communications bandwidth, like this one.
1. The CDROM is on an internal FireWire bus.
2. The system disk is Fibre Channel.
3. SGI hasn't made a big deal about it yet, but the system will accept either MIPS or Intel processors in the same CPU modules. The MIPS processors come on one kind of daughtercard, and the Itaniums (Itania?) on another. You can't mix-and-match MIPS and IA-64 CPUs in the same machine, but you can mix-and-match in the same cluster.
4. The IA-64 based versions of the 3000 series will include the Linux kernel along an some IRIX compatibility layer.
Amusing bits from the page:
Debra Goldfarb, group vice president at analyst firm IDC, agrees: "Modular computing empowers end users to build the kind of environment that they need not only today but over time. SGI, with this product, is really ahead of the curve in the market. We are seeing the [rest of the] industry absolutely trying to catch up" with SGI.
So the rest of the industry is playing "catchup" to SGI ?! I don't really think there's a huge market for large-scale multiprocessor machines when equivalents can be built up easily from cheap hardware and fast network infrastructure. The last time I saw an SGI was the NASA AMES crew using one for their amazing Viz tool, and even they were making mutterings about porting it to NT and Linux for ease of maintenance and actual use.
In addition, SGI Origin 3000 servers and SGI Onyx 3000 visualization systems reflect a return to SGI's core competencies.
At least that's true. The NT machines were a joke. Anyone tried SGI Linux yet?
The whole system has one contiguous view of memory. The NU means "non-uniform" as in the memory access time is non-linear. If a process's memory is located on the processor module it's running on, the memory access is fastest. If it has to jump one module away, the memory access time increases by 100 nanoseconds (roundtrip). Architecturally, it's completely different than anything Sun has to offer. Sun has been promising a NUMA mahcine for years and still hasn't delivered. The closest company to SGI is Compaq(DEC), and there top of the line offering can almost compete with Origin _2000_. All other companies high-end servers use symmetric multiprocessing, which becomes limited as more and more processors try to access the shared memory bus, ultimately bringing in negative returns as you add more processors. This NUMA architecture incurs very little (if any ) penalty by ading more processors, as long as the hardware and OS do a good job of placing processes and memory (keeping them physically near). Also, the machine is _not_ limited by 512 processors. To give an example of the power of this box, a company has certain calculations that they run day to day. On their top-of-the-line Sun hardware, it takes about seven hours. On O3k, it takes seven seconds! What does being a modular system have to do with being a cluster? By being "modular" it simply means that you can plug in more of whatever you want, whenever you want. I believe you can even mix faster cpu modules with existing ones as they become available, protecting your investment. This is not a cluster.
Intel transfer the difficult from Hadware to software, for get more power, programmer need more technology. -- chinaitn
Don't get me wrong, I love SGI's machines and use one daily. Even passed up on a faster PC (running Windows) because I like it so much. But there is no way I could cost justify getting a new one. They simply do not provide enough performance to justify the cost anymore. All the demos of their stuff we've seen doesn't indicate that their new machines are a huge leap in performance. (meaningfully faster to be sure but not nearly enough to justify the cost of a new one) Fortunately for SGI they make a ton of money on each Onyx & Origin they sell but if they aren't careful this could easily evaporate out from under them. They make very cool systems but it is not a well run business IMO. I'll be somewhat suprised if SGI doesn't get bought out by someone in the next year or two.
"This is definitely a great system for anyone who wants to have their computer be the size of several refrigerators."
I foresee a day when computers may be as small as one refridgerator. Probably there will be a world market for no more than 10 of these.
--
Give us our karma back! Punish Karma Whores through meta-mod!
Linux MAPI Server!
http://www.openone.com/software/MailOne/
(Exchange Migration HOWTO coming soon)
System Bandwidth
3200: 11.2 Gigabytes/sec
3400: 44.8 GB/sec
3800: 716 GB/sec
...methinks they skipped a decimal point here.
(if not, please explain!)
---
pb Reply or e-mail; don't vaguely moderate.
pb Reply or e-mail; don't vaguely moderate.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
To fix a few misconceptions: 1) The bricks are (mostly) 3U [5.25"], or 4U [7"] high, and the same bricks are used to construct a wild range of systems, with huge variations in CPU-I/O-storage ratios. 2) In some cases, the bricks will be sold separately and embedded into airplanes, vans, etc, by defense contractors. I'm told the submarine folks really love the idea. 3) In a half-rack (SGI Origin 3200), you can have 2-8 CPUs [1-2 C-bricks], a required I/O brick [I-brick], and either another I/O brick (I, P, or X) or a disk brick (D-brick). 4) People always announce a wide range of systems: realistically, most of these machines will be 1-2 rack systems, just like they are for everybody else. People who buy lots of computers use racks anyway - the last thing in the world they want to do is waste precious floorspace. 5) IRIX already scales to 512P fairly well, and NASA AMES runs individual shared-memory jobs on their older Origin2000. It already saved you a lot of tax money. 6) SGI is not shipping Linux on the MIPS-based machines. This is a "Caterpillar" announcement, with a lot of shoes left to drop, like IA-64-Linux versions coming later. A major point of the brick thing is that you can change bricks while re-using most of what you already had; you can for example, introduce a PCI-X, or later, Infiniband brick without obsoleting older I/O bricks. Also, you can build C-bricks with Intel IA-64s, and those will run Linux, not IRIX. All of the rest of the hardware infrastructure & bricks are the same. 7) SGI is working hard with the Linux community on scalability, i.e., to let it handle more CPUs well without damaging the basic Linux. Personally, I doubt that it will make sense to try to scale Linux to where Irix is, but it will certainly scale big enough to be interesting [say 32-64P in single system image]. Using partitioned hardware, one can get NUMAlink speeds between partitions, and that satisfies many customers. 8)The customer should be able to pick the size of machine, and then cluster that size together. For some customers, 1P + 64MB is just dandy, and they buy clusters of IA-32 boxes. I know customers where the right size happens to be 32P, 16GB of memory, 2 disks, and 3 Ethernets [one full rack], and then they cluster a lot of those. I know customers that cluster 128Ps, and there's one who would cluster 512Ps if they had the money. If the NASA Ames folks had the money, what they really want is a single machine with Petabytes of memory and Petaflops. I was sorry to tell them, Not Likely Soon. 9) Don't get too crazy with the fact these systems can go really big. I've lost track, but I think there are 30,000 of the Origin2000s & 200s out there, and most systems are small to medium. Of course, the big systems account for many CPUs. 10) The NUMAflex brick approach has many subtle benefits, but is hard work. In some thread, people mentioned backplanes ... but there aren't backplanes in the normal sense. Each C-brick has 4 MIPS CPUs, memory, and an ASIC Crossbar, with 2 ports out the back for cables that run (peak) rates of 3.2 GB/sec (2 * 1.6GB) and 2.4 GB/sec for I/O to separate I/O bricks. Each brick has internal circuit boards, but there is nothing that looks like a normal CPU backplane. To do this, you have to be able to run 3meter/5meter cables at these rates, and do tricky circuit engineering. Later versions will independently improve the interconnects as well, not just upgrade the bricks.
With boxen this size, the boundary between a single machine and a cluster tends to get a little blurred anyway. Even SGI are stressing the fact that it's a modular system. Basically, each module has it's own CPUs and memory, and has connectivity to the other modules in the system. What's the difference between that and a conventional cluster? Mostly the phenomenal inter-module bandwidth, but that's just a matter of numbers. Architecturally, is there much difference? OK, so you have a single OS image running across all CPUs, but is that even true any more? Certainly other large systems (e.g., from Sun or Data General) let you run multiple versions of the OS concurrently on a single box as you see fit.
"The invisible and the non-existent look very much alike." -- Delos B. McKown
Something this size does well for meterological simulations, atomic weapons research, something that entails MASSIVE numbers of computations. I wouldn't be surprised if you see these in placed like NCAR (National Center for Atmospheric Research) NCSA, the Government labs like Lost Alamos and Larry Livermore. I still remember seeing NCSA's purple monster 1024 node cluster of Origin 2000's (using an experimental node bridge)
;)
As for how this is different than a Beowulf cluster, look at the bandwidth! Even with switched 10/100 Ethernet as your Beowulf 'backplane' most switches have just enough backplane bandwidth to handle every 100 Mb connection, some have a little less. sgi has always had amazing bandwidth numbers, this is just taken to the N'th degree.
AND this is one machine, one OS, unlike a cluster of many independant machines, much easier to administer.
These are simply awesome machines, now maybe sgi can sell a bah-zillion of them and I can get my Indy sold
g:wq
What if it is just turtles all the way down?
People are asking things like "why would I use this" and "who wants these?" Let me tell you, in the era of bloatware like Oracle and any of the content management systems out there (possible exception of Zope), the incredible scalability of these systems will be a huge selling point. Oracle, for example, is very careful to build and market their software to be monolithic so that you have to buy big hardware to run it, and then they charge you based on the size of the hardware you're running. Thus, they drive the purchase of huge systems like this, and then charge you up the ass for their "Enterprise class" database.
Believe it or not, this is actually the kind of business model that the Fortune 500 are not only happy with, but demand.
Personally, I'd be happy with a database that could run on a loose, fault-tollerant network of a dozen or so small (e.g. 2-processor Intel or Alpha) systems.
Then again, I'd really like to play with some of SGI's big iron....
Geeky girls are often impressed by the size and power of your computer equipment. However, size is not the most important thing, it's how you operate it.
Mas vale cholo, que mal acompañado.
As someone earlier said, most users probably won't go for the full 512 processors.
I see the 8-processor boxes being a hot seller in a lot of research labs, or where people just want a centralized server.
These machines are very similar to an SMP machine from a programmer's perspective. (From a hardware perspective, they're vastly different, each CPU has its own local memory, although the entire system memory is treated as one big block. It just happens that local memory is much faster to access.)
We have an older 8-processor SGI machine at work that people use to do scientific simulations. Rarely are the simulations themselves paralellized, but instead, people log in and the system gives em' a processor all to themselves if one is free. I think my boss is looking to replace it eventually... Any time someone gets a new system, he wants people to run some benchmarks he wrote. My 500 MHz Coppermine gets twice the performance of a processor on the old machine for small problems, but as soon as the dataset gets larger than the CPU cache, the SGI's excellent memory system kicks in.
retrorocket.o not found, launch anyway?
As great as it is to see SGI's moves to utilize Linux, computers like these demonstrate that Irix still has a place in the larger picture. Irix is really a pretty neat operating system, and frankly, it can scale in ways that Linux just isn't ready to yet. As long as SGI is still making systems like these on the high end, I don't see Irix being displaced anytime soon.
Of course, Irix also has a lot of graphics production tools that you don't find on any OS, Linux included. That's something else that'll keep Irix around, at least until equivalents exist. Ideally, we'd see SGI continue to take steps toward open source/Free software, with Irix components.
Anyway, looks like a pretty cool new system from the people who brought us the original colored computer. Can't wait to get my hands on one of these.
yours,
john