More Details Of IBM's Blue Gene/L

Bob Plankers writes "By now we've all heard about IBM's Blue Gene/L, LLNL's remarkable new supercomputer which is intended to be the fastest supercomputer on Earth when done (360 TeraFLOPS). IBM has released some new photos of the prototype, and renditions of the final cluster. Note that the racks are angled in order to permit hot air to escape vertically and reduce the need for powered cooling. The machine uses custom CPUs with dual PowerPC 440 processing cores, four FPUs (two per core), five network controllers, 4 MB of DRAM, and a memory controller onboard. The prototype has 512 CPUs running at 700 MHz, and when finished the entire machine will have 65536 dual-core CPUs running at 1 GHz or more. Stephen Shankland's ZDnet article also mentions that the system runs Linux, but not on everything: 'Linux actually resides on only a comparatively small number of processors; the bulk of the chips run a stripped-down operating system that lets it carry out the instructions of the Linux nodes.'"

Comments

The Racks Are Not Angled

[The Other White Meat]

If you actually look at the picture, closely, you will see that the racks themselves are NOT angled to reduce active cooling.

At the left side of the row of racks, there is an angled cover, which is either decorative, or being used to force cold air down the row of racks. Likely, its just decorative, and the cold air is being forced up from the raised flooring below.

Just like it is in every other enterprise-grade computer room...

No, *Smaller* and Faster

[RalphBNumbers]

What's big about BlueGene/L is that it's small. That 512 processor prototype they mention in this article is the Dishwasher-sized computer you heard about.

BlueGene/L is about driving down the cost of supercomputing, not only in terms of money spent on hardware, but in terms of space, cooling, and maintanance, while at the same time improving scalability.

BlueGene/L is going to put 65,000+ processors in less space, using less power, and costing less, than many of todays >10,000 processor systems.
They do this with a minimalist approach, each processor is a SoaC (System on a Chip), with everything from the memory controller to internode networking to two cores and 4FPUs on the die, and the only other thing in a node besides the processors is a bit of RAM. This allows them to use much less power per node and gives them less heat per node to dissipate, which lets them pack the nodes much closer, which cuts down on internode latency, which increases scalability.

Re:Only a PPC 440?

[be-fan]

-1: Disinformative

RISC vs CISC means very little these days. Most current CPUs have a core even more minimal than RISC chips, but present a CISC (in the case of x86) or RISC (in the case of the G5) interface to the outside. They used the PPC 440 for different reasons:

1) IBM had to do significant custom engineering for it, and they own the PPC 440 core. That allowed them to use it to design an SoC.

2) They needed to add FPU hardware, which is easier to do on a design they own. The PIII only has one FPU, while this chip as 2 FPUs. IBM had to add this to the design, because the regular PPC-440 has no FPUs.

3) The PPC-440 was designed from the beginning to be an embedded CPU. At 1GHz, a stock PPC-440 consumes about 2.5W. Even a low voltage PIII consumes more than that.