More Details Of IBM's Blue Gene/L

Bob Plankers writes "By now we've all heard about IBM's Blue Gene/L, LLNL's remarkable new supercomputer which is intended to be the fastest supercomputer on Earth when done (360 TeraFLOPS). IBM has released some new photos of the prototype, and renditions of the final cluster. Note that the racks are angled in order to permit hot air to escape vertically and reduce the need for powered cooling. The machine uses custom CPUs with dual PowerPC 440 processing cores, four FPUs (two per core), five network controllers, 4 MB of DRAM, and a memory controller onboard. The prototype has 512 CPUs running at 700 MHz, and when finished the entire machine will have 65536 dual-core CPUs running at 1 GHz or more. Stephen Shankland's ZDnet article also mentions that the system runs Linux, but not on everything: 'Linux actually resides on only a comparatively small number of processors; the bulk of the chips run a stripped-down operating system that lets it carry out the instructions of the Linux nodes.'"

Comments

Doom3?

[arcanumas]

Well, it may be able to play Doom3 when it is released.

Infinite

[Raynach]

I'm really impressed with this computer. I think it's going to be the first computer that can finish an infinite loop in under an hour.

Re:Infinite

[Bronster]

If an infinite loop is infinite, how can it be finished?

That would be the sound of the joke wooshing around, and around, and around, and ... your head.

Travelling salesmen.

[Manywele]

This will be sure to boost the effeciency of travelling salesmen everywhere.

Re:Travelling salesmen.

[Vegard]

Actually, we better let the salesmen travel. It's a little known secret that the reason computers are so bad at solving the travelling salesmen problem is that those who design computers are technicians, and everyone knows that tech people hates salesmen, so the longer they spend travelling, the better for the techs.

Subjective...

[Decameron81]

"The prototype has 512 CPUs running at 700 MHz, and when finished the entire machine will have 65536 dual-core CPUs running at 1 GHz or more."

Woah, this is the first time I think a box with 512 CPUs at 700 Mhz each one is crap.

Diego Rey

It's gorgeous...

[mOoZik]

I think I wet my pants.

Re:4 MB DRAM

[otis wildflower]

4MB per CPU, each with 2 processing cores, and an onboard memory controller.

Final version to have 65536 CPUs.

Smells like 256GB to me, which is pretty decent in _any_ book, especially if it lives on the same silicon as the CPU...

Re:Only a PPC 440?

[shaitand]

No, It's a supercomputer. Those are RISC processors, a PPC 440 in reality gives better performance than a CISC processor like the PIII

What's new?

[LeoDV]

So, you mean they're going to build a computer that's going to be bigger, faster and with higher number stats than the current #1? Shocking!

Sorry about the sarcasm, I'm only asking to be proven wrong, but isn't Blue Gene just more of the same, only bigger? Big Mac was interesting because of how cheap it was and because it was the first of its kind to use Macs, the Earth Simulator was interesting because it brought back custom chips for supercomputing as opposed to off the shelf components, we've been reading about IBM's dishwasher-sized supercomputer, articles about efficient supercomputing, so what's new about Blue Gene, besides being newer and bigger?

Once again I'm not bashing, I haven't read much of anything but the /. blurbs, so I'm asking, is it just a bigger supercomputer, or does it have any "real" innovations?

Re:What's new?

[javiercero]

Well... Earth Simulator was bigger and faster... as was big mac. Every advance in computer design and fabrication has been about "bigger and faster." It may sound trivial to you, but that is because you got no idea about what is involved about making things "bigger and faster."

What is significant about blue gene is that is some sort of compromise between off the shelf parts (PPC based Processing elements vs. the Earth Simulator SX based custom vector PEs), and efficient interconnection (plain crappy cluster like the Big mac with a better interconnect at multiple layers starting with dual cores per die).

In the end it all leads to the same goal: tackling bigger problems faster. So it may sound trivial but there is a lot of research going into this baby.

Re:What's new?

[tigertiger]

Beowulf-style cluster are a big waste in terms of additional circuitry and hardware that you do not need in a supercomputer, from I/O busses to the power supply. Of course, since off-the-shelf components are so cheap, it is still cheaper to buy the stuff than to design your own tailor-made circuitry - up to a certain scale.

That is where IBM tries to go: BlueGene's design is based on a system-on-a-chip - everything (except memory) is integrated on a single chip. In the long run, this allows them to build systems much larger than you could with a Beowulf. They are basically aiming for a system where you can easily add computing power by simply putting in a few more chips, and the thing will scale. They are doing the same thing for storage with this brick

BlueGene is a also the first supercomputer marketed to the life sciences. It's interesting to see that it developed from a project at Columbia University called QCDOC for "Quantum Chromodynamics on-a-chip" which did research in computational high-energy physics, and QCDSP before, which used DSP processors to build a supercomputer about ten years ago. Both an instructive example how academic research in the long run becomes industrially relevant, and how science changes.

Re:4 MB DRAM

[javiercero]

Because they are used embedded DRAM, which although not as fast as SRAM it gives more storage in fewer transistors. This leads to a smaller die, and lower power/heat dissipation.

If your p1 runs at the same speed than your P4 for 90% of operations, then there is something wrong with your computer! The HDD is not the bottleneck for most modern computers, as they have enough memory to minimize page faults for most common home computing tasks.... Startup times however may be equal since both machines have to get the data/program from HDD... once stuffs are in memory, buubbie P1....

I'm surprised!

[Epsillon]

The standard of trolling has certainly fallen recently. Where's the SCO licence fee estimate for the finished 65536 processor SMP unit? You got a better class of idiot in those days... ;o)

Re:I'm surprised!

[Ianoo]

IBM better not forget to pay SCO their $45,809,664!!

Re:it's all cool and everything...

[gazbo]

Yes. The thought of a creature with two X chromosomes is horrifying.

See for yourself

[mangu]

save this as "test.c":

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <sys/time.h>
#include <unistd.h>

#define N 1000

struct timeval tv;
struct timezone tz;
long bu, bs, du, ds;

int main()
{
int i;
double t;

printf("Infinite loop test\n");

gettimeofday(&tv, &tz);
bs = tv.tv_sec;
bu = tv.tv_usec;

for (i = 0; i > -1; i++)
;

gettimeofday(&tv, &tz);
du = tv.tv_usec - bu;
ds = tv.tv_sec - bs;
t = (double)ds + (double)du / 1e6;
printf("executed in %13.6f seconds\n", t);
}

compile and link with:

gcc -g -o test test.c

run:

Infinite loop test
executed in 3.888419 seconds

Re:Only a PPC 440?

[be-fan]

-1: Disinformative

RISC vs CISC means very little these days. Most current CPUs have a core even more minimal than RISC chips, but present a CISC (in the case of x86) or RISC (in the case of the G5) interface to the outside. They used the PPC 440 for different reasons:

1) IBM had to do significant custom engineering for it, and they own the PPC 440 core. That allowed them to use it to design an SoC.

2) They needed to add FPU hardware, which is easier to do on a design they own. The PIII only has one FPU, while this chip as 2 FPUs. IBM had to add this to the design, because the regular PPC-440 has no FPUs.

3) The PPC-440 was designed from the beginning to be an embedded CPU. At 1GHz, a stock PPC-440 consumes about 2.5W. Even a low voltage PIII consumes more than that.