IBM's Blue Gene Runs Continuously At 1 Petaflop

An anonymous reader writes "ZDNet is reporting on IBM's claim that the Blue Gene/P will continuously operate at more than 1 petaflop. It is actually capable of 3 quadrillion operations a second, or 3 petaflops. IBM claims that at 1 petaflop, Blue Gene/P is performing more operations than a 1.5-mile-high stack of laptops! 'Like the vast majority of other modern supercomputers, Blue Gene/P is composed of several racks of servers lashed together in clusters for large computing tasks, such as running programs that can graphically simulate worldwide weather patterns. Technologies designed for these computers trickle down into the mainstream while conventional technologies and components are used to cut the costs of building these systems. The chip inside Blue Gene/P consists of four PowerPC 450 cores running at 850MHz each. A 2x2 foot circuit board containing 32 of the Blue Gene/P chips can churn out 435 billion operations a second. Thirty two of these boards can be stuffed into a 6-foot-high rack.'"

Comments

How high?

[Anonymous Coward]

Well the the stack of laptops might be tall, but even the 216 racks would stack up to 1/5 of a mile high.

Depends on what you mean by real world.

[jd]

If you include medical imaging, then computed tomography and computational fluid dynamics are heavily dependent on 3D FFTs, which are in turn heavily parallelizable. In extreme cases (raytracing, for example) where there is next to zero communication between nodes, you get linear scaling with the number of nodes for as many nodes as you like. Well, in the case of raytracing, up to the resolution your "camera" works at. On a modern display, you may be talking one million or so distinct originating points at three colours, typically using "bundles" of rays to eliminate effects, which would normally be 64 rays in size. With something like 250 million cores, you could actually generate an animated feature film from raw data files at the time of showing.

How many of these are "real world"? Well, medical and CFD applications are significant, but hardly what you'd call mainstream, and the raytracing may have been used in Titanic on a smaller scale, but IMAX is under no threat at this time.

Re:Depends on what you mean by real world.

[jd]

Thank you for the compliment. It's equally nice to know that there are active questioners on Slashdot determined to stretch the quality to the limits. In the spirit of providing information, though, I'll add a few links for the perusal and amusement of all. I'm hard on some of the software, but that's not because I could do better. If anything, it's because I have confidence the authors could.

Let's start with a Slashdotting of NASA...

• Scalable Dynamic Chimera Methods for Unsteady Aerodynamics is one of those packages mere mortals like us will have either no use for or will have to just drool over.
• Fully Unstructured Navier-Stokes 3D is a nice Fortran-based CFD, requires some hefty paperwork to obtain, and may need you to use G95 rather than GCC's GFortran, due to compiler bugs.
• OVERFLOW and related CFD software.
• Three Dimensional Multi-block Advanced Grid Generation System is the component that actually lets you do a lot of the necessary grid work for CFDs.
• Viscous Upwind ALgorithm for Complex Flow ANalysis is the hardest of the CFD codes at NASA to obtain, but if you want to work on anything hypersonic, it's the best place to start. Do Not Use hypersonic airflows for CPU cooling.

• Astrophysical Thermonuclear Flash Simulator - well, you never know.
• Geant4, for the subatomic nuclear physicist in your life...
• Open Field Operation and Manipulation is a nice open-source CFD package.
• Parallel Basic Local Alignment Search Tool gives you a parallelized search engine for nucleotides and proteins.
• Stanford Exploration Project provides some nice parallel geophysics applications and tools.
• Tachyon Parallel Raytracer is a nice example of what you can do with parallelism and graphics.

• Kerrighed is an up-and-coming clustering system for Linux. I saw it demonstrated at SC|05 - and was less than impressed. It needed a lot of work at that point. However, it looks like it has improved a lot since then, and it would be unreasonable to not mention it.
• MOSIX is the second-oldest clustering technology to gain a fan following to rival Star Trek. It's very good, though hard to get if you're not in academia. Arguably for entirely fair reasons.
• OpenMOSIX was originally a fork from MOSIX but is now essentially its own clustering technology. Development is nowhere near the speed I'd like, it does need far more eyes, but is well-known and highly regarded. Moshe Bar is also one of the coolest developers I've encountered.

• DAKOTA is a program for profiling parallel applications and should be useful in telling you where you are gaining and losing.
• HPC Toolkit is another toolkit for profiling HPC applications.
• is yet another profiler for parallel software. Between this and the others I've listed, you should have more information than sequential programmers ever get to work with.
• Performance API is a facility used by most of the profiling software to provide an architecture-independent view of performance counters. I have it on good authority that some (now former)

The Dawn of Petaflop Computing!

[i_like_spam]

This announcement is part of the International Supercomputing Conference, which just kicked off today. The new Top500 list will also be announced shortly.

While the new IBM Blue Gene/P system is impressive, I'm more curious to see what sort of new supercomputer Andreas Bechtolsheim of Sun Microsystems has put together.

Here's an interesting quote about Bechtolsheim from the article:

'He's a perfectionist,' said Eric Schmidt, Google's chief executive, who worked with Mr. Bechtolsheim beginning in 1983 at Sun. 'He works 18 hours a day and he's very disciplined. Every computer he has built has been the fastest of its generation.'

Re:The Dawn of Petaflop Computing!

[flaming-opus]

It appears that Sun's design is less revolutionary. It's just a bunch of off-the-shelf blade servers strung together with infinaband. They use the same cabinets, powersupplies, etc as the regular blade server offerings for non-technical computing. It also runs as a regular linux OS, clustered, rather than a supercomputer specific OS, as the Blue Gene does. The big differentiator of the Sun system is the massive 3000 port infinaband switch. I'm sure it's not actually a 3000-port switch, but a bunch of small switches packed together, running over printed circuit boards, rather than cables.

Sun's design is affordable, and probably has a pretty decent max performance, and pretty reasonable power/memory per node. However, it's not as exotic as IBM's design. The IBM design has fantastic flops/watt and flops/square-foot performance. However, each node is really wimpy, which forces you to use a LOT of nodes for any problem, which inreases the necessary amount of communication. Some problems work really well, others, not so much.

IBM has limited blue gene to a small number of customers, all with fairly large systems. I suspect that's because it's very difficult to port an application to the system, and get good performance.

Re:How far behind are desktops from super-computer

[flaming-opus]

A tricky question, but not all that interesting. A fast server processor is within a factor of 4 of the fastest supercomputer processor in the world. That does not mean that you can do equivalent work with the server processor. Among other things, processing performance (gigaflops) of a CPU, is no longer the interesting part of a supercomputer. (It never really was) memory bandwidth, interconnect bandwidth and latency, and I/O performance are the more interesting features of supers. 12 year old Cray processors still have five times the memory bandwidth of modern PC processors, and twenty times the I/O bandwidth.

You'll notice, that 98% of the supercomputers, sold in the last 10 years, all use server processors. (Blue Gene actually uses an embedded systems processor, but it's the same idea) However, in the late 80's putting 256 processors in a super was cutting edge. In the 90's, a few thousand. Soon you'll see a quarter million cores. So supers are actually getting faster at a higher rate than are desktops, at least by most measures.