IBM's Blue Gene powered by Linux

bigjnsa500 writes "Linux will be the main operating system for IBM's upcoming family of 'Blue Gene' supercomputers--a major endorsement for the operating system and the open-source computing model it represents. Blue Gene/L, the first member of the family, will contain 65,000 processors and 16 trillion bytes of memory. Due in 2004 or 2005, the system will be able to perform 200 trillion calculations per second. Lawrence Livermore National Laboratory will use the system for performing nuclear weapons simulations." Blue Gene has been announced for some time, but it's cool to see how it's shaping up.

Comments

SCO Linux

[petecarlson]

Hmm, I wonder why they chose to use SCO's OS. You would think with all the lawsuits they would try to stay away from SCO's software...

ASCI Red Storm runs SuSE Linux

[Boone^]

...and it's got 10,368 2 GHz Opterons. (link)

ASCI Red Storm google search

in other news

[t0ny]

In other news, Levi's has announced a lawsuit against IBM, citing the name of the server line could confuse their customers.

How many apples is that?

[goombah99]

Wow 650,000 processors. what's that in equivalent g5 flops, say 4000 or so :-P

kidding aside, are these based on the novel IBM design for having small clusters of wimpy processors sharing sections of memory. The concept being to have each processor running slowly, almost stalled waiting on a memory fetch. (while seeming stupid at first glance, its really diabolically clever since now you can junk all the long pipelines and branch prediction stuff: every single byte that comes from memory will be used by some CPU requesting it, thus you minimize the memmory buss buttle neck that is, ultimately, the limit on most processing).

if this is that design then that 65,000 processors indeed may not be quite as much computing horespower as it sounds. it might indeed be comparable to a smaller handful of G5s.

or maybe i'm full of crap.

Re:How many apples is that?

[ocelotbob]

Uh, IBM makes the G5, or rather, the PPC970. I think they of all people would know whether or not the processor is suitable for the task at hand. Don't you agree?

Re:How many apples is that?

[tychay]

You say:

The g5 supercomputer mentioned on slashdot before never performed what it originally claimed.

What was the claim? The only bogus claims I heard regarding the Terascale (G5 cluster) were:

1. Ignorant people taking Rpeak and multiplying by the number of machines and,
2. Wired taking an accurate claim of the clusters performance on 128 CPUs and extrapolating it to 2200. In the article, the manager of the top500 noted that the G5 cluster might take #3 and contend for #2.
3. A whole bunch of FUD from people like you who have some reason to wish the people working on this project ill simply because they chose Macs to do it.

Then a New York Times report using old data reported 7.1 teraflops Rmax--enough to put it at #3 on the old list and #4 on the new--NYT forgot to mention that there have since been three new clusters that made the top 10, one of which slightly edged out the Terascale.

Of course, by the time that was reported, the figure was revised to 8.3 Tflops and now, officially reported (both on the current Top500 and by the head of Terascale) as 9.555 Tflops (60% efficiency) with the stipulation they could probably get 10% more. A pretty umapproachable #3 spot in the Fall500 and the first sub-$100 million dollar system to break the 10 teraflop mark.

Go look at the current benchmarks, where are the Pentium clusters that are above it? Where are the Itanium clusters above it? Where are the Athlon clusters above it? Oh, I'm sure there will be some (probably in the Spring2004 500), but where are they all right now? How much do the current ones on the list cost (answer: no less than $30 million). Sounds to me the wishful-thinking, poor-reporting Wired and the Mac zealots were closer to the truth than FUD-meisters and the anti-Mac zealots.

The most efficient top 10 supercomputer right now is also the most powerful: The NEC EarthSimulator at about 80%. I'd imagine we should expect a 60-80% efficiency from the big budget Blue Gene/L. And in my book there is nothing wrong with the current 60% efficency of Terascale--anyway it probably says a lot more about how good Infiniband is than it does about how good the Mac is.

But the writing is on the wall. There is nothing special about the the 970 (G5), Virginia Tech could have done the same thing with an Opteron or Itanium2--it would have taken more processors and cost twice as much: ~$10 million best offer for the systems as opposed to $5.7 million list price paid for the Macs (subtracting $1.5 million for the Infiniband cards, routers, and cabling).

The take home point is not that they did it with Macs or Mac OS X instead of (your favorite CPU) and Linux. The take home point is: these guys built a top 10 supercomputer in a fraction of the time (months as opposed to years) at a fraction of the cost (<$10 million as opposed to >$100 million).

Yes, like the Crays of the old days (and today) there will always be those who need something like Blue Gene/L and IBM is happy to supply them. But a whole new generation of supercomputers will be built on-demand and out of commodity PC hardware and a good set of software running on an OS that doesn't charge for all the CALs. Right now the 970 is easily the best performer for LinPak. So much so, they can pay educational list price which included such worthless features as an Apple-tooled case, overpriced RAM, gigabit cards, and Radeon graphics cards, firewire, usb2.0, digital audio, iTunes and other iApps, and a OpenGL based desktop. Since the 970 is made by IBM, I'd hazard a guess that IBM would be happy to supply these people too. Whether they choose to run Linux, MacOS X, or something else.

Nuclear Weapons

[jonhuang]

Somewhere, there's an open source developer who's just realized that his work is being used to the development of nuclear weapons. All jokes about derivative works aside, I think it's a good time to consider the implications of this.

Re:Nuclear Weapons

[cgranade]

As opposed to if he charged for it, in which he would have just made blood money, and implicitly accepted the use of the technology. In this case, he/she can't say that s/he endorses the use, because they released the technology from their hands. To be sure, it is still disturbing, but not in the same way...

Nuke simulations?

[gumpish]

Why are computers still being used for simulating nuclear weapons tests?

Are they trying to pack more megatons of destructive force into each warhead? Don't the major world governments have enough quantity to preclude the need for more powerful units?

Or are the tests run to design "safer" and/or more localized implementations? (Awww, looks like Big Brother has a soft spot after all...)

Re:Nuke simulations?

[goombah99]

Why are computers still being used for simulating nuclear weapons tests?

Historically, the modern reason for computer simulation of nukes is to put a stopper in the nuclear proliferation genie. The logic is convoluted but sensible. The idea is that first you get a test ban treaty. Second, You offer economic and power production aid to all countries that dont develop nuke engineering or let you control their plutonium bearing nuke waste.

this creates a situation where nuke weapon engineering has to be done either in secret (since there no civial reactor technology to produce plutonium) or if done overtly, they still cant test their weapons. Neither can we.

this leaves everyone in a delightful position of 1) not being perfectly certain their nukes will work when delivered. thus they are not good offensive weapons. (imagine what would happen if pakistan launched on india and it were a dud).
2) yet they still make good defensive weapons since even though its not tested it doesn;t mean it wont work.

which is sort of nice. it discourages both developement and first use. world is MAD but better off.

Unfortunately the US would never go for this if they did not have a way of testing their own weapons. So they do it in silico rather than in nevada. This allows us the political will to go through with this. a better world results. THe clock is ticking. we know the weapons will work now but they are aging.. will they work in say ten years. THis is where computer simulations come in. within ten years we should be able to model nukes and nuke aging on one of these machines at a level that gaurentees our readiness. or maybe if this test ban thing works we can just scrap them all in ten years.

that was the plan. But now with about 30 countries with potential nuke development capability this plan maybe about to break down. thus we go to plan B.

plan B is we use these big computers to design new reactors that dont produce plutonium. We sell these to the countries. now they can have nuke power without creating weapons grade plutonium. Again every body happy.

except of course N. Korea.

Wait a sec..

[Epistax]

The IBM research team is currently running a large Linux cluster to simulate Blue Gene.

So then why don't have we have the simulation of Blue Gene run a simulation of Blue Gene two, and that run a simulation of a quantum computer, and that run a simulation of Deep Thought? Then that can run a simulation of the rest of the universe.
Then the two will bicker and argue about who's real, whom created whom, and millions of Matrix freaks will yell "I told you!!!" to those who have ridiculed them so many, many years.

Re:Um, maybe IBM should concentrate on making mone

[lucabrasi999]

not having a profit-making strategy.

Ahhh, you are forgetting the army of overpriced IBM consultants that you'll have to hire to install the thing.

Is this news?

[phch]

The referenced article is dated October 2002. Is this a mistake, or is this old news?

Anyhow, going to the Blue Gene web page, there is a document dated Nov 2002, an overview of BlueGene/L. An excerpt:

The approach we have adopted is to split the operating system functionality between compute and I/O nodes...

The compute node operating system, also called the BlueGene/L compute node kernel, is a simple, lightweight, single-user operating system that supports execution of a single dual-threaded application compute process...

I/O nodes are expected to run the Linux operating system, supporting the execution of multiple processes. Only system software executes on the I/O nodes, no application code.

What video card to use with this bad boy?

[Anonymous Coward]

Do you go with ATI or Nvidia?

Good frame rate for Quake 3??

AA on or off?

VSynch on or off?

What *I* Can't Wait For...

[philovivero]

When we get a supercomputer like this and the end of the article isn't "Some company will use this to find newer more efficient ways of killing people" but instead "Some university will use this to find ways of improving society at large."

I'm dreaming. I know.

And in related News...

[Ieshan]

Michael Jackson has released a new hit single that denounces this upstart of a project as anything but his lover.

Bug report

[S.I.O.]

Dear Linus,

the kernel is becoming slightly unstable with more than 10 trillion bytes and 65000 CPUs, please try to reproduce the situation. See the attached memory dump file.