IBM Open Sources Supercomputer Code

eldavojohn writes "IBM has announced at the LinuxWorld conference that they are now hosting all their supercomputing stack software as open source from the University of Illinois. From the article: 'The software will initially support Red Hat Enterprise Linux 5.2 and IBM Power6 processors. IBM is planning to add support for Power 575 supercomputing servers and IBM x86 platforms such as System x 3450 servers, BladeCenter servers and System x iDataPlex servers. The stack includes several distinct software tools that have been tested and integrated by IBM. These include the Extreme Cluster Administration Toolkit (xCAT), originally developed for large clusters based on Intel's commodity x86 architecture but now modified for clusters based on IBM's own Power architecture. xCAT is used in the National Nuclear Security Administration's Roadrunner Project at Los Alamos National Laboratory in New Mexico — a hybrid cluster currently ranked by the official Top 500 list as the world's most powerful supercomputer.' For several years, Linux has been a strong tool for supercomputing."

Comments

Sweet

[0100010001010011]

Now I have something to run on that spare Power4 I have laying around in the basement.

Re:Sweet

[VoyagerRadio]

Seriously, though: would the Power6 have been the successor to the chip family we call Apple's G5? (You know, if Apple had continued using PowerPC chips?) I'm researching this using Wikipedia but haven't yet discovered the answer...any Slashheads know?

Re:Sweet

[pleappleappleap]

I don't know about you, but I actually do have extra POWER hardware in my basement. And my living room. And my dining room.

Re:Sweet

[pleappleappleap]

No. The PowerPC architecture is mostly a subset of POWER. The POWER processors have all of the instructions of PowerPC, plus more. And they're built a little more robustly, since they're designed for the enterprise server market.

Re:Sweet

[EvilRyry]

Something power6 derived anyways. Apple always wanted their chips with the Alitvec instructions which weren't part of any of the other power series. They also didn't want to pay a whole lot for these custom chips which they order in relatively small quantity. Its little wonder IBM didn't rush to get them new CPUs, they're probably happy Apple is just leaving them alone.

Re:Sweet

[pleappleappleap]

IBM's later chips have AltiVec, even though they have nothing to do with Apple anymore.

Re:Sweet

[davester666]

Really? Small quantities? IBM is really selling so many supercomputers that they need to produce more than a million of these Power chips every year?

More likely, the requirements that Apple wanted/needed for it's PowerPC chips were different enough from where IBM wanted for the Power chip line, and IBM/Apple couldn't come to a financial agreement to produce the PowerPC chips that Apple needed [as in, the combination of cost per chip/capabilities of chip/when chip would be available that Apple wanted and what IBM could/would provide were different enough that they decided to go their separate ways.

And I would guess that IBM really didn't want Apple to sell their POWER chips, because Apple's XServe line would probably significantly impact IBM's sales of their POWER blades...

Re:Sweet

[Anpheus]

If each supercomputer contains a hundred thousand CPUs, they only need to sell TEN supercomputers.

If each mainframe contains a hundred CPUs, they only need to sell ten thousand mainframes.

See, those are quantities that help make sense of this. A Blue Gene/P installation can use up to nearly 900,000 processors alone.

So yes, IBM probably does ship more CPUs than Apple does. IBM doesn't just fabricate and sell Power chips either, so I'd say there's probably a pretty wide margin.

Re:Sweet

[ksd1337]

I hear Vista runs decent on this hardware.

Re:Sweet

[Anonymous Coward]

With the Intel based XServes, do you really think Apple is taking market share away from Dell? From what I can tell Apple's XServes don't really take away anyone's market share, they seem to be growing a niche market for easy rendering/encoding servers.

Re:Sweet

[tyrione]

Keep thinking that when Apple surpasses 10 Million Intel packaged systems or greater by mid 2009, year over year.

Re:Sweet

[Anpheus]

There's an IBM chip in every one of the three major consoles, which have sold around 50 million units in 2 years.

And that's a small part of their business. IBM sells a lot more chips, period, than probably anyone other than Intel or maybe there's an ARM manufacturer that does more business. But IBM also fabs ARM CPUs, so there you go.

Great

[thammoud]

Just when there no longer any COBOL programmers around.

Re:Great

[pleappleappleap]

COBOL for supercomputers? I don't think so. More FORTRAN. COBOL is used for business code. These machines are primarily used for modeling.

Re:Great

[Nerdfest]

These machines are primarily used for modeling.

You mean like on a runway across the top of the machines? You may have just thought of a way to re-ignite interest in supercomputing.

Re:Great

[ragefan]

COBOL for supercomputers? I don't think so. More FORTRAN. COBOL is used for business code. These machines are primarily used for modeling.

Yeah, I think of a single good reason to run business code on an International Business Machine! </sarcasm>

Re:Great

[rubycodez]

in the 70s and 80s COBOL compilers were available from Cray and CDC to run on their supercomputers.

We don't host xCAT

[dlapine]

Try hereinstead. And yes, xCAT kicks butt if you want to run a linux cluster. More so, now that it's open source.

Re:We don't host xCAT

[Anonymous Coward]

Uh, yes we do.

ftp://linuxpatch.ncsa.uiuc.edu/OpenHPC/

Re:We don't host xCAT

[dlapine]

I sit corrected. IBM has allowed NCSA to host a special version of xCAT for RHEL 5.2 on PPC. You can find it in here.

Re:We don't host xCAT

[CronoCloud]

Hmmm. YDL 6 is CentOS based. I wonder if this would run on a PS3.

But can I play Quake on it?

[Anonymous Coward]

Well, can I?

Re:But can I play Quake on it?

[ag3ntugly]

who cares if it plays quake, can it play crysis?

Re:But can I play Quake on it?

[X0563511]

Being a supercomputer, I would imagine it could play Crysis comfortably in a pure virtual machine, emulating a graphics card as well.

Re:But can I play Quake on it?

[negRo_slim]

Potentially, but would programming responsiveness not be an issue in a system just designed to digest data, the IPS might be there but there could be more to it then that.

Re:But can I play Quake on it?

[X0563511]

Hmm, true. But, I bet a renderfarm on this would be awesome!

Re:But can I play Quake on it?

[pleappleappleap]

You could probably set this up with a supercomputer displaying over parallel high-speed links to a medium-sized to large SGI Origin 3000-series.

Re:But can I play Quake on it?

[hdparm]

And you're absolutely sure this would run Crysis?

Re:But can I play Quake on it?

[Hollinger]

But can I play Quake on it?

Actually, you can. A couple of games are available on the IBM AIX software FTP site. You have your choice of Quake or Quake 2, even. Since POWER6 still runs binaries from the previous generations of software, go give it a try. Of course, you'd probably need to launch one instance of Quake per thread to really stress the system, since I'm pretty sure Quake's not multithreaded. ;-)

~ Mike

aix?

[Anonymous Coward]

why won't they open source the AIX code?

Re:aix?

[swrona]

Because IBM would lose control of the product, and it would be difficult to support multiple different versions, or "flavors" as they might become known as. Oh wait, they'd probably lose some money somewhere along the way too.

Re:aix?

[galoise]

mmm, 'cause they are still trying to sell it as-a-product?

Re:aix?

[hostyle]

And open sourced linux vendors never do that, (raises eyebrows) or do they!?!

Re:aix?

[jeremyp]

No, they don't.

They sell support and services around the distribution. Even packaging the product up onto a CD with some printed manuals can be considered a service.

Obligotory

[seanonymous]

Imagine a Beowolf cluster of these clusters.

Re:Obligotory

[pleappleappleap]

That would be exceedingly stupid. Why not just make a larger cluster over the high-speed clustering medium instead of throwing unnecessary Beowulf overhead into the process.

Re:Obligotory

[conspirator57]

you joke, but when the limit of the number of nodes that can participate in a cache-coherent, shared memory architecture is n and the number of nodes the customer bought is 10n and the customer wants to get on the top 100 list, you build the cluster, run the benchmarks, and lament the lie (to yourself and coworkers).

Re:Obligotory

[treeves]

Better yet, imagine an obligatory Beowulf cluster of spell-checkers.

Re:Obligotory

[seanonymous]

Yeah, what the hell, Firefox, I thought you were on my side.

Visualization

[Anonymous Coward]

With the help of the xCAT administration tool we no longer need to imagine but now can in fact visualize a Beowulf cluster of these.

Yes, but does it run Linu...

[Anonymous Coward]

Oh wait, nevermind.

Hmm

[inKubus]

So guess we won't have to imagine a beowulf cluster of this. Phew, meme crisis averted.

the real goal

[xzvf]

I work for IBM, but this is speculation. The vast majority of money generated and earned on large Linux clusters came from selling hardware and services. This can only help generating that business.

Re:the real goal

[Anonymous Coward]

even the hardware itself is slowly becoming a commodity, so services is where its at for sure

Re:the real goal

[mikael]

I heard this story about Cray. If you bought a supercomputer and support contract from them, they would throw in the free construction of an office block to run the computer in.

Just in time

[TechJones]

Just in time, I really needed a supercomputer. Let's play Global Thermonuclear War

Re:Just in time

[Anonymous Coward]

haha funny.

mod parent up.

I'm confused about IBM

[contrapunctus]

They open source stuff and they patent ridiculous stuff. Am I supposed to like them or not?

Re:I'm confused about IBM

[pleappleappleap]

If you don't want them to have to patent "ridiculous stuff" ensure that the business environment changes so that they don't need to.

Re:I'm confused about IBM

[contrapunctus]

ensure that the business environment changes so that they don't need to

Thanks, I'll get right on that. Wow.

Re:I'm confused about IBM

[pleappleappleap]

I picked up on the sarcasm, but that really is our duty as customers and voters in this regard.

Re:I'm confused about IBM

[contrapunctus]

Sorry, but your comment was similar to:
If you don't want people around the world to starve, ensure that they have enough food.

This is not specific enough to be meaningful.

Re:I'm confused about IBM

[Hurricane78]

But that's exactly the point.
Food actually hurts starving countries even more than it helps them.
This is, because normally nature falls in a balanced state by itself.
If you give them more food, they can raise more kids. Not a bad thing, except if afterwards you don't provide food for them too, and so on.
Fact is: The land can't support more people, so if there are more born than there die, they will die too.
This will happen to the whole world, as soon as it reaches the global limit.

So the best thing you can do in teach them. Teach them your tricks for how to optimize their resources. Teach them better ways to act on the global market. Teach them logic, reasoning, "basic" knowledge, and how to defend themselves, so they can't be oppressed and exploited that easily.

Therefore, the most important target right now should be, to make the Internet, and it's huge global knowledge base and collaboration / trade system available to *everyone* even people who can't read and live in a clay hut. (There does only need to be one nerd-prototype in the village who can teach them stuff from the net in exchange for a part of their food.

I think sending food and clothes is more cruel than to do nothing. (Note that i did not make a statement about the cruelty of doing nothing.)

Re:I'm confused about IBM

[Nerdfest]

I tend to think they use open source software as a way to sneak high priced consultants in to sell high priced software and support, but I might be being paranoid.

Re:I'm confused about IBM

[contrapunctus]

But see, I wasn't specific enough to say "give them food" I said "ensure that they have enough food" which could mean teaching or anything I want.

My whole point was that making vague statements is meaningless. They don't mean anything. They could mean anything.

Re:I'm confused about IBM

[warsql]

Except that hunger is a political problem.

Photoshop this Supercomputer Dialysis Machine

[Nymz]

Since /. was adding pictures, Farking them is only a natural extension of expression. Supercomputer Dialysis Machine

Re:Photoshop this Supercomputer Dialysis Machine

[daybot]

Since /. was adding pictures, Farking them is only a natural extension of expression. Supercomputer Dialysis Machine

Brilliant :)

IBM

[Anonymous Coward]

IBM has always been good quality in my eyes.
What can I say I am a bias fanboy who is a sucker for a legacy.
IBM has been supporting the Open source community but has the community returned the favor?

patents?

[Anonymous Coward]

will it be offered in paper or plastic ?

The Alpha Troll laughs at you.

[Anonymous Coward]

Power is so tomorrow non-existant technology.
It will never match the power and scope of existing DEC, Compaq, and HP Alpha ev5 and ev6 processors.

If Apple chose the DEC Alpha back when it was looking for alternatives from Motorolla, then the market would have progressed a lot better than the Intel and AMD crap that has become today. Ever since then, Alpha was chopped intellectually into Intel and AMD, SGI's MIPS has all but disappeared into stagnation, Sun's SPARC has dwindled to sh!t, laptops are crude and delicate, and 3Dfx has disappeared into nVidia.

The market has died, because they didn't go with Alpha.

California Payroll Fix!

[blckholehorizon]

maybe we could set up a super computer now in the state of California to fix their COBOL programs, to exhaustively iterate though all combinations, of COBOL statements, and eventually it will come up with a solution to their payroll problems. We could have it race with programming monkeys, or ninjas. Whoever loses gets to arm wrestle Arnold.

Er, hasn't xCAT been open source for years?

[Troy Baer]

I mean, it has a SourceForge page whose mailing list archives go back to 2001, fer cryin' out loud.

Now some of the "OpenHPC" stuff appears to be new, but not all of it appears to originate from IBM. For instance, part of it appears to be a repackaging of the SLURM batch system from LLNL. The one thing that looks like a genuine contribution from IBM is the "Advance Toolchain" stuff, but even that appears to draw heavily from existing open source code bases like valgrind.

Cluster administration software should only be FS

[shlompo]

I used to be in charge of administrating the lab cluster at the MOSIX project (http://www.mosix.org). The tools we used back then, where series of scripts, that performed all possible configurations you'll ever need... we called it CLIP (CLuster Installtion Package). My two years experience taught me two things:
1. It's sometimes easier to script your way through, instead of adapting existing administration tools. You'll just have a peek first, of course...
2. But when you must, you'll encounter a modification you'd want very quickly.
So my advice would be only accept open source administration systems. As i'm sure others have reached the same conclusions i had, This is actually a win-win move by IBM, and i'm sure they'll get more users, and more income following.

Alright. now compare this to what microsoft

[unity100]

did in latest oscon. what do you see ? rock solid commitment compared to empty pr. you know which of them pertains to which company ...

I really want to see someone answering this :

[unity100]

IBM has been supporting the Open source community but has the community returned the favor?

Re:I really want to see someone answering this :

[msormune]

There's a difference in OFFERING source code for something, and making it ACTUALLY open source. Slashdot mixes these two all the time.

Re:I really want to see someone answering this :

[msormune]

Oh to answer you question: Yes, the community can return the favor by using Java. The reason for IBM "giving away" Eclipse was to get programmers to accept Java.

Re:I really want to see someone answering this :

[houghi]

By supporting and contributing to Open Source, they become part of that community, so yes they have returned the favor to at least them selves.

The community are not just the individuals. It is everybody and everything that contributes. They have the ability to use anything from Open Source. If they desire to do so or not is up to them, not to the community.

So the question is not if the community has given anything, but if IBM has taken anything and I would say yes. They are using code that is written by others.

Re:I really want to see someone answering this :

[Anonymous Coward]

You mean, besides contributing on the order of millions of volunteer man hours developing the open source software which is the foundation of the whole movement?

Nope...

[Anonymous Coward]

xCAT prior to 2.0 was under a particularly restrictive license disallowing, among other things, redistribution. It's had some of the makings of an open source project, but not the license to allow it to really be so. It has been free as in beer, but that was all. It was written in interpreted languages, so the non-opensource nature was not as obvious.

Now it's under EPL (a BSD-like license).

Re:Nope...

[jabuzz]

And even now, the jewel in the crown which is GPFS is still close source, pay for.

Sun did this years ago...

[Anonymous Coward]

With Sun Grid Engine and Sun Visualization System. Last year Sun bought Lustre, which is GPL licensed, and Sun contributes to the Open MPI project.

http://gridengine.sunsource.net/
http://en.wikipedia.org/wiki/Sun_Microsystems#HPC_solutions

Ask and you shall receive

[Gazzonyx]

Eclipse. Every summer they do a multi project release on the same day. This year the release was called Ganymede. The release announcement follows (emphasis added):

The Eclipse Foundation and the entire Eclipse community are pleased to announce the availability of the Ganymede Release, the annual release train developed by the Eclipse community. The Ganymede Release is a coordinated release of 23 different Eclipse project teams that represents over 18 million lines of code.

IBM didn't write all that code ;)