'Blue Waters' Supercomputer Lucky To Exist

Nerval's Lobster writes "One could argue that the University of Illinois' "Blue Waters" supercomputer, scheduled to officially open for business March 28, is lucky to be alive. The 11.6 petaflop supercomputer, commissioned by the University and the National Science Foundation (NSF), will rank in the upper echelon of the world's fastest machines—its compute power would place it third on the current list, just above Japan's K Computer. However, the system will not be submitted to the TOP500 list because of concerns with the way the list is calculated, officials said. University officials and the NSF are lucky to have a machine at all. That's due in part to IBM, which reportedly backed out of the contract when the company determined that it couldn't make a profit. The university then turned to Cray, which would have had to replace what was presumably a POWER or Xeon installation with the current mix of AMD CPUs and Nvidia GPU coprocessors. Allen Blatecky, director of NSF's Division of Advanced Cyberinfrastructure, told Fox that pulling the plug was a 'real possibility.' And Cray itself had to work to find the parts necessary for the supercomputer to begin at least trial operations in the fall of 2012."

Comments

Alive? Has it achieved sentience?

[billstewart]

You can only say it's "lucky to be alive" if you think it's alive. The standard theorem of AI is that when anything AI-ish gets developed, people say "Oh, that's not really Intelligence, that's just {Pattern Recognition / Expert System solving / Machine Vision / OCR/ etc.}" But if it is actually alive, then it's lucky somebody noticed so they know not to turn it off.

Re:Alive? Has it achieved sentience?

[ThePeices]

Greetings pedant!

You make many logical and valid points as any pedant worth their salt is apt to do, but unfortunately it all falls to pieces when I point out that "lucky to be alive" is actually what we like to call "a figure of speech".

But please don't feel bad, misunderstandings like these happen to pedants of all ages, we are human after all.

In all honesty, we here at Slashdot forgive you.

Re:Alive? Has it achieved sentience?

[NatasRevol]

Speak for yourself. I think he's an asshole for typing out that pointless, obvious statement.

Can't have it both ways..

[Junta]

because of concerns with the way the list is calculated

Read: because it won't look as nice as throwing the Rpeak of 11.6 Petaflops out there, and the ration of Rmax to Rpeak will look poor as well.

I know the Top500 is a BS, single dimensional metric. It is valid to call it out on that. However, to do so while also trumpeting '11.6 Petaflops' is disengenious since it is also a BS single dimensional metric that in many ways can be pulled completely out of ones ass, which is even worse than a measured value. HPC Challenge Benchmark has the noble goal of measuring the character of an HPC system in a more holistic manner, but no one pays as much attention to it. When occasionally a supercomputer installation does skip a Top500 submission, people tend not to think about that installation so much.

Of course, it's completely bizarre that placement in any such global list is a factor in purchasing and design at all. It really should be about the specific needs of the group funding it and how they are met, not some penis measuring contest.

Re:Can't have it both ways..

[rmstar]

Of course, it's completely bizarre that placement in any such global list is a factor in purchasing and design at all.

There is a whole lot of politics involved in HPC. The only way to justify the ridiculous costs (not only in hardware, but also in energy) for this type of machines is by convincing university management that they are going to look really good ("We will be recognized as global players if we have a machine in that list", etc, etc.). This is indeed bizarre, but the way it is.

The people who build and manage these machines aren't usually terribly interested in the needs of scientists running jobs on it. They are just obsessed in piling up hardware and getting the credits, like race car builders. The needs of the scientists tend to be an afterthought. Quite often, and I have that as an insider, these machines idle around because a lot of scientists have a nice quad-core machine under their desks fullfilling their computing needs just fine, and getting something to run on the monster machine is a hassle.

On top of that, many of the applications that actually run on these machines are things like least-squares fitting problems, but solved using genetic algorithms (which is ridiculous) mainly due to the ineptitude of the scientists involved. Or absurd brute force Monte-Carlo simulations done simply because some scientists don't know or understand differential equations. It so happens in many disciplines that doing something with an HPC machine using some stupid but expensive algorithm will get you more hype credits than doing it smartly on last decade's laptop in a tenth of the time and one millionth of the electrical energy.

So, yeah, it's completely bizarre. That the good folks of the 'Blue Waters' machine rebel sounds very nice. I wonder if they are just deluded and missing the only point of building such a machine, though (which seems to be to race in the Dongarra 500 oval track challenge).

Re:Can't have it both ways..

[Cyrano de Maniac]

That might be painting supercomputer owners with a bit too broad of a brush.

The NASA Advanced Super Computing Division (www.nas.nasa.gov) is crammed full of supercomputers of various designs (clusters, single-system image behemoths, co-processing, etc) and from everything I've ever heard they run at insanely good utilization levels. If I remember a presentation from one of their chiefs correctly, they achieved this level of utilization not only with great technical management and know-how, but by consolidating a number of disparate less efficient smaller supercomputer centers across NASA. They also have staff whose job it is to work directly with researchers to help them code their supercomputing applications in a manner to get as much performance as possible out of the computer -- greatly improving the poor/inefficient code that a scientist may produce. As a result the division's existence ends up saving NASA money overall, the result of which is that various science organizations are able to spend a greater portion of their funds on core research rather than inefficiently utilized supercomputing resources, a net win for them and for taxpayers.

The other impression I walked away with is that the only obstacle to doing even more science with their supercomputer resources was the dollars to obtain more systems (i.e. the demand for cycles outstrips the center's currently available supercomputer resources). I seem to recall the chief mentioning that today's systems are running calculations that were science-fiction level of impossible 15-20 years ago, and that several researchers have ideas for supercomputing applications that would require capabilities still hundreds or thousands of times better than could be built today. The need for more supercomputing capability, at least at NASA, is effectively boundless.

So I think the parent's portrayal is overly broad. An inefficiently used supercomputer is primarily a management failure, and the quality of management varies from organization to organization, as it does in any field. What I've hard about NAS sounds like they did it right. I have no doubt there are organizations who've done it wrong.

Disclaimer: I work for a major supercomputing company, in particular the one that supplies a huge portion of NAS's hardware.

Re:NCSA should lose it's NSF funding anyway.

[Dop]

I will not comment on the originally submitted system (IBM) vs what was installed and the reasons behind that. However...

As a former employee of NCSA, I think I can shed a little light on this, at least from the employment side. To my knowledge, they never fired an admin, at least not recently (largely because firing people at the University of Illinois is extremely difficult, even when it's totally justified in some cases). Certain admins left for better opportunities, but I can hardly blame them.

You mention both Cray and NCSA having issues finding new folks. That's two-fold in my opinion. Very few people _want_ to move to Champaign, IL. You have to be the kind of person that wants to live in a college town that's a good 2 hours from a big city, surrounded by corn fields, and has shitty winters (oddly I'm that person). In addition to that, partly because of the horrible State of Illinois budget issues and the fact that NCSA is a department of the UofI, they don't pay market rate for qualified individuals. They used to justify this by really good benefits, but those have all been eroded.

In a market where the best of the best (often working remotely from wherever they want) are making more than NCSA managers, it's no wonder they can't find anyone to fill technical positions. I'm not sure if other NSF funded institutions are in any better shape. Would Blue Waters really be better off at another location? I'm not sure.

All that said, I'm extremely grateful for my time at NCSA and the amount I was able to learn with state of the art technology. It's just that working will cool stuff (and great people) doesn't pay the bills anymore.