A Look Inside the NCSA

Peter Kern writes "The National Center for Supercomputing Applications (NCSA) is one of the great supercomputing facilities in the world and is home to 'Abe', one of the top 10 supercomputers on the current Top 500 list. TG Daily recently toured the facility and published a stunning report about their computing capabilities (more than 140 teraflops), power requirements (a sustained 1.7 megawatts), enormous 20-ft chillers in four cooling systems and other installations that keep the NCSA online."

Comments

That's a lot of number crunching

[Junior J. Junior III]

Who knew Mosaic was so bloated? No wonder no one uses it anymore.

Job requirements...

[ushering05401]

Just out of curiosity... does anyone know minimum requirements for getting on as a server tech in a place like that?

Really contemplating computing power like they describe is a pretty far out exercise for a small time programmer like me... What sort of people get employed at these places?

Regards.

Re:Job requirements...

[Timesprout]

Apparently you need to be 20 feet tall and have a very relaxed attitude and a cool head.

Re:Job requirements...

[morgan_greywolf]

I'd make sure I had ample experience in the systems and networking administration arenas. Know multiple flavors of UNIX, know Linux, and know multiple clustering technologies -- everything from shared-memory architectures to high performance clustering to grid computing to high availability systems. Know the systems available from multiple vendors -- IBM, HP, Sun, Red Hat, Veritas. Knowing storage area networking is pretty smart also. Know networks -- understand them at all levels in the OSI and TCP/IP models. Understand application and system-level debugging. Understand how to analyze the performance of a complete system, from the application level all the way to the lowest levels of an individual node.

Oh, and being able to think on your feet, the ability to communicate with engineers and scientists, and being very organized and able to work independently doesn't hurt either.

Re:Job requirements...

[Linker3000]

...so CompTIA's Network+ and a bit of bench experience should do the trick eh?

Re:Printable Link - All in one page

[WED Fan]

In 10 years, this will be on the desktop, everyone will yawn because we have been boiled frogs and it won't impress us then. In 10 years, you'll look at someones tie clasp computer and say, "Wow, I remember when that took up an 8 by 18 block of my desk."

In 10 years, DARPA will announce the shut down of the Quantum Computing Project because it will be discovered that every time Red Hat Mandriva Winux OS/Q green screens, a parallel universe winks out of existance.

In 10 years, they'll slap wheels on your grandmother's behind and call her a wagon.

Fire Protection System

[BlueLightSpecial]

Is there some way to perform a graceful shutdown before the water gets pumped and released? If the supercomputers are still on when the water is discharged from the pipes, wouldnt that damage the systems? If they dont want to use halon why not use a more computer-friendly compound like FM-200 (http://en.wikipedia.org/wiki/FM-200/)

/.ed already

[blhack]

1.21 gigaflops and their webserver is an old guy with tourette syndrome yelling HTML code into a tin can on a string.

Re:UIUC FTW!

[panzagloba]

I have actually been in the newer facility dozens of times when I worked as an intern for the Architect on the building. I actually drafted the final drawings for this project. It is a VERY nice facility, with some pretty cool under-floor cooling systems and things like that. I am pretty sure I have 3D digital models of the facility somewhere in my work records.

The lecture auditorium bites the big one though, purple seats? Nasty. The Seibel Center accross the mini-quad is a much more interesting building though, at least to an architect. :)

Cause of Global Warming

[ISoldat53]

New computer simulations indicate that supercomputers are a major source of global warming.

Re:It's not "the NCSA"

[Intron]

Wouldn't want to get confused with that other large supercomputer customer.

6 hour runs?

[Vellmont]

The most surprising thing in the article was how inelegantly they've solved the problem with inevitable hardware failure. That is, limiting runs to only 6 hours. It seems like there just HAS to be a better way to handle the problem than this! Virtualization sounds a bit tricky, so why not just write the software to handle hardware errors in the first place? I.e. produce results, check to see if there was a hardware failure, if so, re-do.

Maybe they already do this, and the reporter didn't catch it. But it'd surprise me if they didn't have better solutions than just hoping nothing bad happens during a run.

Re:6 hour runs?

[Vellmont]

Much of the software which is run at the NCSA is home-grown software written by computational scientists, not computer scientists.

I've seen code written by computational guys before. While not really terrible, it's not terribly re-usable or maintainable. Obviously these guys don't study computer science, but I truly think there's gains to be made if they understood the tool they were using better.

  For many of these massively parallel codes, written on top of MPI, fault tolerance really isn't all that easy. For a commercial production code on the order of Gaussian, this may be doable, but for bleeding-edge research codes, it may be a better use of the (human) time to push the algorithms rather than worry about fault-tolerance. From the user's perspective, jobs that are killed due to a hardware failure have their service units refunded, so there isn't a huge incentive to worry about it.


As long as your job runs under 6 hours, sure. But if it takes over 6 hours, you're already doing some kind of saving and re-starting. That's probably about 80% of what I'm talking about, just on a larger scale. I bet ou're right though, it's all going to come to a head as there's more and more components that could fail, so it has to be fixed at a higher level, or the programmer level. Maybe you can fix the problem with virtualization, but how much of a performance hit do you take, or how much costlier is the machine?

Re:Wow!

[Tmack]

Let's see with 2 you can get 140 * 2 = "The BlueGene/L reached a Linpack benchmark performance of 280.6 TFlop/s ("teraflops" or trillions of calculations per second)."

3 and they could be #1 in the world :)

And you can power 711 of them with one Mr Fusion!

Tm