Slashdot Mirror


$208 Million Petascale Computer Gets Green Light

coondoggie writes "The 200,000 processor core system known as Blue Waters got the green light recently as the University of Illinois at Urbana-Champaign and its National Center for Supercomputing Applications (NCSA) said it has finalized the contract with IBM to build the world's first sustained petascale computational system. Blue Waters is expected to deliver sustained performance of more than one petaflop on many real-world scientific and engineering applications. A petaflop equals about 1 quadrillion calculations per second. They will be coupled to more than a petabyte of memory and more than 10 petabytes of disk storage. All of that memory and storage will be globally addressable, meaning that processors will be able to share data from a single pool exceptionally quickly, researchers said. Blue Waters, is supported by a $208 million grant from the National Science Foundation and will come online in 2011."

34 of 174 comments (clear)

  1. imagine... by spiffmastercow · · Score: 5, Funny

    nah, nevermind

  2. $208 Million Petascale Computer Gets Green Light by Naughty+Bob · · Score: 5, Funny

    I'm glad they've given it a green light.

    Imagine having all that computer power, and not even knowing if it was switched on!

    --
    "Be light, stinging, insolent and melancholy"
  3. You know, that IS impressive but... by Xaedalus · · Score: 2, Funny

    Can it figure out how to brew the 'perfect' cup of coffee?

    --
    Here's to hot beer, cold women, and Glaswegian kisses for all.
    1. Re:You know, that IS impressive but... by InlawBiker · · Score: 2, Informative

      I think you meant tea.

  4. Naive question... by religious+freak · · Score: 2, Interesting

    Yes, I know this is probably a very naive question, but has anyone here actually had the privilege of working on one of these things? I mean, what do they actually use this for?

    I think it's awesome, but are there any concrete advancements that can be attributed to having access to all this computing power?

    Just wondering...

    --
    If you can read this... 01110101 01110010 00100000 01100001 00100000 01100111 01100101 01100101 01101011
    1. Re:Naive question... by Naughty+Bob · · Score: 2, Funny

      I mean, what do they actually use this for?

      I think it has been designed to run IE8 beta 2.

      --
      "Be light, stinging, insolent and melancholy"
    2. Re:Naive question... by Glith · · Score: 4, Funny

      Come on now. Let's be serious. They're trying to play Crysis.

    3. Re:Naive question... by serviscope_minor · · Score: 4, Interesting

      I don't use one myself, but I know people involved with supercomptuers. They are used for large simulations. Often this comes down to solving large systems of linear equations, since at the inner step finite elements need solutions to these large equation systems. The point is, the larger the computer the larger the grid you can have. This involves simulating a larger volume, or simulating the same volume in more detail (think, for example of weather systems).

      As for concrete advancemants? I'm not in the biz, so I don't know, but I expect so. Apparently they're also used for stellar simulations, so I expect the knowledge of the universe has been advanced. I would be suprised if they haven't seen duty in global warming simulation too.

      --
      SJW n. One who posts facts.
    4. Re:Naive question... by Deadstick · · Score: 4, Informative

      Weather modeling comes to mind, both terrestrial and space.

      rj

    5. Re:Naive question... by mikael · · Score: 4, Informative

      These machines are used to work on simulations that involve aerodynamics and hydrodynamics, quantum electrodynamics (QED), or electromagnetohydrodynamics. All of these simulations require that a mathematical model is constructed from a high density mesh of data points (2048 ^ 3). Blocks of such points are allocated to individual processors. Because of this, each processor must be able to communicate at a high speed with its neighbours (up to 26 neighbours with a cubic mesh).

      Usually, the actual individual calculations per element will be take up less than a page of mathematical equations, but require high precision, so the data values will be 64-bit floating point quantities. A single element might require 20 or more variables. Thus the need for some many processors and high clock speed.

      --
      Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
    6. Re:Naive question... by Ilgaz · · Score: 5, Informative

      Do you notice neither USA or Russia blows a portion of planet to test nuclear weapons anymore? It is because the planet is so peaceful so further research is not required? Unfortunately no.

      These monsters can simulate a gigantic nuclear explosion in molecular level.

      Or for peace purposes, they can actually simulate that New Orleans storm based on real World data and pinpoint exactly what would happen.

    7. Re:Naive question... by Ilgaz · · Score: 3, Interesting

      Did you know that a very credible FAQ mentions Apple purchased a Cray for manufacturing/design and someone actually saw them emulate MacOS on that monster?

      http://www.spikynorman.dsl.pipex.com/CrayWWWStuff/Cfaqp3.html#TOC23

      I bet they tried some games too :)

    8. Re:Naive question... by blantonl · · Score: 2, Funny

      Or for peace purposes, they can actually simulate that New Orleans storm based on real World data and pinpoint exactly what would happen.

      Right.

      That's why the City of New Orleans evacuated to Baton Rouge.

      --
      Lindsay Blanton
      RadioReference.com
    9. Re:Naive question... by adona1 · · Score: 3, Funny

      Actually, no, they're future proofing their computer for Duke Nukem Forever :)

      --
      Between the falling angel and the rising ape
    10. Re:Naive question... by Rostin · · Score: 4, Informative

      I'm working on a PhD in chemical engineering, and I do simulations. I occasionally use Lonestar and Ranger, which are clusters at TACC, the U. of Texas' supercomputing center. Lonestar is capable of around 60 TFLOPS and Ranger can do around 500-600 TFLOPS. A few users run really large jobs using thousands of cores for days at a stretch, but the majority of people use 128 or fewer cores for a few hours at a time.

      My research group does materials research using density function theory, which is an approximate way of solving the Schroedinger equation. Each of our jobs usually uses 16 or 32 cores, and takes anywhere from 5 minutes to a couple of days to finish. Usually we are interested in looking at lots of slightly different cases, so we run dozens of jobs simultaneously.

      The applications are pretty varied. Some topics we are working on -
      1) Si nanowire growth
      2) Si self-interstitial defects
      3) Au cluster morphology
      4) Catalysis by metal clusters
      5) Properties of strained semiconductors

    11. Re:Naive question... by cibyr · · Score: 2, Funny

      I bet they tried some games too :)

      Nonsense! Everyone knows there aren't any games for mac :P

      --
      It's not exactly rocket surgery.
    12. Re:Naive question... by bh_doc · · Score: 5, Funny

      electromagnetohydrodynamics

      And quantum electroptical tomographics. See, I can make shit up, too...

    13. Re:Naive question... by dlapine · · Score: 2, Informative

      For a reasonable sample of the things that can be done on a supercomputer, start here: http://www.ncsa.uiuc.edu/Projects/. Those are just the things running at NCSA.

      Followup with this, as the science gateways for the TeraGrid are designed to let scientists worry more about the science part and less about the programming part. Part of the reason to build bigger supercomputers is to let non-programmers get work done as well. By having more cycles available, the TeraGrid can allow access for codes that are easier for the average scientist to use, even if they don't make the best use of the machine. Not everyone is a wiz at parallel programming, and we shouldn't expect an expert in say, biology, to be just as expert in computer science.

      --
      The Internet has no garbage collection
  5. Re:oblig. by Bill,+Shooter+of+Bul · · Score: 2, Informative

    No. This is Urbana,Illinois. HAL 9000 would be more appropriate.

    --
    Well.. maybe. Or Maybe not. But Definitely not sort of.
  6. More crap code by kramulous · · Score: 2, Insightful

    Cool thing about the globally addressable petabyte. That way people writing really crappy code that don't bother thinking about their memory storage can just thrash away. And who cares about pipeline stalls.

    I find it funny how the people who have never been formally trained with writing in a language (Mathematics, and just science in general) write the best codes while the majority of the IT people I see write the most appalling code I've ever seen. I think it has something to do with the fact that the science people don't pretend to know everything and are much more willing to learn something new while the IT people already know everything.

    --
    .
  7. Star Trek "Data" rated at 60 Teraflops by peter303 · · Score: 4, Interesting

    I just saw The Measure of a Man episode on the Star Trek Labor Day marathon. Data has a speed of 60 Teraflops and 100 petabytes of storage. That used to seem large in the late 1980s. (Episode were Data goes on trial whether he is a machine or sentient.)

    1. Re:Star Trek "Data" rated at 60 Teraflops by Bones3D_mac · · Score: 2, Interesting

      About a decade or so ago, I remember someone very crudely trying to ballpark the amount of storage that would be needed to contain the raw data of the entire human brain complete with a lifetime of experience at around 10 terabytes. Needless to say, that seems incredibly unlikely by today's standards.

      Even if something like this were possible (storage not withstanding), the data itself would likely be unusable until we sufficiently understood just how our brains work with their own data enough to create a crude simulation to act as an interpreter. And, even with that, it's probably safe to assume that each brain sampled will likely have highly unique methods of storage and recall, each requiring their own custom-built brain-simulation interpreter.

      Somehow, I don't think we'll be seeing anything close to this happening within our lifetimes short of violating our ethics regarding the rights of human life. Basically, something to the effect of strapping someone down while we inject their brain with nanobots designed to disassemble the brain one cell at a time, and then emulate the cell that was just removed, until the entire brain has been replaced with a nanobot driven substitute. (Only with a few added features to allow communication with external devices.)

      --


      8==8 Bones 8==8
  8. Re:Yes, but the article doesn't address a few ques by QuantumRiff · · Score: 4, Funny

    It will not run 32 bit linux, so of course, the admins in charge are going to bitch about the lack of adobe flash support.

    --

    What are we going to do tonight Brain?
  9. Re:Not so sure its the first by Phat_Tony · · Score: 2, Interesting

    Yeah, that was my thought. Roadrunner at Los Alamos sits at the top of the 500 list with Rmax 1,026,000. I don't know enough about benchmarks to distinguish between "Rmax" and "sustained petascale," but it is achieving over a petaflop. Maybe someone here can tell us more about linpack vs. whatever they're using for this new one. I notice the article linked in the story mentions Roadrunner at the end, but without saying how it compares in speed. It doesn't seem to say by what specific measure this new computer's speed surpasses a petaflop.

    --
    Can anyone tell me how to set my sig on Slashdot?
  10. It's said... by jd · · Score: 2, Interesting

    ...Apple used to use a Cray to design their new computers, whereas Seymoure Cray used an Apple to design his.

    More compute power is nice, but only if the programs are making efficient use of it. MPI is not a particularly efficient method of message passing, and many implementations (such as MPICH) are horribly inefficient implementations. Operating systems aren't exactly well-designed for parallelism on this scale, with many benchtests putting TCP/IP-based communications ahead of shared memory on the same fripping node! TCP stacks are not exactly lightweight, and shared memory implies zero copy, so what's the problem?

    Network topologies and network architectures are also far more important than raw CPU power, as that is the critical point in any high-performance computing operation. Dolphinics is quoting 2.5 microsecond latencies, Infiniband is about 8 microseconds, and frankly these are far far too slow for modern CPUs. That's before you take into account that most of the benchmarks are based on ping-pong tests (minimal stack usage, no data) and not real-world usage. I know of no network architecture that provides hardware native reliable multicast, for example, despite the fact that most problem-spaces are single-data, most networks already provide multicast, and software-based reliable multicast has existed for a long time. If you want to slash latencies, you've also got to look at hypercube or butterfly topologies, fat-tree is vulnerable to congestion and cascading failures - it also has the worst-possible number of hops to a destination of almost any network. Fat-tree is also about the only one people use.

    There is a reason you're seeing Beowulf-like machines in the Top 500 - it's not because PCs are catching up to vector processors, it's because CPU count isn't the big bottleneck and superior designs will outperform merely larger designs. Even with the superior designs out there, though, I would consider them to be nowhere even remotely close to potential. They're superior only with respect to what's been there before, not with respect to where skillful and clueful engineers could take them. If these alternatives are so much better, then why is nobody using them? Firstly, most supercomputers go to the DoD and other Big Agencies, who have lots of money where their brains used to be. Secondly, nobody ever made headlines off having the world's most effective supercomputer. Thirdly, what vendor is going to supply Big Iron that will take longer to replace and won't generate the profit margins?

    (Me? Cynical?)

    --
    It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
    1. Re:It's said... by Bill+Barth · · Score: 3, Informative
      You could not be more wrong.

      Considering that we've got SDR IB with under 2 microseconds latency for the shortest hops (and ~3 for the longest), I think you need to go update your anti-cluster argument. :) The problems with congestion in fat trees have virtually nothing to do with latency. Yes massive congestion will kill your latency numbers, but given that you don't get cascades and other failures causing congestion without fairly large bandwidth utilization, latency is the least of your worries that that point. Furthermore, the cascades you talk about also aren't common except in extremely oversubscribed networks or in the presence of malfunctioning hardware. We do our best to use properly functioning hardware and to have no more that 2:1 oversubscription (with our largest machine not being oversubscribed at all).

      MPICH ain't that bad (heck, MPICH2, even just it's MPI-1 parts might be considered to be pretty good by some). MPI as standard for message-passing is fine. I'd love to hear what you think is wrong with MPI and see some examples where another portable message passing standard does consistently better. Though it's a bit like C or C++ or Perl in that there are lots of really bad ways to accomplish things in MPI and a handful of good ones. It's low-level enough that you need to know what you're doing. But if you believe anyone that tells you they have a way to make massively parallel programming easy, I've got a bridge you might be interested in.

      Finally, I don't know of much in the way of a "supercomputer" that's using TCP for it's MPI traffic these days, so you can put that old saw out to pasture as well.

      --
      Yes...I am a rocket scientist.
  11. Can You Imagine by nickswitzer · · Score: 2, Funny

    The amount of porn you can download with this thing? Isn't that the number one thing the computer has evolved to?

  12. Don't worry by EEPROMS · · Score: 2, Funny

    in 40 years some kid will laugh at your pathetic attempt at geek coolness when you mention the Bluewater and say "wow your old, Im amazed anyone needed a warehouse just for one petaflop even my Wango-matic game cube has 50 petaflops"

  13. F@H is already past 2.5 Petaflops by Anonymous Coward · · Score: 2, Interesting

    Folding @ Home easly trounces this puny supercomputer.

  14. Re:Yes, but the article doesn't address a few ques by KGIII · · Score: 2, Funny

    Nah but it will finally run Vista.

    --
    "So long and thanks for all the fish."
  15. Can't take another 40 by Main+Gauche · · Score: 4, Funny

    in 40 years some kid will laugh at your pathetic attempt at geek coolness when you mention the Bluewater and say "wow your old..."

    Forty more years of the kids saying "your"? Kill me now! :)

  16. Re:Not so sure its the first by DegreeOfFreedom · · Score: 2, Informative

    Blue Waters will be the first to deliver a sustained petaflop on "real-world" applications, meaning various scientific simulations. Specifically, the program solicitation required prospective vendors to explain how their proposed systems would sustain a petaflop on three types specific types of simulations, one each in turbulence, lattice-guage quantum chromodynamics, and molecular dynamics.

    Granted, Roadrunner was the first machine to deliver a petaflop on the Linpack benchmark (though certainly IBM's own implementation of it). The benchmark does nothing more than set up and solve a system of linear equations. Roadrunner solved a system of 2,236,927 equations (in other words, it had a 2,236,927-by-2,236,927 coefficient matrix) in 2 hours.

    But Blue Waters is planned to deliver a petaflop on applications that normally don't sustain >80% of theoretical peak; these applications are lucky to get near 20%.

  17. Re:How many human brains is that? by Surt · · Score: 2, Interesting

    2020 seems unlikely. A reasonably accurate real-time synaptic simulation can run maybe 100 neurons on a high end pc today, probably less. A human brain has about 100 billion neurons, so we're 1 billion times short in computation. Last time I checked, GPUs had not yet been used in neuron simulation, so I'll even give you that we may be 1000 times better off. That's still 1 million X improvement needed to match the brain, or roughly 20 more generations of computer hardware, at a generous 18 months, that leaves us at 30 years, 2038.

    I will be seriously surprised if an even vaguely accurate simulation of the human brain is running before 2050.

    --
    "Who is the Journal of Quantum Physics going to believe?" --Stephen Hawking
  18. Or by Colin+Smith · · Score: 2, Informative

    Simulating nuclear explosions.

     

    --
    Deleted