Slashdot Mirror


NNSA Supercomputer Breaks Computing Record

Lecutis writes "National Nuclear Security Administration (NNSA) Administrator Linton F. Brooks announced that on March 23, 2005, a supercomputer developed through the Advanced Simulation and Computing program for NNSAs Stockpile Stewardship efforts has performed 135.3 trillion floating point operations per second (teraFLOP/s) on the industry standard LINPACK benchmark, making it the fastest supercomputer in the world."

59 of 266 comments (clear)

  1. Neat by neccoant · · Score: 3, Interesting

    It's amazing that we were stalled at 50TFLOPS for two years, and are piling on the FLOPS now.

    1. Re:Neat by the_2nd_coming · · Score: 3, Insightful

      the increased flops is simply a function of the fact that they are expanding the number of nodes.

      --



      I am the Alpha and the Omega-3
    2. Re:Neat by Anonymous Coward · · Score: 3, Funny

      I think they were waiting for final specs on Doom3.

    3. Re:Neat by woah · · Score: 3, Interesting
      The reason is, of course, that we've been stuck with sameish desktop performance as well. Which correlates with supercomputer performance, since nowdays most of them use Intel/AMD processors.

      Just goes to show that Moore's law won't hold forever.

    4. Re:Neat by imsabbel · · Score: 3, Insightful

      You are misstaken.
      We didnt STALL at 30Gflops, its just that the 30Gflops were SO much better than everything else available that it took a couple of years to catch up and overtake it.

      If you average over the last 10 years, the the Earth simulator was a bump above moores law and now we are back on track.

      --
      HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
    5. Re:Neat by brsmith4 · · Score: 5, Informative

      That's not how linpack works. Sure, increasing your number of nodes will give definite performance advantages to course-grained, embarassingly parallel applications, but Linpack is not one of these applications. As well, Linpack should not be used as a guide for raw floating point performance, but is much better suited to gauge throughput.

      Linpack does its benchmarks using a more fine-grained algorithm, creating lots of communications for Message Passing to share segments of dense matrices for rather large linear systems. Not only is the number of nodes a factor, but so is the interconnect speed. If that cluster was using GigE for its interconnect, its Linpack benchmarks would not be nearly as impressive. Haven't RTFA but its likely that BlueGene/L is using Myranet or Infinband for its interconnect (or possibly a more proprietary backplane style interconnect, though that cluster is way too big for that).

      These latest generations of high-speed interconnects (esp. Infinband) have brought clusters closer to the point of being near shared-memory performance and hence is more of a throughput test than anything else.

      This description of the HPL benchmark (The "official" name for the Linpack benchmark) should provide some clarity as to how memory-dependent Linpack actually is:

      The algorithm used by HPL can be summarized by the following keywords: Two-dimensional block-cyclic data distribution - Right-looking variant of the LU factorization with row partial pivoting featuring multiple look-ahead depths - Recursive panel factorization with pivot search and column broadcast combined - Various virtual panel broadcast topologies - bandwidth reducing swap-broadcast algorithm - backward substitution with look-ahead of depth 1.

      http://www.netlib.org/benchmark/hpl/

      They took a lot of time to get Linpack to be less shared-memory dependent, like adding the swap-broadcast algorithm (which i'm fairly certain was absent in the old mainframe version of Linpack), to make it more "fair" to run on a cluster versus a shared memory set up. However, on a typical cluster, Linpack can push your interconnect pretty hard, esp. if you are stuck on GigE. However, Linpack has _lots_ of settings and parameters to "tune" the benchmark for your particular cluster.

      My point: Linpack/HPL is not an overall flops benchmark for a cluster. It measures the performance not only of double precision CPU performance, but also the performance of a cluster's interconnect.

    6. Re:Neat by JQuick · · Score: 2, Insightful

      Actually Intel compatible clusters in the supercomputer rankings are not all that compelling. True, linux cluster did fare very well for several years as measured by price/performance. Also it is true that about 63% of the top 500 supercomputers are Intel or Intel compatible.

      Despite this, the majority of systems at the top of supercomputer top 500 chart are based on the POWER architecture, not Intel chips.

      The POWER based systems, including BlueGene and PowerPC systems, are all much better on both price/performance, and Watt/performance basis. Intel chips do have a per chip advantage over PowerPC on many work-flows. However, when scaled, they directly consume more power, and indirectly require even more power to run higher capacity air conditioning.

      Looking at the top of chart reveals that Intel compatible systems are a small minority. In the top 10, as of November 2004, are one NEC, HP, SGI, Alpha, Xeon, and Itanium, and 5 members of the power family. IBM powers more than half of the top 100, with all other chip families dividing the remaining half.

      It is cheaper to build and operate a supercomputer cluster using either Apple or IBM gear (running either Macos X or Linux) than to do so using Linux on Intel these days.

    7. Re:Neat by imsabbel · · Score: 3, Interesting

      Well, in fact the truth is a right in the middle.
      Linpack is VERY easy to parallize. Earth simulator and other vector machines get over 85% of their theoretical processing power with linpack, and even clusters with relatively abyssmal interconnects are still in the 50% range.

      Lots of computational problems need orders of magnitutes more inter-node communication, up to the point where linpack doesnt even matter anymore and clusters and vector computers with the same linpack score are a factor of 10 or 20 apart.

      --
      HI O WISE PRINCE. WHT TOOK U SO DAM LONG?
    8. Re:Neat by kayak334 · · Score: 2, Informative

      Myranet or Infinband

      Just some minor corrections and informaton for those interested.

      Myricom is the company, Myrinet is the protocol. Infiniband is an open protocol. Myrinet has a maximum speed of 2.2Gb/sec while Infiniband can scale up to 30Gb/sec on a 16x PCI-E card and a 12x port on the switch.

      As for what BlueGene/L uses, I don't think I'm at liberty to discuss that.

    9. Re:Neat by brsmith4 · · Score: 2, Informative

      Were you correcting my spelling? Because I always make that mistake (myranet... it's myrinet damn it!). You know what I meant though ;) It looks like BlueGene/L is using a hybrid backplane/hypertorus interconnect where a whole bunch of "machines" (more like system-on-a-chip) are connected via a backplane, then that case of "machines" is connected to another case in the same rack on some number of layers of interconnect. Then the racks are connected using some other protocol. Though you may not "be at liberty" to discuss this, the top500 site already disclosed an ample amount of information on the subject for any beowulfer to get the general idea of what type of interconnect topology/setup BlueGene/L is using.

      And I quote:

      The nodes are interconnected through multiple complementary high-speed low-latency networks, including a 3D torus network and a combining tree network. The physical machine architecture is targeted to be most closely tied to the 3D torus, a simple 3-dimensional nearest neighbor interconnect which is "wrapped" at the edges. An independent combining tree network provides for fast global operations, such as global max or global sum.

      http://www.top500.org/sublist/System.php?TB=2&id=7 101

      Enjoy.

  2. and its only half the machine too! by rebelcool · · Score: 5, Informative

    wait till its fully online.

    --

    -

    1. Re:and its only half the machine too! by JPM+NICK · · Score: 2

      Scientists at LLNL for the first time have performed 16-million-atom molecular dynamics simulations with the highest accuracy inter-atomic potentials necessary to resolve the key physical effects to successfully model pressure induced rapid resolidification in Tantalum. As a shared resource, BlueGene/L has enabled scientists studying the effects of voids in metal failure at Los Alamos National Laboratory to perform standard accuracy molecular dynamics simulations with more than 2.1 billion atoms.

    2. Re:and its only half the machine too! by RicktheBrick · · Score: 4, Insightful

      It would seem to me that with the great accuracy and concrete penetrating weapons that we possess that we could establish a future date when the United States would destroy all of our nuclear weapons. I believe that we could do this unilateraly. I believe we could wipe out all the leadership of any country that dared to launch an attack on our country and hopefully we will be able to intecept their missile before they reach our cities. I do not believe that we can morally tell anyone not to possess nuclear weapons unless we dedicate ourself to a date when we do not possess them too.

    3. Re:and its only half the machine too! by Henry+V+.009 · · Score: 2, Insightful

      I do not believe that we can morally tell anyone not to possess nuclear weapons unless we dedicate ourself to a date when we do not possess them too.

      Then the solution is to immorally tell other nations that they can't have them. That way you have neither the stupidity of unilateral disarmament nor the stupidity of looking the other way on nuclear proliferation. I hope that this bit of Life 101 helps you out there.

    4. Re:and its only half the machine too! by Profane+MuthaFucka · · Score: 4, Funny

      Easy cowboy, before you start proclaiming that we could wipe out all the leadership of an arbitrary country, let's just get Osama first. OK?

      --
      Fascism trolls keeping me up every night. When I starts a preachin', he HITS ME WITH HIS REICH!
    5. Re:and its only half the machine too! by einhverfr · · Score: 2, Informative

      You have a point. Nuclear weapons were a heavily stabilizing force in the cold war because they made it impossible for any leader to consider all-out war with the other country. THings are more complex now, and nuclear proliferation is a different issue. Yet it is not a simple issue. On one hand, nuclear weapons continue to help prevent horrible war crimes like the firebombings of civilian population centers (Dresden, Tokyo) because it is simply too risky to do this. Yet they themselves are effective simply because they represent this risk.

      And the real risk is what happens if a group which is unbeholden to a public body, such as an international terrorist group, obtains such a device. They would be able to strike with one of these weapons but be immune to any counterattack.

      --

      LedgerSMB: Open source Accounting/ERP
    6. Re:and its only half the machine too! by InadequateCamel · · Score: 2, Insightful

      I get your point, but I think that to describe this in terms of police is a little inaccurate because that really is the police's job. Perhaps this is closer to a heavily-armed civilian militia (I hesitate to say "Mafia" or "gang", but you could argue points there too) taking on the police role?

  3. From the press release... by Zebra_X · · Score: 3, Informative

    This performance was achieved at Lawrence Livermore National Laboratory (LLNL) at only the half-system point of the IBM BlueGene/L installation. Last November, just one-quarter of BlueGene/L topped the TOP500 List of the world's top supercomputers.

    Is there anything that will be able to touch this when it's complete?

    1. Re:From the press release... by Daniel+Boisvert · · Score: 4, Interesting

      The closest I've heard of is the Cray X1E, but even that only claims 147 TFLOPS.

    2. Re:From the press release... by As+Seen+On+TV · · Score: 5, Informative

      The X1E isn't intended to be a fastest-in-the-world supercomputer. It's intended to be a low-cost scalable vector system. The fact that it's fast is great, but it's not its main design feature.

      Now, the X2, on the other hand, is a whale. They're talking 150 TFLOPS at roll-out next year (unimpressive) and 300 TFLOPS after the block 10 update the year after that (very impressive).

      Of course, the X2 isn't working yet, so who the hell knows. But it's fun to think about.

  4. Blue Gene? by eth8686 · · Score: 2, Informative

    Didn't IBM push Blue Gene to 180'something teraflops recently?? News story herer

    1. Re:Blue Gene? by EBorisch · · Score: 5, Informative

      This is Blue Gene. Read the article...

  5. imagine by dario_moreno · · Score: 3, Funny

    a Beowulf cluster of these !

    --
    Google passes Turing test : see my journal
  6. Wow! by FlyByPC · · Score: 5, Funny

    Just imagine running Fractint on this puppy!

    --
    Paleotechnologist and connoisseur of pretty shiny things.
    1. Re:Wow! by ucblockhead · · Score: 2, Interesting

      Heh. I guess I wasn't the only one who christianed a new machine by running fractint on it. Gave it up around 1998 because there was just no point.

      --
      The cake is a pie
  7. Steroids by tiktok · · Score: 4, Funny

    There was another machine that had already beaten that record, but unfortunately failed a diagnostic test for banned substances...

  8. Did you RTFA? by Donny+Smith · · Score: 5, Informative

    > has performed 135.3 trillion floating point operations per second (teraFLOP/s) on the industry standard LINPACK benchmark, making it the fastest supercomputer in the world."

    Did you read the fucking article?

    "This performance was achieved at Lawrence Livermore National Laboratory (LLNL) at only the half-system point of the IBM BlueGene/L installation. Last November, just one-quarter of BlueGene/L topped the TOP500 List of the world's top supercomputers."

    See, this is the SAME supercomputer that has already topped the list last November, so the latest record did NOT make it the fastest supercomputer in the world.

    It already had been the fastest supercomputer in the world.

  9. Wow. by TsukasaZero · · Score: 2, Funny

    Slap an X850 in there and you've got some serious Doom 3 action.

  10. Yeah, but... by Black+Jack+Hyde · · Score: 3, Funny

    ...will it run NetHack?

  11. Re:hmmmmm... by a1cypher · · Score: 5, Interesting

    Just for a point of reference, does anybody know how many floating point operations a 3.2ghz processor can do per seccond?

    I know its not 3.2billion because most micro operations take at least 3 or 4 clock cycles.

  12. Earth Simulator by Anonymous Coward · · Score: 3, Insightful

    I rather miss the time when the world's most
    powerful supercomputer was used to study our
    planet. It was something to be proud of, actually.
    These machines are essentially weapons. Pity, that.

    1. Re:Earth Simulator by lp-habu · · Score: 2, Insightful

      Historically, I'll think you'll find that a great many technological advances were made with the original purpose of killing other beings -- usually other humans. Seems to be one of the basic human characteristics. Pretty effective, too.

  13. Link to the list by dnaboy · · Score: 4, Informative

    FYI the top 500 supercomputers list is maintained at http://www.top500.org/.

  14. Re:hmmmmm... by Yartrebo · · Score: 3, Informative

    With SSE instructions, you can process 4 floats at once, so I'm guessing that 3.2 GHz processor can do a few gigaflops.

  15. You are not seeing the larger picture by ravenspear · · Score: 2, Insightful

    This is the same flawed logic that people try to apply to NASA on here a lot, that we shouldn't send people to the Moon/Mars until we fix all the problems on earth.

    Halting scientific research to worry about all of our other problems is the wrong approach for many reasons. It is often scientific advances which lead to improved quality of life in many other areas of society.

  16. Dupe by karvind · · Score: 3, Informative

    Didn't we cover this before ?

  17. LINPACK usage? by Gleepy · · Score: 2, Interesting

    I think of LAPACK as being much more up-to-date for benchmarking.

    --
    Gleepy the Hen. More intelligent than the average hen.
  18. This *is* Blue gene. by daveschroeder · · Score: 2, Informative

    RTFA

    Or, at least the article's title:

    "NNSA Supercomputer Breaks Computing Record: Exceeds 100 TERAFLOPS DOE/NNSA and IBM partnership on BlueGene/L, a tool for national security"

  19. Human Intelligence? by kyle90 · · Score: 2, Interesting

    Isn't the human brain supposed to be equivalent to a supercomputer running at about ~100 teraflops? And if so, shouldn't this computer be smarter than us?

    --
    Real_men_don't_need_spacebars.
    1. Re:Human Intelligence? by Kethinov · · Score: 4, Interesting
      Isn't the human brain supposed to be equivalent to a supercomputer running at about ~100 teraflops? And if so, shouldn't this computer be smarter than us?
      In Star Trek TNG 2x09 Data was quoted at having a total memory capacity of somewhere around 90 petabytes with a total linear computational speed of 60 trillian operations per second.

      One would say this supercomputer is already more than twice as smart as Data!
      --
      You're right, I wouldn't steal a car. But if it were possible, I sure as hell would download one!
    2. Re:Human Intelligence? by trippy · · Score: 2, Funny

      Maybe so, but this thing is HUGE compared to Data. If Data were as large as this, the Enterprise would need a trailer to lug around his fat ass.

    3. Re:Human Intelligence? by myukew · · Score: 2, Informative

      No. The human brain has about 10^11 neurons, each with about 1000 connections to other neurons. Every neuron can fire about 200 times a second. So scientists expect the human brain to have about 20 PFlop/s. Still a little faster than blue gene...

  20. Re:More important issues by tgamblin · · Score: 5, Informative
    Despite the fact that BlueGene/L is being built to simulate nukes, this kind of research does impact some of these other issues, and there is government money going into them. Here are some examples... The National Center for Atmospheric Research uses supercomputers to simulate effects of pollution and global warming, and projects like LEAD are using grids with supercomputers attached to predict weather. Check out some of the projects at RENCI, as well. There's NIH-sponsored genetic research in addition to the weather stuff.

    It may be sad that we live in a world where nuclear weapons research is driving the computing power, but it doesn't mean that the power of BlueGene/L isn't going to be used for thousands of other peaceful scientific applications, too.

  21. Re:hmmmmm... by tgamblin · · Score: 5, Informative

    Depends on the problem and the memory performance as much as it does on the GPU. There's no good answer to that question. For kicks though, this paper has some measurements for matrix multiply using ATLAS. It's comparing a Pentium 4 to an NV40 GPU. The P4 wins at about 7 GFlops, and the NV40 loses due to horrible memory performance. That's pretty ironic considering that the NV40 has quite a few more FPU's, and that they're in parallel. It's a good example of why you can't ever say for sure how a processor's going to perform until you test it on a real workload.

  22. Re:hmmmmm... by Swedentom · · Score: 2, Informative

    An Apple Xserve G5 does 30+ gigaflops.

    --
    Sig Nature
  23. Re:AMazing by mikael · · Score: 2, Insightful

    Or we can model protein folding even faster.
    Run more accurate climate simulations even faster.
    Run population simulations even faster.
    Run CAD/CAM simulations even faster.

    --
    Vintage computer adverts: http://www.vintageadbrowser.com/computers-and-software-ads
  24. Re:hmmmmm... by fafalone · · Score: 2, Informative

    A 3.2GHz Intel Xeon processor performs 6.4gflops, but clock speed isn't the only determining factor.

  25. Human Intelligence is More than Speed by Ted+Holmes · · Score: 2, Interesting
    One of the landmarks we needed to pass in order for computers to approximate Human intelligence is the processing speed.

    Estimates are that the Human brain computes somewhere between 100 Teraflops and 1000 Teraflops,
    and Google was performing somewhere between 100 and 300 Teraflops. in late 2004.

    P.S. Since doing that bit of research, every time Google checks my spelling and responds with "did you mean..." the hair stands on the back of my neck :)

    But it's more than processing speed. It needs to have the software to do things like decision making, analysis, reasoning, evaluating, judging, information-organizing, learning, logic etc. which would normally require a human to perform.

    We're not far off though...

  26. More Computer Power=Fewer Nuclear Explosions by Nova+Express · · Score: 4, Insightful
    To study the effects of different nuclear weapon designs, there are basically two approaches:

    1. Throw massive amounts of computing power at the problem (as done here), or:
    2. Actually set off a nuclear weapon.

    Having massive computing power in the hands of Lawrence Livermore scientists reduces or even eliminates the need for U.S. nuclear forces to actually detonate nuclear and thermonuclear explosions.

    Of course, some people would prefer to see the United States undertake unilateral nuclear disarmament, something they've been advocating since SANE/FREEZE was telling us we could trust the Soviet Union in the 1980s. Only today they claim we can trust Kim Il Jong and the mullahs of Iran more than the democratically elected government of the United States, just as they claimed we could trust Leonid Breshnev and Yuri Andropov more than we could trust Ronald Reagan. Their views are every bit as ill-conceived now as they were then.

    --
    Lawrence Person (lawrencepersonh@gmailh.com (remove all "h"s to mail)

    http://www.lawrenceperson.com/

    1. Re:More Computer Power=Fewer Nuclear Explosions by ozborn · · Score: 3, Insightful

      Of course, some people would prefer to see the United States undertake unilateral nuclear disarmament, something they've been advocating since SANE/FREEZE was telling us we could trust the Soviet Union in the 1980s. Only today they claim we can trust Kim Il Jong and the mullahs of Iran more than the democratically elected government of the United States, just as they claimed we could trust Leonid Breshnev and Yuri Andropov more than we could trust Ronald Reagan. Their views are every bit as ill-conceived now as they were then.
      Nice strawman you've constructed, but pray tell who are these "some people" you are talking about? I challenge you to cite a single press release, webpage or publication by any independent NGO (even kooky ones) pushing for nuclear disarmanment that claims Kong Il Jong can be trusted. I can't think of any disarmament/peace group that would be opposed to 3rd party bilateral weapons inspections.

  27. DOE's Senior Activity Center by Animats · · Score: 2, Interesting
    The "stockpile stewardship program" is basically a senior activity center for retired physicists. They have busywork projects to keep people thinking about how to design nuclear weapons. DOE is worried that all the old bomb designers will die off, and no new ones will replace them.

    Remember, everything in the inventory was designed with far less compute power than today's desktops.

    1. Re:DOE's Senior Activity Center by T5 · · Score: 2, Informative

      DOE's stewardship program is not for retired scientists, but current ones. The laboratory directors at the nuclear labs (Sandia/LLNL/maybe others) are required to certify the stockpile as being ready to go each year. Their supercomputers are the only way to test the aging stockpile without actually detonating a few to see which designs age better than others.

      And let's remember that almost everything in the current arsenal was designed and actually tested, not just worked up via computer. It takes a whole lot more computing power to run the thermodynamic and nuclear codes for simulation than it does to validate designs.

  28. Re:AMazing by brsmith4 · · Score: 2, Informative

    A slightly larger dose of logic would tell you that NASA has nothing to do with this cluster, that it belongs to the NNSA or the National Nuclear Security Agency. They are probably more interested in testing new reactor designes or running simulations to demonstrate the effects of an aircraft crashing into one of their reactor domes (though I honestly believe that no one really believes that will happen).

  29. Can we qualify this a bit? by Moderation+abuser · · Score: 4, Insightful

    "making it the fastest supercomputer in the world"

    Or rather the fastest supercomputer with published LINPACK results. There are a number of reasons that agencies with supercomputers might not want to publish results.

    --
    Government of the people, by corporate executives, for corporate profits.
  30. 135.3 trillion floating point operations per seco by alex4u2nv · · Score: 4, Funny

    135.3 trillion floating point operations per second

    Does this mean we can't slashdot it?

  31. You of course know what this means. by Allnighterking · · Score: 2, Funny

    You can now open a Mozilla session in under a minute!

    --

    I'm sorry, I'm to tired to be witty at the moment so this message will have to do.

  32. OK then... by caveat · · Score: 2, Insightful

    How 'bout we use Blue Gene for climate modeling, and start setting off full-yield nuclear tests to insure the viability of the stockpile? I don't terribly like the idea of nukes, but the genie is out of the bottle and there's no stuffing it back in - we need to have the things, and if god forbid we ever have to use them, I'd like to see them work properly. Seriously...unless you use one of the interconnect cables to garrote somebody, these computers are hardly "weapons", quite the opposite in fact.

    --

    Facts do not cease to exist because they are ignored. - Aldous Huxley
  33. The belly of the beast by theufo · · Score: 3, Insightful

    Here's an article describing some of the specs.

    http://www.llnl.gov/asci/platforms/bluegene/talks/ gupta.pdf

    It's from the days when BlueGene/L was still relatively small, but the basic design hasn't changed since then.

    Turns out it's split into I/O and computing nodes. The 1024 I/O nodes run Linux. Each controls 64 dual-cpu nodes, which use simplistic microkernels written from scratch using Linux as an example.

    The network architecture sounds funky: apparantly it's based on a torus!

  34. You Gotta Love This... by Nom+du+Keyboard · · Score: 2, Funny
    Scientists at LLNL for the first time have performed 16-million-atom molecular dynamics simulations with the highest accuracy inter-atomic potentials necessary to resolve the key physical effects to successfully model pressure induced rapid resolidification in Tantalum.

    You just gotta love a sentence like that!

    --
    "It's the height of ridiculousness to say for those 9 lines you get hundreds of millions."