Slashdot Mirror

← Back to Stories (view on slashdot.org)

LLNL/RPI Supercomputer Smashes Simulation Speed Record

Posted by timothy on from the in-a-world-of-make-believe dept.

Lank writes "A team of computer scientists from Lawrence Livermore National Laboratory and Rensselaer Polytechnic Institute have managed to coordinate nearly 2 million cores to achieve a blistering 504 billion events per second, over 40 times faster than the previous record. This result was achieved on Sequoia, a 120-rack IBM Blue Gene/Q normally used to run classified nuclear simulations. Note: I am a co-author of the coming paper to appear in PADS 2013."

79 comments

  1. rPi is different from RPI by stewsters · · Score: 5, Funny

    Was i the only one who thought for a second that this was about a raspberry pi cluster?

    1. Re:rPi is different from RPI by DrData99 · · Score: 2, Funny

      Yes.

    2. Re:rPi is different from RPI by Anonymous Coward · · Score: 2

      No

    3. Re:rPi is different from RPI by TheAngryMob · · Score: 2

      No. For us old-timers, RPI stands for Rockwell Protocol Interface.

      http://encyclopedia2.thefreedictionary.com/Rockwell+Protocol+Interface

      POS Modems....

      --

      Don't just game, Dungeoneer
    4. Re:rPi is different from RPI by Narcocide · · Score: 1

      Slashdot's comment filter kept me from responding "No"

    5. Re:rPi is different from RPI by Capt.DrumkenBum · · Score: 1

      Another vote for Raspberry Pi.
      More than a little disappointed. :( Where is my Credit crad? I think I need to order some Raspberries.

      --
      If I were God, wouldn't I protect my churches from acts of me?
    6. Re:rPi is different from RPI by Orne · · Score: 1

      I guess you have a different definition of old-timers, since it was founded in 1824 and became an official Institute in 1832.

    7. Re:rPi is different from RPI by Anonymous Coward · · Score: 0

      They didn't have acronyms back in the 19th century, silly.

    8. Re:rPi is different from RPI by RabidReindeer · · Score: 2

      They didn't have acronyms back in the 19th century, silly.

      I always thought that acronyms were invented by IBM.

      They used so many of them that the same 3 letters often applied to 5 different products. At the same time.

    9. Re:rPi is different from RPI by ArcadeMan · · Score: 1

      No you weren't, although I was wondering how many millions of rPI they needed to achieve this.

      --
      Get free satoshi (Bitcoin) and Dogecoins
    10. Re:rPi is different from RPI by Anonymous Coward · · Score: 0

      And it invented the @ symbol! You tribute RPI everytime you send an email!

    11. Re:rPi is different from RPI by Anonymous Coward · · Score: 0

      Was i the only one who thought for a second that this was about a raspberry pi cluster?

      No, like you I also thought about the Raspberry Pi, but then I took a misleading-article-title to the knee...

      meh this joke get's old fast, should have made a joke about a beowulf cluster of RPi's instead

    12. Re:rPi is different from RPI by maxwell+demon · · Score: 1

      They didn't have acronyms back in the 19th century, silly.

      I always thought that acronyms were invented by IBM.

      They used so many of them that the same 3 letters often applied to 5 different products. At the same time.

      See? They already had quantum computing!

      --
      The Tao of math: The numbers you can count are not the real numbers.
  2. Only Warp 2.7? by maxwell+demon · · Score: 2

    I was already running Warp 3 in 1995! :-)
    (OS/2 Warp 3, to be exact)

    --
    The Tao of math: The numbers you can count are not the real numbers.
    1. Re:Only Warp 2.7? by olsmeister · · Score: 2

      At present, we are now at {Warp Speed 2.7}. It will be nearly 150 years before we expect to reach {Warp Speed} 10.0.

      And then, Delta Quadrant here we come!

  3. Simulation of what? by Anonymous Coward · · Score: 0

    So new super computer managed to "achieve a blistering 504 billion events per second". All the summery says is the computer normally does "classified nuclear simulations". So These events are what? What is is simulating?

    1. Re:Simulation of what? by Bill,+Shooter+of+Bul · · Score: 4, Funny

      No, those events are Who. Simulating is How. What is calculated.

      --
      Well.. maybe. Or Maybe not. But Definitely not sort of.
    2. Re:Simulation of what? by flayzernax · · Score: 3, Funny

      Cats.

    3. Re:Simulation of what? by maxwell+demon · · Score: 1

      The computer is performing a musical?

      --
      The Tao of math: The numbers you can count are not the real numbers.
    4. Re:Simulation of what? by flayzernax · · Score: 1

      Cats in space =)

    5. Re:Simulation of what? by jameshofo · · Score: 1

      A liberal coming to terms with building a new nuclear power plant in the US

      --
      Good leaders run toward problems, bad leaders hide from them.
    6. Re:Simulation of what? by fast+turtle · · Score: 1

      Miss Piggy Does Not Like. It's Pigs in Space. Hyyuhh!

      --
      Mod me up/Mod me down: I wont frown as I've no crown
    7. Re:Simulation of what? by icebike · · Score: 1

      So new super computer managed to "achieve a blistering 504 billion events per second". All the summery says is the computer normally does "classified nuclear simulations". So These events are what? What is is simulating?

      The summary said the "coordinated 2 million cores", without saying where they were, or why they needed two labs from opposite sides of the country to do so. The summary seems long on self promotion and short on details if you ask me.

      --
      Sig Battery depleted. Reverting to safe mode.
    8. Re: Simulation of what? by DJefferson · · Score: 3, Informative

      The simulation was a well-known parallel discrete event benchmark called PHold. It is not a model of any particular physical system, but is more of a stress and scalability test for the simulator, in this case the ROSS simulator developed at RPI. PHold has particularly fine-grained events, which stresses the synchronization mechanism known as Time Warp, implemented ROSS with support for reverse computation. It stresses the scalability of the Global Virtual Time commitment mechanism (used for I/O, error detection, storage management, and termination detection). And because PHold has no locality in its communication, it greatly stresses the underlying communication layer, MPI. The general idea is that a simulator that can achieve high performance on PHold at very large parallel scale can achieve high performance on just about any realistic, load balanced discrete event simulation at that scale.

    9. Re:Simulation of what? by RoverDaddy · · Score: 1

      Third base.

      --
      RETURN without GOSUB in line 1050
  4. The Bomb by femtobyte · · Score: 0

    Now, if only we found better uses for top supercomputers than assuring our WMD supply is always in tip-top shape for mass murder.

    1. Re: The Bomb by Anonymous Coward · · Score: 1

      It let's us get by with fewer bombs. Fewer bombs means less chances for mistakes. And I am very glad that the superpower states cannot fight each other directly because indirect war is bad enough.

    2. Re: The Bomb by femtobyte · · Score: 1

      The US military budget is about as much as the next 10 biggest national military budgets *combined.* The US isn't one player in a delicate balance of superpowers; it is a massive unilateral force, driven by greed and paranoia to utterly irrational levels of military spending. No matter how much the US has, war hawks clamor for more. "Fewer bombs" is a sick joke in the context of the ridiculous number of bombs the US has. Scrap 90% of our military, and we'd still be an untouchable superpower.

    3. Re: The Bomb by Anonymous Coward · · Score: 1

      While that is true (or true-ish; there is no reason to believe the PRC's public budget numbers), the US spends its defense money so inefficiently that it doesn't have as much strength as the next ten national militarizes combined, or anywhere close to that.

    4. Re:The Bomb by fiordhraoi · · Score: 1

      Maybe we could program them to scour the internet for comments that take a completely unrelated tangent to the original article, for the purposes of expounding on whatever irrelevant social agenda/issue the commentator has stuck up his behind? And then it could automatically delete them.

    5. Re: The Bomb by jameshofo · · Score: 1

      Its just that our stupidity is smarter than theirs!

      --
      Good leaders run toward problems, bad leaders hide from them.
    6. Re: The Bomb by RabidReindeer · · Score: 1

      The US military budget is about as much as the next 10 biggest national military budgets *combined.* The US isn't one player in a delicate balance of superpowers; it is a massive unilateral force, driven by greed and paranoia to utterly irrational levels of military spending. No matter how much the US has, war hawks clamor for more. "Fewer bombs" is a sick joke in the context of the ridiculous number of bombs the US has. Scrap 90% of our military, and we'd still be an untouchable superpower.

      I think that the sad truth is that the primary purpose of the military budget is to serve as a welfare program whereby congresscritters can hand out jobs to their constituents and pretend that it's not "wasteful gummint spending" because it's FREEDOM, DAMMIT!

      An awful lot of money gets spent on horribly expensive military toys that the Pentagon claims not to want or need just because someone in Congress could get facilities opened back home to make and/or service them. You could replace quite a few bridges - and the Interstate highways connecting them - for the price of a single Osprey, if I have my numbers right.

    7. Re: The Bomb by Anonymous Coward · · Score: 0

      The U.S. and Europe control way more than half of the worlds 70 Trillion dollar a year economic output/ money flow. It is shared wealth. Efficiency has nothing to do with it. When you have controlling interest, you CONTROL it.
      There are perhaps 2,000 people on "the list"
      Carlos Slim, and Warren Buffett are not on the list. (They do get invited to parties).
      The President of the United States is not on "the list". The President of the United States does what he is told.
      This is people like Barrack Obama, and George Bush we are talking about, so save you outrage ok?
      George Senior originally wasn't on the list, but he saw the list. So he was able to get himself and his whole family powerful positions in government.
      fall of Rome to French revolution to American revolution. The controllers have nowhere left to run. You can have enough money. You can never have enough bombs.

    8. Re:The Bomb by Anonymous Coward · · Score: 0

      I'm with you. It seems every few months livermore or some other US military institute is "setting new records" for speed used by computers for "classified" work. Wonder what it costs to keep replacing these machines with faster ones (at taxpayer expense), and when the race will end? The fact its working on "classified" work but yet the computer, its location and capabilities are published so proudly is not lost on me.

    9. Re: The Bomb by bedeutungslos · · Score: 1

      We came to this state by basically unilaterally assuming responsibility for the defense of Europe. It was complicated, and had to do with the question of German rearmament after WWII (and, in no small measure, Vietnam before the US was really involved). But the short answer for why the US spends so much on defense is that we have chosen to carry all our allies. Could Taiwan support ten carriers at sea? No... but will they need ten carriers to wage an effective campaign if the day ever comes? Yes. And we have chosen to commit to providing them. It's not like we don't get anything from Taiwan in return. Or ROK... or back in the day, FRG and the rest of Western Europe. Part of all that is maintaining a capable nuclear arsenal, and we use computer simulations rather than live tests to assure the efficacy of our stockpile. I think that's a perfectly good reason to buy lots of computers and employ lots of scientists. I guess you've got some specific, well-funded, noble alternative to retask all these resources on?

    10. Re: The Bomb by femtobyte · · Score: 1

      I guess you've got some specific, well-funded, noble alternative to retask all these resources on?

      Well, nuclear anti-proliferation is a pretty nice start, which just requires enough funding to yank the plug out of the wall. But, since you've already presumably got funding for operation and research personnel for the bomb-maintenance tasks, you could just re-task that along with the computer (not designing bombs is a good start in itself). I don't currently have any personal pet projects that need a supercomputer, but perhaps I could refer you to the poster "aussie.virologist" further down the thread noting this could be handy for viral simulations. Researching new antiviral drugs (and releasing the results free to the world so anyone can manufacture them) seems like a pretty "noble alternative" to assuring we can initiate global nuclear holocaust at a minute's notice.

    11. Re: The Bomb by Anonymous Coward · · Score: 0

      Nuclear weapons are not paid for out of the DOD budget. They are a DOE funded.

    12. Re:The Bomb by nateb · · Score: 1

      Commentator is not a word. HTH. HAND.

      --
      -- Nate
    13. Re:The Bomb by ratbag · · Score: 1

      From Oxford Dictionary of English:

      commentator |ËkÉ'mÉ(TM)nteÉtÉ(TM)|
      noun
      a person who comments on events or on a text.
      â a person who commentates on a sports match or other event.

      Commenter may have been more appropriate in the circumstances, I'll grant you.

    14. Re: The Bomb by tehcyder · · Score: 1
      Spooky factoid: "the list" is an anagram of "illuminati".

      Laugh all you like, the guy's onto something.

      --
      To have a right to do a thing is not at all the same as to be right in doing it
    15. Re:The Bomb by MachineShedFred · · Score: 1

      You mean, like assuring our WMD supply is always in tip-top shape for deterring mass murder?

      Because that's what nukes have been doing for the last 60 years.

      --
      Slashdot still doesnâ(TM)t support Unicode after it was added to the HTML standard in 1997.
  5. can you put the paper online? by Trepidity · · Score: 5, Insightful

    Note: I am a co-author of the coming paper to appear in PADS 2013.

    I clicked hoping to read the paper, but the actual paper doesn't seem to be posted, only the abstract. The ACM copyright policy explicitly allows authors to "Post the Accepted Version of the Work on ... the Author's home page", so there is no legal barrier to the authors putting a PDF online. Doing so would of course increase readership of the paper, so ought to benefit everyone.

    --
    10 PRINT CHR$(205.5+RND(1)); : GOTO 10
    1. Re:can you put the paper online? by Lank · · Score: 5, Informative

      I didn't realize that it was acceptable to post it before the conference even happened. But you're right so here it is.

      --
      Gotta get me one of these!
    2. Re:can you put the paper online? by Trepidity · · Score: 2

      Thanks! My own policy is that I don't post draft or submitted versions, but once something is finalized (camera-ready final copy as it's going to appear in the proceedings), I'll post the PDF online.

      One plus side for those who care about such things is that it'll get into Google Scholar faster—GS is surprisingly good at picking these PDFs up in its crawls and figuring out how to index them.

      --
      10 PRINT CHR$(205.5+RND(1)); : GOTO 10
    3. Re:can you put the paper online? by Anonymous Coward · · Score: 0

      Near top of second column: "Renssleaer"

    4. Re:can you put the paper online? by Darth+Snowshoe · · Score: 1

      Knowledge and Thoroughness, yo.

  6. keys still safe from brute force by Anonymous Coward · · Score: 0

    So that is 2**39 keys per second. To brute force an 80 bit key (full key space) is 2 ** 41 seconds. That is something like 64000 years.

    1. Re:keys still safe from brute force by SuricouRaven · · Score: 2

      This is a simulation, events. The summary doesn't say what the events are, but probably more complicated than just testing a key.

      Besides, brute-forcing a key wouldn't be best done on general-purpose or even GPU. An ASIC would be the fastest, and you can be confident such chips would be easily within the capability of any major and a lot of not-to-major governments. So you're looking at a chip that can do, as a back-of-the-envelope, a key every cycle and clocked at 1.2GHz - standard for a lot of systems, as a sort of performance-per-watt peak. Times 64 cores per chip, times eight chips per PCI-e card, times eight processor cards per 2U case, times 42/3=14 systems per rack (leave space for cooling and switch), that's 1.2 * 64 * 8 * 8 * 14 = 68812 GK/s per rack.

    2. Re:keys still safe from brute force by Anonymous Coward · · Score: 1

      This article is about discrete event simulation, not something as embarrassingly easy to paralleled as brute-forcing a block cipher.

    3. Re:keys still safe from brute force by Anonymous Coward · · Score: 1

      ,that's 1.2 * 64 * 8 * 8 * 14 = 68812 GK/s per rack.

      That gives you about 2**40 keys/(rack*second)
      If you bought 1000 racks (of this highly specialized hardware) you are still looking at a 32 year wait.
      I think that the keys are probably still safe.

    4. Re:keys still safe from brute force by Anonymous Coward · · Score: 0

      I think that the keys are probably still safe.

      If this was the government you were talking about, I'd agree, the government can't do anything right.
      This is not the government. This is the NSA. If they need to, they will buy 100,000 racks.
      Bluffdale isn't being built to study the weather. (well, no supercomputer is, but that's another conspiracy theory)
      Your keys are not safe.

  7. Fast money by Anonymous Coward · · Score: 1

    I wonder how much money you could make mining bitcoins on that for one minute.

    'You earn $400,000 by using this computer for 12 seconds.'

    1. Re:Fast money by burning-toast · · Score: 1

      And then at the end of the month you get an electric (+cooling / water) bill for $400,000. Doh!

      - Toast

    2. Re:Fast money by jkflying · · Score: 1

      Yes, but in the mean time you earn the interest on the $400k. Just make sure you mine on the first day of the month and pay the bill on the last.

      --
      Help I am stuck in a signature factory!
    3. Re:Fast money by tehcyder · · Score: 2

      It's good to see that you've thought this through properly.

      --
      To have a right to do a thing is not at all the same as to be right in doing it
  8. LLNL Supercomputer, not RPI by 1729 · · Score: 1

    Headline is incorrect: Sequoia is at LLNL, not RPI.

    1. Re:LLNL Supercomputer, not RPI by 1729 · · Score: 1

      And now the headline has been updated to "LLNL/RPI Supercomputer...", which is STILL INCORRECT. Sequoia is a DOE computer at LLNL:

      http://en.wikipedia.org/wiki/IBM_Sequoia

  9. This could be good... by aussie.virologist · · Score: 4, Interesting

    I'd be interested in seeing if this system could run our full Poliovirus simulations (consisting of around 3.5 million atoms). I've run our simulations on the BlueGene/Q at VLSCI using 32,768 cores (65,536 threads) and have been getting a very respectable 11.2 nanoseconds per day of simulation data using NAMD. Some data on our full virus simulations can be found here... (VIDRL supercomputer simulation page). Hey Lank, maybe you can help me figure out a way to crack the millisecond mark for our full-virus sims??? Great work and cheers from down under :-)

    1. Re:This could be good... by DJefferson · · Score: 2

      I would think that the macroscopic behavior of 3.5 million atoms in (poly)crystals or in a fluid or plasma states are within the capability of Sequoia. That's about 2 atoms per core and per GB of RAM. But the complex dynamics of proteins, DNA, RNA, and any other complex polymers that comprise the polio virus interacting with, say, a cell membrane, are still probably out of reach for accurate calculation in a reasonable amount of time.

    2. Re:This could be good... by aussie.virologist · · Score: 2

      Agreed, at this point we are looking at virus dynamics in response to drug binding events and gross alterations in conformational structure in response to significant changes in temperature and ionic content. So for these simulations, the longer the better. I dream of a day when we can model complex host cell interactions and hopefully I will a grey bearded old man still full of enthusiasm when these sort of simulations are considered "run of the mill". Your work helps to keep me excited about the future of HPC and how it can benefit not only my research, but humanity's understanding of the world as a whole. Cheers.

    3. Re:This could be good... by ratbag · · Score: 1

      At the risk of getting all mushy and sentimental - thank you aussie.virologist, and your ilk, for doing something worthwhile with all these processor cycles available to the world.

    4. Re:This could be good... by jkflying · · Score: 1

      I find this topic extremely interesting, and it is a field I could see myself getting involved in, however my background is undergrad elec/mech with my MSc. in robotics/mapping/AI. I've also done a ton of simulation work via Robocode. What kind of background topics would I need to still learn to do this kind of work? I'm guessing quantum physics and chemistry along with some more hardcore comp-sci.

      --
      Help I am stuck in a signature factory!
    5. Re:This could be good... by aussie.virologist · · Score: 2

      Hey thanks "ratbag" for your kind words. The work that Barnes et al. are doing is so important for researchers like us. It opens the door for us to answer questions in a manner that even 5 years ago was considered "ambitious" to say the least. I am very lucky to be in a position where I have access to resources that allow me to explore new ways of answering some very old questions about how viruses behave, with the added bonus that we may hopefully be able to contribute to making the world just a little bit better. Fingers-crossed.

      "jkflying" I started off by working in electronics engineering when I left school, funnily enough I was running a company with some friends designing and building robotics systems, mainly focusing in animatronics. I wanted to start using my robotics background to work in the development of prosthetic limbs, but ended up changing the focus of my undergrad from anatomy and physiology to pathology, specifically microbiology with a lot of biochemistry thrown in. My post-grad was in computational biology. I actually started doing the simulation work after playing around with the tutorials on the VMD/NAMD website at the University of Illinois. I would recommend doing them, it's great nerdy fun and it gets you thinking about the different ways that you apply the techniques.

      Have a great day:)

  10. So? by Anonymous Coward · · Score: 0

    How well does quake run on it?

  11. Finally a hope for RPI students by Anonymous Coward · · Score: 0

    Now we male RPI students can finally have a chance at deciphering the algorithm of how to attract women!

  12. So how many bitcoins could it mine? by Anonymous Coward · · Score: 0

    Cool, how many bitcoins do you think this thing could mine in a day?

    1. Re:So how many bitcoins could it mine? by Freddybear · · Score: 1

      Not enough to pay the electric bill.

  13. Not that impressive - just running a benchmark by Anonymous Coward · · Score: 1

    This experiment didn't perform any useful computation - they just ran PHOLD, a benchmark that sends messages between nodes in a random pattern. It's a benchmark that's specifically tailored to perform well with the Time Warp synchronization algorithm for parallel discrete event simulation. Although Time Warp performs great in theory, it relies on rolling back program state when it detects a synchronization error, and is notoriously difficult to implement in practice for large simulations.

    Furthermore, these big machines are going to be used mostly for continuous (i.e. physics) simulations; this test was a discrete event simulation. Keep your eyes peeled for the winner of the Bellman-Ford prize, which is awarded to the highest sustained supercomputer throughput *when working on a real science problem*.

    1. Re: Not that impressive - just running a benchmark by DJefferson · · Score: 2

      I have to disagree. PHold was not designed to run well under Time Warp. It was designed as a stress test for any parallel discrete event simulator, whether based on Time Warp or not, and in particular originally to compare optimistic to conservative synchronization algorithms. Also, Sequoia is much less biased toward regular geometry continuum simulations that other world class supercomputers. It has no GPUs, for example. Machines of this class will be used more and more in the future for discrete simulations such as network models, or agent-based models, or for huge data problems, or for mixed continuous-discrete models such as of the power grid.

  14. what OS please? by softcoder · · Score: 1

    and the operating system it runs is?

    1. Re:what OS please? by DJefferson · · Score: 4, Informative

      It runs a custom IBM OS specifically designed for Blue Gene/Q. It proveds an API very similar to Linux, but with some restrictions, e.g. static limits on threads, no process forking, and custom MPI messaging instead of a TCP/IP stack.

  15. It was an LLNL supercomputer, not an RPI supercomp by DJefferson · · Score: 4, Informative

    The title to this piece is wrong. The supercomputer in question was Sequoia, the Blue Gene/Q supercomputer located at Lawrence Livermore National Laboratory. Some preliminary work was done on a smaller RPI BG/Q machine, however. (I am a coauthor of the paper.)

  16. Subsidy by Anonymous Coward · · Score: 0

    They've been doing the same thing for decades, looking after the same nuclear stockpile, it hasn't changed, their job hasn't changed. They don't simulate anything new that needs a supercomputer of that power, because they've been doing it with far less powerful computers for decades.

    This is just a subsidy to the US computer industry, disguised as 'classified' work.

  17. John... by FireXtol · · Score: 1

    Gustafson would be proud.

    --
    Enlightenment is the elimination of that which is unnecessary.
  18. +++ATH0 by dutchwhizzman · · Score: 1

    +++ATH0 +++ATH0 +++ATH0 If you're still here, you're not on an imitation Rockwell modem.

    --
    I was promised a flying car. Where is my flying car?
    1. Re: +++ATH0 by madprof · · Score: 1

      This isn't funny and it doesn't work like that. I would say try again, but please don't bother.

  19. You must be new too slashdot! by Anonymous Coward · · Score: 0

    Every summary that has any thing to do with IBM and it Power Architecture making some type of positive mile-stone will be misleading and ambiguous. The only times that they will provide more specific information about IBM and it Power architecture is when an x86 HPC builder such as cray, dell, or hp reaches some benchmark that was set by the Power architecture. Note: no mention of the processor be used in the Blue Gene/Q is the 18 core 64-bit Power A2 which is manufactured using a 32nm process while consuming only about 50-watts of power.

  20. Go Rensselaer! by Anonymous Coward · · Score: 0

    makes me proud to be a graduate of RPI :)