Slashdot Mirror

← Back to Stories (view on slashdot.org)

Titan Supercomputer Debuts for Open Scientific Research

Posted by samzenpus on from the greased-lightning dept.

hypnosec writes "The Oak Ridge National Laboratory has unveiled a new supercomputer – Titan, which it claims is the world's most powerful supercomputer, capable of 20 petaflops of performance. The Cray XK7 supercomputer contains a total of 18,688 nodes and each node is based on a 16-core AMD Opteron 6274 processor and a Nvidia Tesla K20 Graphical Processing Unit (GPU). To be used for researching climate change and other data-intensive tasks, the supercomputer is equipped with more than 700 terabytes of memory."

54 of 87 comments (clear)

  1. Re:Police officer charged with plan to cook,eat wo by Anonymous Coward · · Score: 1

    I don't get the context... are you suggesting that this supercomputer could have predicted this, if used for pre-crime analysis, ala Minority Report?

  2. Re:Police officer charged with plan to cook,eat wo by Big+Hairy+Ian · · Score: 4, Funny

    Damn if I'd just been a petaflop faster I'd have had 1st post!

    --

    Build a Man a Fire, and He'll Be Warm for a Day. Set a Man on Fire, and He'll Be Warm for the Rest of His Life.

  3. Instead of distracting it with 'climate change' by fustakrakich · · Score: 2, Insightful

    Why not have it figure a way of helping us build clean energy sources and reduce contamination? The climate changes all the time. We should learn to live with it.

    --
    “He’s not deformed, he’s just drunk!”
    1. Re:Instead of distracting it with 'climate change' by mcgrew · · Score: 3, Insightful

      I see that the guy who moderated you insightful is as ignorant of computers' workings as you are. Computers don't figure things out. There is no such thing as an "electronic brain" or a "thinking machine." Computers are nothing more than huge electronic abacuses. They don't figure things out, the scientists figure things out (theorize) and then test their theories using computerized models when they can't do direct testing.

      --
      Free Martian Whores!
    2. Re:Instead of distracting it with 'climate change' by Antipater · · Score: 1

      Yeah, because none of our current power sources create any heat.

      --
      Everything is better with chainsaws.
    3. Re:Instead of distracting it with 'climate change' by fustakrakich · · Score: 1

      Jeebus! I'm just saying that climate change has become a bullshit distraction and nothing more than a way to get unlimited funding, but all these people are as hysterical about it as they are about terrorism. We don't need to use megawatts of power to predict what's going to happen 50 or 100 years from now like it's all gonna happen over a five second event. Oh well, I guess I have to assume you didn't get the gist of my original post, which was pretty much do what you can to reduce pollution regardless how the climate changes. Devote the processing to things that matter.

      --
      “He’s not deformed, he’s just drunk!”
    4. Re:Instead of distracting it with 'climate change' by baldrad · · Score: 1

      Tell that to the people who made the 2-tonne black monstrosity in the loading bay down the hall.

      Dude don't be so racist

    5. Re:Instead of distracting it with 'climate change' by riverat1 · · Score: 1

      Global warming is a bigger threat to us than terrorism is.

  4. Re:Police officer charged with plan to cook,eat wo by Anonymous Coward · · Score: 1

    isnt a petaflop when some animal righrs activist makes an outrageous claim that everybody laughs at?

  5. Newegg does it again! by WOOFYGOOFY · · Score: 1

    I'm waiting for the ShellShocker promo in my email before I upgrade to this baby.

  6. 700TB not as exciting as it sounds by HappyHead · · Score: 4, Interesting

    The memory they list as an exciting "700+TB" is not actually all that exciting - if you divide that by the number of nodes, and then the number of CPU cores, that leaves only 2GB of ram per CPU core, which is pretty much standard for HPC cluster memory. The only thing impressive about this really, is the number of compute nodes involved, which any single submitted job will _not_ have access to all of. I manage similar, though smaller, research clusters myself, and frankly, the only clusters we had that had less than 2GB per CPU core were retired long ago. Essentially, this means they're running the cluster with the minimum amount of memory that is considered acceptable for the application.

    1. Re:700TB not as exciting as it sounds by K.+S.+Kyosuke · · Score: 2

      Would it be actually useful? Yes, you'd gave more memory in total, but any given amount of memory for a computational job would have constrained bandwidth. As far as I understand it, this is the Achilles' heel of modern machines: What use is a large memory to you when you can barely keep the execution units busy, even with caches? Especially in HPC, whenever the coherence of accesses just isn't there.

      --
      Ezekiel 23:20
    2. Re:700TB not as exciting as it sounds by Anonymous Coward · · Score: 1

      Not really... some applications (e.g., fluid dynamics simulations) scale just fine.

    3. Re:700TB not as exciting as it sounds by Anonymous Coward · · Score: 3, Informative

      Actually, that's not quite true: it is possible to submit a job request for all 18,688 compute nodes and in fact the scheduling policy gives preference to such large jobs. It's true that there aren't very many applications that can effectively use all that many nodes, but there are a few (such as the global climate simulations). You're correct about the amount of ram per CPU core, though.

    4. Re:700TB not as exciting as it sounds by Marillion · · Score: 1

      The XE6 that my team uses allocates jobs reservations at the node level. Each job gets a whole node of 16 cores with 32G ram. If you have a memory intensive task, you only run use as many cores as will fit in the available memory. It's a trade-off: some tasks will waste RAM, some will waste CPUs?

      --
      This is a boring sig
    5. Re:700TB not as exciting as it sounds by HappyHead · · Score: 1

      Larger memory per node is useful when manipulating stupidly huge data sets. Sometimes speed isn't the most important aspect in getting the calculations done, and other factors come into play, like memory size/bandwidth, disk space available, speed of that diskspace, and even network connectivity if you're doing MPI programming.

      While I realize it would be great to teach everyone efficient programming techniques, so they could streamline their memory usage down to the bare minimal, it's not always possible, and sometimes it's just not practical to do - our users come from pretty much all disciplines, from Physics, Biology, Chemistry, Engineering, and even a few from History. (Yes, a History researcher using HPC to do calculations and simulations. He's actually doing some pretty neat stuff.) Teaching that diverse a group of people to program super efficiently is not going to work - they're not interested in making super awesome code, they just want their numbers crunched, and are only willing to learn the bare minimal to get it running. The worst cases tend to get assigned a staff member to consult with them and get their code cleaned up so that they don't break the clusters, but with a few thousand users, we can't do that with everybody - most of them would never show up to the classes anyways.

    6. Re:700TB not as exciting as it sounds by gentryx · · Score: 2

      Titan is a capability machine which distinguishes it from capacity machines. As such it designed for large/extreme scale jobs (which includes full system runs). I expect the techs are just now prepping Linpack for the next Top500 at SC12.

      The ratio of 2 GB/core isn't going away anytime soon. The reason is: a) the speed per core is stagnating, thus adding more memory per core just means that one would end up with more memory per core that it could process in a timely manner and b) if you need more memory, you'll just allocate more nodes, the additional cores you get by that don't exactly hurt.

      --
      Computer simulation made easy -- LibGeoDecomp
    7. Re:700TB not as exciting as it sounds by gentryx · · Score: 1

      If you need more memory, simply allocate more nodes. Problem solved. Hardly anyone needs more than 2 GB/core.

      --
      Computer simulation made easy -- LibGeoDecomp
  7. Is the K20 really a GPU? by CajunArson · · Score: 2

    GPU means graphical processing unit. Now consumer GPUs are pressed into service for compute tasks like BOINC & folding, but they are still GPUs (they can still do graphics).

    Does Nvidia even bother to put in the graphics-specific silicon and output hardware on the K20, or should these things really be called.. I dunno.. "compute accelerators" or something like that?

    --
    AntiFA: An abbreviation for Anti First Amendment.
    1. Re:Is the K20 really a GPU? by TeknoHog · · Score: 1

      General Processing Units

      --
      Escher was the first MC and Giger invented the HR department.
    2. Re:Is the K20 really a GPU? by Shatrat · · Score: 1

      Maybe you should consider the origin of the word? http://en.wikipedia.org/wiki/Graph_(mathematics)

      --
      09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
    3. Re:Is the K20 really a GPU? by CajunArson · · Score: 1

      You have it backwards... graphics have been around since the time of the caveman. "Graph theory" only came into existence in the late 19th century and took its original cues from hand-drawn graphs..which are a type of graphics. Plus, adding and multiplying numbers, which is basically what the K20 does on a huge scale, is by no means an operation that is limited to graph theory.

      --
      AntiFA: An abbreviation for Anti First Amendment.
    4. Re:Is the K20 really a GPU? by CajunArson · · Score: 1

      That's the problem though... the K20 is definitely not "general" but highly specialized. Throw the right type of problem through optimized CUDA code and it'll run great. Throw it the wrong type of computational problem and it'll go nowhere fast. That's specialized instead of general.

      --
      AntiFA: An abbreviation for Anti First Amendment.
    5. Re:Is the K20 really a GPU? by Anonymous Coward · · Score: 1

      With teraflops of single and double precision performance, NVIDIA® Kepler GPU Computing Accelerators are the world’s fastest and most efficient high performance computing (HPC) companion processors. [nvidia.com]

      The K20 really is still built on graphics-specific architecture, but it would be a waste to include the output hardware, just think of all the servers that have never had a monitor attached.

  8. On a more serious note by WOOFYGOOFY · · Score: 2, Interesting

    It's a great and important tool for policy makers to be able to crunch this magnitude of data, but not being able to do this is not the problem wrt climate change.

    The problem is purely political, specifically, American conservatives are denying this science the same way they deny the science of evolution, the same way they deny the overwhelming proof that smoking causes cancer and second hand smoke does the same, the same way they denied CFCs caused a hole in the ozone layer and risked all our lives on that occasion also.

    On the one hand you have hard working, selfless scientists who at this point are sacrificing their personal lives, financial security, their sanity and risking literal criminal prosecution from out-of-control attorneys generals who are drunk blind on power and dieology to continue to speak the truth, Cassandra-fashion, to a heedless and reckless nation.

    On the other you have people who have never worked a day in their lives to earn the just authority to advise and inform Congress on this topic nevertheless holding forth, just stealing the authority the other group has worked to earn and effectively screaming "NO FIRE" in a burning theater, inducing people to do nothing when in reality they must do something in order to survive.

    The first group fits perfectly my definition of hero .

    The second fits perfectly my definition of murderer.

    1. Re:On a more serious note by WOOFYGOOFY · · Score: 4, Interesting

      You've got to be kidding.

      Climate scientists are routinely subjected to death threats t themselves and their families by the people you now claim just want a rational debate. They are subject to politically motivated FOIA searches, public ridicule and accusations that they are lying, corrupt, faking data. They have their emails stolen and their personal lives wrecked through constant harassment. All that is now SOP for the right wing lunatic deniers. The Glenn Becks. The Koch brothers. Murdoch. The Cucinellis. FoxNews employees. Lord Monckton. The Heritage Foundation the religious right Ayn Rand amphetamine addicts et al. ...

      So you've got to be kidding.

      The rational debate you want has been happening for the past 30 years, in scientific journals and at symposiums and conferences where rational debate on technical matters occurs. Did they join it? Can they understand it, or do they just tell themselves they can? If you can't understand the arguments then you need to listen to experts who can. That's the nature of modern society; that's reality. And when 97% of all qualified experts agree , then that is a bright clear line on the other side of which lies willful and deliberate manslaughter and murder. Just ask any court how it works.

      Just because someone can't accept reality doesn't mean they are exempt from the morality that applies to their judgement and actions which issue as a consequence of their reality denial.

      If I am thoroughly convinced I can perform brain surgery and fake my way into an operation, I can expect to be prosecuted when I am outed. Ditto Lord Monckton and the Koch Brothers and all the wretched animals at the Heritage Foundation.

      It's not about rational debate, it's not about convincing anyone through data or studies or the application scientific method . We know that because, as human society defines that process,- and it is human society that gets to define that process and not the right wing- that has already taken place .

      Don't like the outcome? Tough shit.

      Here's the game they're playing.: "You can't prove it. " . You can't prove I was lying. You can't prove I didn't believe my own horseshit. You can't prove that I was not perfectly conscious of the fact that the position I was espousing was contrary to reality. I'm safe inside my own brain where only I know the truth. So you can't prosecute me, because I 'm entitled to my opinion.

      But you know what ? People make laws. They make laws with the directed and specific purpose of punishing anti-social behavior that harms other innocent humans. We call that behavior "criminal". and it's criminals- through their actions- who ultimately decide what laws we write into existence for the sole purpose of stopping and punishing them.

      What these people believe is that law is something which will not pursue them wherever they go, whatever they do irrespective of the real world consequences their actions have on humanity.

      Wrong. Dead wrong.

      In Nuremberg we hung Germans for breaking laws we made up after the fact - ex post facto lawmaking- specifically to address their crimes. They also thought they were in some sort of legal safe harbor, since what they had done they had done to their own citizens acting as agents of a sovereign nation. And they were right. That is until the day we decided they were wrong. On that day, we made up a new crime -Crimes Against Humanity. Then we tried them for it. Then we hung them for breaking it.

      It's criminals who decide what behavior comes to be seen as criminal. The delusion that the law will not, cannot for some reason follow you THERE is just that- a delusion.

    2. Re:On a more serious note by gander666 · · Score: 1

      Damn. And my Mod points expired yesterday.

      --
      Suppose you were an idiot and suppose you were a member of Congress ... but I repeat myself. - Mark T
    3. Re:On a more serious note by WOOFYGOOFY · · Score: 1
      You need to read history. What yo're saying is exactly what the South said during the run up to the Civil War. What they said was- we are not convinced. We are of the (then scientific / religious) belief that Negroes are not human. We do not believe as you do and you must either convince us of your side through rational arguments or do nothing about slavery.

      This is what every cult that wants to impose its alternative-reality on main stream society says. "Hey, you can't prove you're right. We are not convinced., We dissent. We have a right to dissent and a right to live as we see fit .You cannot impose your will upon us! It's immoral.

      That's why the South called the Civil War the "War of Norther Aggression". Because they were just living their lives, and conducting their affairs as their belief system bid. Then the North came along and provoked them and would not leave them be.

      You need to read history if you think the "we are not convinced, Sir. " meme bear any weight.

    4. Re:On a more serious note by WOOFYGOOFY · · Score: 1

      Riiiiiiight..... because there's a moral equivalence between people who hate others because they hold contrary opinions on this or that social issue and people who hate others because the actions of those other people lead directly and indisputably to the death of millions of innocent people.

  9. Miss the Days by rwise2112 · · Score: 1

    I don't know if I'm alone in this, but I kind of miss the days when supercomputers wern't just clusters of off the shelf components. I feel we've lost something.

    --

    "For every expert, there is an equal and opposite expert"
    1. Re:Miss the Days by timeOday · · Score: 2

      On the other hand, the modern GPU is much closer to the vector units of "classical" supercomputers than anything minis/PCs of that era had.

    2. Re:Miss the Days by rwise2112 · · Score: 1

      True enough. I had an account on a Convex C2 supercomputer when I was in university which was very vectorised. In our department, we were able to mount the 9-track tapes ourselves. Yeah, that's how old I am!

      --

      "For every expert, there is an equal and opposite expert"
    3. Re:Miss the Days by jaharris87 · · Score: 1

      I don't know if I'm alone in this, but I kind of miss the days when supercomputers wern't just clusters of off the shelf components. I feel we've lost something.

      HPC is being forced to use off-the-shelf components. There is the funding for R&D of application-specific hardware.

    4. Re:Miss the Days by SecurityGuy · · Score: 1

      +1. It's hard to care about a "faster" computer when faster just means more nodes. Wow, how are we ever going to top that one? Just build one with more nodes. It's become much more a question of money than innovative technology.

  10. Oak Ridge by Antipater · · Score: 1

    Rather than another supercomputer, couldn't they spend the money on actually upgrading Oak Ridge's infrastructure so the buildings aren't falling apart, and 80-year-old nuns can't walk through the perimeter fence?

    --
    Everything is better with chainsaws.
    1. Re:Oak Ridge by jbeaupre · · Score: 2

      Don't worry. Any intruder will be scanned and sent to the gaming grid.

      --
      The world is made by those who show up for the job.
    2. Re:Oak Ridge by Andy+Prough · · Score: 1

      Yeah, those nuns - HUGE problem.

    3. Re:Oak Ridge by SeanAhern · · Score: 1

      Wrong plant. ORNL is a completely separate laboratory from Y-12, even though they're located in the same city.

  11. Official Sumper Computing List by nsharifi · · Score: 1

    The official list of Top 500 (last updated 2012/06) states "Sequoia - BlueGene/Q, Power BQC 16C 1.60 GHz, Custom" as the number one super computer. Sequoia is nearly as powerful as Titan.

  12. Re:What everybody wants to know by oodaloop · · Score: 1

    What about if it blends, runs linux, or what it does in Soviet Russia?

    --
    Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
  13. It's because he used a COMPUTER to plot his crime by Andy+Prough · · Score: 1

    Computers are evil - didn't you know?

  14. Which OS? iOS6 or Windows 8? by Andy+Prough · · Score: 1

    Cause I mean - their like, so FAST. Right? Right? Surely not grubby old, crufty old Linux - right?

    1. Re:Which OS? iOS6 or Windows 8? by nsharifi · · Score: 1

      You are absolutely wrong. 75% of super computers run on Linux. Go and see.

    2. Re:Which OS? iOS6 or Windows 8? by Andy+Prough · · Score: 1

      You are absolutely wrong. 75% of super computers run on Linux. Go and see.

      Shocking! Say it ain't so! It must be because nasty old Linux stole all that technology from Bill Gates and Steve Jobs.

    3. Re:Which OS? iOS6 or Windows 8? by fa2k · · Score: 2

      You are absolutely wrong. 75% of super computers run on Linux. Go and see.

      I thought that sounded low, so I went and checked at http://i.top500.org/stats . Linux has 92.4 % of the top 500. Then you have "Unix" at 4.8 and Mixed at 2.2.

    4. Re:Which OS? iOS6 or Windows 8? by Andy+Prough · · Score: 1

      No, actually Linus stole it from the likes of Dennis Ritchie and Ken Thompson. Oddly these names rarely ever get mentioned in Linux discussions. I guess it really burns the fanbois to pay respects to the true giants of computing.

      Shocking! Say it ain't so! It must be because nasty old Linus stole all that technology from Dennis Ritchie and Ken Thompson and Darl McBride.

      Happy now, Darl ... errrr .... I mean, "AC"?

  15. Is it lighter than a MacBook Air? by Andy+Prough · · Score: 1

    Can it play Angry Birds? How many fart apps will run on it? These are the important questions.

    1. Re:Is it lighter than a MacBook Air? by oodaloop · · Score: 1

      Yes. All of them. I agree.

      --
      Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
  16. I had the same feeling.... by bdwoolman · · Score: 1

    But then again it does tell you that the off-the-shelf components we all use are none too shabby. For, as we are all too sick of hearing, the boxes we use right now well outpace those custom-built super computers created in the days of yore. Okay. Maybe not even yore, maybe even less time than that. But still...

    --
    "No fear. No envy. No meanness." Liam Clancy
  17. What? by ZonkerWilliam · · Score: 1

    20 petaflops of performance...

    ...700 terabytes of memory

    Pfffft that all?!

  18. Re:Imagine a beowu... by Anonymous Coward · · Score: 1

    Wait, isn't the irony that they used AMD cpus, but Nvidia GPUs in the same system?

    Maybe combined with the fact that last I checked the AMD GPUs were lower watts per flop?

    Sounds like they made this cluster explicitly to increase the rate of global warming. And by modeling it on the cluster, always be just behind the curve as the cluster tips the balance it's supposed to be simulating :)

    (And yes I know that they're probably using Nvidia because the majority of their tasks leverage cuda, but it's still funny!)

  19. My visualizations by SeanAhern · · Score: 3

    My favorite part of the article is the photo that accompanies it. Two of my scientific visualizations are on there, the red/yellow picture of an Alzheimer's plaque being attacked by drugs (behind the N of TITAN) and the silver structure of a proposed ultra-capacitor made from nanotubes (to the right of the N).

  20. K20 ~ kepler : better perf/watt than GCN/SI by slew · · Score: 1

    Although previous generation AMD used to be more flops/watt, the new generation of Kepler GPUs from Nvidia are quite a bit better than AMD's current generation (GCN / southern islands).

    http://hothardware.com/Reviews/NVIDIA-GeForce-GTX-680-Review-Kepler-Debuts/?page=15

    FYI: The K20 used in the Titan system are Kepler based.

  21. Slashdot Throwback sarcasm! by jameshofo · · Score: 1

    I wonder how that would do in a Beowulf cluster!

    --
    Good leaders run toward problems, bad leaders hide from them.