The Supercomputer Race

CWmike writes "Every June and November a new list of the world's fastest supercomputers is revealed. The latest Top 500 list marked the scaling of computing's Mount Everest — the petaflops barrier. IBM's 'Roadrunner' topped the list, burning up the bytes at 1.026 petaflops. A computer to die for if you are a supercomputer user for whom no machine ever seems fast enough? Maybe not, says Richard Loft, director of supercomputing research at the National Center for Atmospheric Research in Boulder, Colo. The Top 500 list is only useful in telling you the absolute upper bound of the capabilities of the computers ... It's not useful in terms of telling you their utility in real scientific calculations. The problem with the rankings: a decades-old benchmark called Linpack, which is Fortran code that measures the speed of processors on floating-point math operations. One possible fix: Invoking specialization. Loft says of petaflops, peak performance, benchmark results, positions on a list — 'it's a little shell game that everybody plays. ... All we care about is the number of years of climate we can simulate in one day of wall-clock computer time. That tells you what kinds of experiments you can do.' State-of-the-art systems today can simulate about five years per day of computer time, he says, but some climatologists yearn to simulate 100 years in a day."

Comments

The true best measurement

[Anonymous Coward]

Is how many libraries of congress it can read in a fortnight.

Re:The true best measurement

[A non-mouse Coward]

Is how many libraries of congress it can read in a fortnight.

Nope.
PROGRAM HelloWorld
PRINT *, "Hello World"
END PROGRAM HelloWorld

Re:The true best measurement

[ari_j]

Define "read."

Re:The true best measurement

[blair1q]

a furlong (plus one ear)

Re:The true best measurement

[JYD]

Isn't the bottleneck mostly I/O in this case, rather than CPU, since the said operation requires very little floating point operation, which is what the TOP500 is essentially all about.

Re:The true best measurement

[aldm]

Run Slashdot without this message:
A script on this page may be busy, or it may have
stopped responding. You can stop the script now, or you
can continue to see if the script will complete.
Continue
Stop script

Re:The true best measurement

[ari_j]

See my other comment regarding the difference between reading as in streams and reading as in books.

Weather Day After Tomorrow

[Alan_442]

I year for the day when they can predict the weather 48 hrs from now and be 99% accurate. Also, the same with hurricanes.

Re:Weather Day After Tomorrow

[Alan_442]

yearn ...that is.

Re:Weather Day After Tomorrow

[Fishbulb]

Don't hold your breath; it'll disrupt the predictions.

Re:Weather Day After Tomorrow

[geekoid]

48 hours forecasts are very accurate. Hurricanes aren't so bad either,.
I remember when the big leap was talking about a 3 day forecast to viewers, not it's 7-10. Still with decent accuracy.

Now I note you said 'predictions' and that will never happen. I am assuming you meant forecasts.

Re:Weather Day After Tomorrow

[smittyoneeach]

The mondo-flop race,
As the hair on your face,
You yearn to displace,
So do it with grace.
Burma Shave

Re:Weather Day After Tomorrow

[geezer nerd]

I can remember when the big desire of weather simulation supercomputers was to take less than 24 hours to do a 24-hour forecast. IIRC back in the second half of the '70s there was a big government-funded effort to build special fluid-dynamics oriented new machines to break that barrier.

44 years ago 1-5 megaflops was hot! What excitement we felt when the CDC6600 was installed at my university!

Back in '85 I was part of a startup building a mini-Cray, reimplementing the Cray instruction set in a smaller, cheaper box. I remember we focused on the Whetstone benchmark a lot, and it turned out that the Whetstone code really was bound up by moving characters around while formatting output strings, etc. We paid very careful attention to efficiently coding the C library string handling routines, and that got us more performance payback than anything we could do to optimize the arithmetic. One needs to understand the benchmark being used.

Re:Weather Day After Tomorrow

[Deadstick]

Well, if you don't know if there's gonna be a hurricane, then you don't know what the weather's gonna be, do you?

rj

Re:Weather Day After Tomorrow

[Iamthecheese]

Now get off his lawn!

Re:Weather Day After Tomorrow

[geezer nerd]

I know this is Slashdot, and penetrating, pithy, to-the-point comment cannot be expected. But I really have to ask, just what does that comment mean? And how is it relevant to anything in the thread?

Simulation

[gringer]

Simulate 100 years of climate in a day? Here's my code:

echo -e "sunny\nrainy\ncloudy" | rl -rc 36525

Re:Simulation

[swillden]

What is rl?

Re:Simulation

[Kneo24]

Relive! The machines have sentience! RUUUUUUN!

Re:Simulation

[monsul]

http://ch.tudelft.nl/~arthur/rl/
From the webpage: rl is a command-line tool that reads lines from an input file or stdin, randomizes the lines and outputs a specified number of lines. It does this with only a single pass over the input while trying to use as little memory as possible.
Didn't know about it either. Seems marginally useful

Re:Simulation

[Surt]

The Random Line picker.

Re:Simulation

[GroeFaZ]

You jest, but that's exactly the point. "Simulated years per day" is about as meaningless a metric as it gets, because, as you proved, that number depends on the complexity of the underlying climate model, and also on how well the software was written, i.e. if it is optimized for both the hardware and the model to be computed.

Both these factors are hard/impossible to control and to standardize, and the only factor that does not change is the actual hardware and its peak/sustained performance, so it's the only sensible metric.

Re:Simulation

[gringer]

It's a simple line randomisation program. I use it a lot in the work that I do for short shell scripts, which involves doing lots of permutations of lines of text files (or just sequences of numbers). Once that code gets put deeper into a loop (and hence becomes more of a limiting factor in terms of excecution time), I substitute it for faster stuff written in Java, Perl, R, C++, or whatever else takes my fancy at the time. shuf is a similar program which seems to be in the coreutils of Debian, but doesn't allow resampling from the brief reading I've had regarding its use.

Re:Simulation

[Anonymous Coward]

Simulate 100 years of climate in a day? Here's my code:

echo -e "sunny\nrainy\ncloudy" | rl -rc 36525

Historic note: The Princeton Institute for Advanced Studies (home of Einstein, et al.) had a big internal argument in the early 50s about whether or not to buy its first computer. (Magnitudes less memory and power than a PC has now). Many of the staff were happy to stick with chalk and pencils. The pivotal argument that caused the Board of Directors to back those who wanted one was that a computer would allow them to quickly and accurately predict the weather. Hah.

Re:Simulation

[rbanffy]

"Seems marginally useful"

For you, you insensitive clod. Someone with feelings wrote it!

Re:Simulation

[Anonymous Coward]

I've spend weeks looking for a command such as rl.
Thanks.

(pick random file from random subfolders)

Re:Simulation

[swillden]

Thanks. For Debian/Ubuntu users, it's in the package "randomize-lines".

Financial modeling and spying better funded

[Ritz_Just_Ritz]

Sadly, while predicting the weather and better understanding it ultimately helps a lot of people, I suspect a LOT more computing power is thrown at more mundane things like predicting where the financial markets are going to be based on a gazillion data inputs. Probably even better funded are the vast datacenters around the world that fondle communications and other data for the spymasters. I doubt those computing resources are represented in the annual supercomputing lists. :)

Re:Financial modeling and spying better funded

[Quarters]

But, in a roundabout way the financial market simulator will ultimately help the weather simulator's performance. Everyone knows that business apps are written in VB. That means the financial simulator folks need MUCH more powerful supercomputers to run their code at anything close to appreciable speed. That same machine will run well coded weather apps blazingly fast!

Re:Financial modeling and spying better funded

[Alwin Henseler]

Right, and let's not forget supercomputers used for simulating nuclear weapon designs, or things like that. Only the latest & greatest is good enough when talking about ways to destroy fellow humans.

Re:Financial modeling and spying better funded

[blair1q]

I suspect no computing power is being thrown at predicting where the financial markets are going.

A lot is thrown at pretending to predict it, but it's brilliantly obvious that the output of such things is no better than chicken entrails or the last two presidential elections.

Re:Financial modeling and spying better funded

[The Master Control P]

And by the time we come up with your "indisputable evidence" the northwest passage will probably be ice-free all year round, we'll be in the middle of a mad scramble to raise city ocean barriers another 10 feet because another massive chunk of Antarctica collapsed into the sea, and you'll be demanding indisputable evidence that those calthrate blowouts that are happening more and more up north will be a real problem.

Some problems really are bad enough that they require proactive attention. Stop hiding behind "B b but it has to be indisputable" as an excuse to do nothing.

Re:Financial modeling and spying better funded

[sumdumass]

Right now, it is mainly manufactured evidence. There is still a chance that no indisputable evidence will ever be found and that we are barking up the wrong tree.

Before people start blasting me for blasphemy, As soon as you stop looking at things objectively, you stop pursuing knowledge and start believing with faith instead of looking for the truth. I don't care how convinced you might be that X=Y, if there is no room for Y to equal Z without Z equaling X, you have lost the scientific principle and are acting on faith alone. Not being able to falsify something and not being allowed to are very similar and have the same effects.

Re:Financial modeling and spying better funded

[PingPongBoy]

I suspect a LOT more computing power is thrown at more mundane things like predicting where the financial markets are going to be based on a gazillion data inputs.

I wouldn't say mundane.

For one thing, any intelligent deep thought requires the gazillion inputs because of the sheer number of factors and an error or bad approximation in just a few inputs could throw off the decision or computation. Granted, the financial markets are typically so frenzied that hardly anyone knows whether to buy or sell, but we would do well to build computers that are capable of market analysis.

Data input is one of the biggest hindrances to computing. Take a typical office, and there are people laboriously entering information because that is the only way to get it right. It shouldn't take petaflops to scan a wrinkled page with scribbles or listen to voice in a noisy room.

Re:Financial modeling and spying better funded

[Anonymous Coward]

>I doubt those computing resources are represented in the annual supercomputing lists.

One are, place 11, 102800 GFlops (Rmax) dedicated to listen to bits and bytes sent in Sweden.

Re:Financial modeling and spying better funded

[robthebloke]

I suspect a LOT more computing power is thrown at more mundane things like predicting where the financial markets are going

Even my tea leaves say down... ;)

Re:Financial modeling and spying better funded

[peawee03]

Tropospheric CO2 and O3 run around on a roughly 10,000 year cycle, between 150-275 ppm for CO2. This has been shown through ice cores in Antarctica, and other methods proven to give us a real look at historic atmospheric data. In the last 150 years, CO2 has gone through the roof.

In addition, global O3 concentrations used to be around 10 ppb. Currently, in the midwest region of the USA, it stands around 40 ppb. Through PopFACE and AspenFACE experiements, along with work done all over the world, it has been proven that even today's elevated O3 levels negatively impacts the ability for trees to act as a carbon sink, which then results in more CO2 in the air.

Where does this lead us? CO2 accounts for roughly 9-26% of the greenhouse effect, which is both undisputed and, overall, a Very Good Thing for life on Earth (it's what tempers the climate on the surface). In laboratory experiments, it also has been shown that in columns of air composed similarly to our atmosphere at different points in history, heat is trapped by the test atmospheres in amounts similar to what we're seeing here on the ground on earth when you crank up the CO2 levels to how we've cranked them up.

I haven't discussed models here at all. I've only discussed experiments and statistical analysis based on observational data.

Re:Financial modeling and spying better funded

[gadget junkie]

Sadly, while predicting the weather and better understanding it ultimately helps a lot of people, I suspect a LOT more computing power is thrown at more mundane things like predicting where the financial markets are going to be based on a gazillion data inputs. Probably even better funded are the vast datacenters around the world that fondle communications and other data for the spymasters. I doubt those computing resources are represented in the annual supercomputing lists. :)

There are a couple of misperceptions here.

Both the problems described, modeling years of weather models or modeling financial instruments, suffer from a definite flaw: they are not mathematical problems in the "high school" sense of the world, i.e. it is not possible to prove that there is only one finite solution that is demonstrably right.
Financial models are "fit to reality": you take a long time series, make a few wild guesses, throw it into a Cray-2, and look what the model says. Lather, rinse, repeat. they work, most of the times.....Bank regulators just recently allowed banks to discard simpler risk controls if they proved that they had a financial modeling tool that did not get their accepted measure of risk wrong more that 3 days out of 120, and these models were mostly sophisticated mathematicals and statistical implementations....and then Lehman goes belly up. Mind you, they were using computer modeling as well.

[DISCLAIMER: I've been working in finance since 1988, and I believe equally in advanced financial modeling, the tooth fairy and Santa]

Weather modelling in yearly time scales suffers from the same flaw, in my view: unless you have a long enough set of possible inputs , it's not verifiable in reality.
If I try to make prediction over next week, I only need a week to see if my model is horribly wrong. If I want to test it in different seasons, 1-2 years can give me a good enough hint about the accuracy of a model.
If I am trying to predict average weather changes on a 20 years time scale, I need 20 years of historical data to get 1 (one) result. another year to get two, etc. exaggerating a lot, it's like modeling to predict red or black at the roulette: 50% of the models will get it right the first time, and 25% over two tries. definitely not enough to know if it works.

Re:Financial modeling and spying better funded

[jstott]

Weather modelling in yearly time scales suffers from the same flaw, in my view: unless you have a long enough set of possible inputs , it's not verifiable in reality.

Weather modeling (what the National Weather Service does) and climate modeling (what global warming folks do) are different problems.

Weather modeling has lots of data available (365 forecasts per year, per model pixel) and its easy to verify predictions against weather station measurements. The problem is that it's impossible to specify the current conditions with sufficient accuracy for more than a 3--5 day forecast [sensitive dependence on initial conditions].

Climate modeling, on the other hand, has much less historical data to work with, but since it deals with average temperatures instead of day-to-day weather changes its also much less sensitive to the initial conditions. That's how come we can talk about 10,000 year climate changes but still can't tell if it will rain next weekend or not.

-JS

Re:Financial modeling and spying better funded

[gadget junkie]

[...]

Climate modeling, on the other hand, has much less historical data to work with, but since it deals with average temperatures instead of day-to-day weather changes its also much less sensitive to the initial conditions. That's how come we can talk about 10,000 year climate changes but still can't tell if it will rain next weekend or not.

-JS

what you are saying is a literal truth: we can talk about 10.000 years climate changes..... but we are unable to pinpoint the real reasons behind the little ice age, apart from noting the coincidence of part of it with the Maunders minimum, which marks a low in sunspot activity.

In a sense, and within the confines of being sure about the models, we have advanced very little in our understanding of climate. weather forecasting has really improved, as you rightly reminded, and three day forecasts are regularly issued, with a much greater granularity than twenty years ago. my perception is that, apart from saying "most of the scientist agree", many a climatologist would be very hard pressed to bring to slashdot an explanation sufficiently convincing to withstand Occam's razor (or slashdotting). Remember, Columbus had to be a great salesman to convince the Spanish to fund his exploration, because the conventional wisdom was an hazy notion of the Earth (flat like a cake, and the conventional wisdom of the times?). Not bad, considering that Eratosthenes of Cyrene had calculated its diameter very exactly in the 3rd century B.C., further proof that Science is not a matter of majority vote, and that occasionally science can go backwards.


Mind you, in my view the same mistake is what caused the recent financial crash ; bank people, analysts and other pundits forgot their Popper and thought that since the models had worked for a long time they all made sense; but that's like saying that since you cause the sun to rise every day by waking up, you do not want to risk oversleeping in the morning: it works until you get out for a bout of drinking with the buddies, and the morning after you realise that you slept till midday...and the sun is already up.

Imagine

[sexconker]

Imagine a beowulf cluster of these.

Flops not useful?

[zippthorne]

But.. The whole point is to test the model, and the models change, don't they? Surely we're not just simulating more "years" of climate with the current batch, but improving resolution, making fewer simplifying assumptions, and hopefully, finding ways to do the exact same operations with fewer cycles.

How can you possibly evaluate supercomputers in any other way except how many mathematical operations can be performed in some reference time? And.. some serial metric if the math is highly parallel, since just reducing the size of vectors in those cases wouldn't actually result in those flops being useful for other tasks.

Re:Flops not useful?

[geekoid]

That's just the problem, people want to hear raw numbers, but those are useless.
How well can it do the specific task it needs to do is the actual question. It's a hard one, to be sure.

Re:Flops not useful?

[corsec67]

Flops wouldn't test how well the interconnects work.

Since you say "increase the resolution of the model", you are expanding the size of the model, and how much data must be used by all of the nodes of the computer.

Since how important the interconnect properties are is dependent on the model, with almost no communication needed, like for F@H, to a problem that needs all of the nodes to have access to a single shared set of data, it would be very hard to quantify performance in one number.

Unfortunately, there are more than a few fields where marketers want a single number to advertise in a "mine is bigger than yours" competition, and come up with a metric that is almost worthless.

Re:Flops not useful?

[Fishbulb]

Because the climate simulation model they use does a LOT of inter-process communication. Each piece of the calculation depends on what's going on around it.
Ever see footage of manual calculation rooms NASA used to have*? Imagine if every one of the calculations those people were doing depended on the previous calculation they did, AND all of the previous calculations of their eight nearest neighbors.

Now you know why that atmospheric model has a benchmark rated in "century/months" - the number of centuries of "model"-time that can be calculated per month of "wall clock"-time.
That they're working toward a century per day is pretty amazing, especially when they're also tightening the resolution down from 20 sq. km. to under 5 sq. km. by 6+ levels of atmosphere.

* - http://www.jpl.nasa.gov/news/features.cfm?feature=1327

Note: I used to work in the same division of NCAR as Rich Loft. Brilliant guy.

Re:Flops not useful?

[Salamander]

How can you possibly evaluate supercomputers in any other way except how many mathematical operations can be performed in some reference time?

Simple: you evaluate how much actual work it can perform across the entire system per unit time, where "actual work" means a mix of operations similar to some real application of interest. The whole problem here is that practically no real application is as purely focused on arithmetic operations as Linpack. Even the people who developed Linpack know this, which is why they developed the HPCC suite as its successor. It's composed of seven benchmarks, including some (e.g. stream triad) that mostly stress memory and some (e.g. matrix transpose) that mostly stress interconnects. If you want to get an idea how your application will perform on various machines, you determine what mix of those seven numbers best approximates your application, assign appropriate weights, and then apply those weights to the vendor numbers. Then you negotiate with the two or three most promising vendors to run your application for real. SPEC should have put an end to simplistic "single figure of merit" comparisons, or if not them then TPC, SPC, etc. Sadly, though, there's still always someone who comes along and tries to revive the corpse.

Re:Flops not useful?

[jd]

To be honest, I thought most people already knew about and used HPC Challenge, which produces 7 different benchmarks covering different types of mathematical problem, memory bandwidth and communications bandwidth. I also imagined people would use MPI-IO for measuring MPI performance, that the numbers on the Top500 was simply because it's hard to track a vast number of stats in a meaningful way.

Of course, if it's actually the case that people are dumb, lazy or in marketing, then that would explain why we don't get a full range of stats, even though the tools have existed for many years and are certainly widely known.

Re:Flops not useful?

[jd]

Oh, and you can perform further diagnostics by using DAKOTA and KOJAK to profile a standard MPI application across the supercomputer. Since these profilers give you internal details, you should be able to see where specifically slow-downs occur - be it a processing quirk, a system library flaw or a networking glitch. Hell, I wish someone paid me to benchmark supercomputers. I can pretty much guarantee RoleMaster fans will love the numbers, even if nobody else can cope with the stats.

Re:Flops not useful?

[rockmuelle]

"How can you possibly evaluate supercomputers in any other way except how many mathematical operations can be performed in some reference time? "

It's much more subtle than that. Most programs, including weather simulations, use a large amount of data stored on disk and in RAM. The problem with LINPACK as a benchmark is that, for all practical purposes, it ignores this cost by using a few very specific linear algebra operations that have very low communication/computation ratios. The LINPACK number is only relevant if your program is primarily based on operations that have this characteristic.

Unfortunately, most scientific codes (weather simulations included, of course), have evolved past simple implementations based on dense matrix-matrix multiplication (the particular kernel that gives the peak performance number) and include a number of steps that perform closer to the speed of the memory bus than the speed of the processor (sparse matrix operations, which make simulations tractable with millions of variables work this way). There's also the simple fact that very few programmers are even aware of the techniques required to achieve even 50% of peak performance on a kernel as simple as matrix-matrix multiplication. And, the cost of getting past 50% in programmer time is rather high. So, even if scientific codes could be optimally implemented, there's almost no chance they are.

Most people in HPC (myself included) have reached the point where the Top 500 list is a fun curiosity, but has little relevance to actual practice of supercomputing. Optimizing memory bandwidth and interconnects is much more important than raw FLOPS.

Still, I applaud the Roadrunner project. They took some serious risks to pull it off and created a very impressive computer. It's too bad that it will most likely be a one-off implementation (yeah, you can buy QS-22s from IBM, but I doubt they'll be around for too long).

-Chris

Re:Flops not useful?

[ceoyoyo]

Actually, Linpack is not embarrassingly parallel so it DOES test how well the interconnects work, to some extent.

The top 500 list is interesting, but if you're building a supercomputer to make a certain rank you have too much money and you should really give me some.

You build a supercomputer to perform some task or class of tasks. If it gets you on the list, cool.

Re:Flops not useful?

[mr_mischief]

The hard part of that is that my hypothetical high-end supercomputer application and your hypothetical high-end supercomputer application don't do the same thing, and so we must weight those values differently.

To get a single unbiased score, there must be one simple test or a group of unweighted ones. If you're giving an equal weight to different benchmarks, then the overall score might still mean something. The single value from equal weighing still won't be very predictive for applications that stress one part of the system more than others.

It'd be nice if there was a top 500 in each of the HPCC categories and a top 500 overall, but I'm not sure who's going to collect, organize, and present all that info.

Re:Flops not useful?

[blahplusplus]

"Simple: you evaluate how much actual work it can perform across the entire system per unit time, where "actual work" means a mix of operations similar to some real application of interest"

Personally I'd like to see supercomputing power start automating detecting bad design and potential failures of someone elses code, not to mention optimization.

I've always wondered what you could do with google and a few supercomputers coming the open source database where you can deduce what a function does without needing comments based on analysis on many types of projects and code.

I'm sure if we threw enough money at it we could start making software engineering a lot better then it is today.

Re:Flops not useful?

[RulerOf]

But.. The whole point is to test the model, and the models change, don't they?

Alas, no. It's suspected that the closer your computer climbs to the top of this list, the larger your penis becomes.

Re:Flops not useful?

[bryce4president]

simply because it's hard to track a vast number of stats in a meaningful way.

Do you think they could build a supercomputer to do that?

Re:Flops not useful?

[jd]

Heisenbenchmarks. They could either use a supercomputer for that, or know how fast that supercomputer was, but they couldn't do both at the same time.

Re:Flops not useful?

[Salamander]

Actually, Linpack is not embarrassingly parallel so it DOES test how well the interconnects work, to some extent.

Yeah, and your head is useful as a hammer to some extent. It's a very small extent in either case, far short of the observation making a dent in the claim it supposedly refutes. Linpack is embarrassingly parallel enough to make it absolutely useless for comparing systems' memory or interconnect speed, which makes it almost entirely useless for comparing those systems' ability to run real-world code outside of an increasingly irrelevant domain.

Re:Flops not useful?

[ceoyoyo]

If you buy a supercomputer on specs then you deserve what you get. Supercomputer benchmarks are useful for PR, technobating and bragging rights. Linpack works just fine for all of those. Well, maybe not so well for the really specialized clusters. Which are probably the ones that people who suggest switching have some special interest in.

Anonymous Coward

[Anonymous Coward]

The people who want to simulate 100 years of climate a day will, when they get it, want to simulate 2000 years a day.

Re:Anonymous Coward

[superdave80]

Yeah, that will be real useful.

"There is a 90% chance of rain in Omaha on Wednesday, October 6th, 4008"

Re:Anonymous Coward

[Anonymous Coward]

They'll just have to move to venus, where they have 5832-hour days. Of course keeping the supercomputer cool is going to be tricky.

Re:Anonymous Coward

[Surt]

It's marginally more useful if it predicts 0% chance of rain because the average surface temperature of the planet has exceeded 100C on Wednesday, October 6th, 4008

Re:Anonymous Coward

[Anonymous Coward]

No, intel will be releasing a server-farm model that only needs a tent...

Re:Anonymous Coward

[The Master Control P]

Oh, we only wish. Weather is a chaotic system, which means that "nearby" numerical solutions diverge from each other in the system's phase-space at an exponential rate.

Thus, running a simulation longer requires an exponentially shorter timestep to keep the errors supressed. Worse, it also demands exponentially more accurate initial conditions since any initial errors are amplified. I doubt we'll ever see serious "the weather will be..." forecasts for more than a few weeks, mainly due to the IC problem.

Benchmark your application

[straponego]

A quality HPC vendor will give you the opportunity to benchmark your application before you buy a system or cluster. Most will have standard codes installed, but you should also be able to arrange for a login to build and run your own code on their test clusters. This is the only way to guarantee you're getting the best bang per buck, because the bottleneck in your particular applications may be memory, IO, interconnect, CPU, chipset, libraries, OS... An HPC cluster can be a big purchase, and it performance and reliability can make or break careers. Don't trust generalized benchmarks unless you know that they accurately reflect your workload on the hardware you'll be purchasing.

I agree

[friedmud]

I write massively parallel scientific code that runs on these supercomputers for a living... and this is what I've been preaching all along.

The thing about RoadRunner and others (such as Red Storm at Sandia) is that they are special pieces of hardware that run highly specialized operating systems. I can say from experience that these are an _enormous_ pain in the ass to code for... and reaching anything near the theoretical computing limit on these machines with real world engineering applications is essentially impossible... not too mention all of the extra time it costs you in just getting your application to compile on the machine and debug it...

My "day-to-day" supercomputer is a 2048 processor machine made up of generic Intel cores all running a slightly modified version of Suse Linux. This is a great machine for development _and_ for execution. My users have no trouble using my software and the machine... because it's just Linux.

When looking at a supercomputer I always think in terms of utility... not in terms of Flops. It's for this reason that I think the guys down at the Texas Advanced Computing Center got it right when they built Ranger ( http://www.tacc.utexas.edu/resources/hpcsystems/#constellation ). It's about a half a petaflop... but guess what? It runs Linux! And is actually made up of a bunch of Opteron cores... the machine itself is also a huge, awesome looking beast (I've been inside it... the 2 huge Infiniband switches are really something to see). I haven't used it myself (yet), but I have friends working at TACC and everyone really likes the machine a lot. It definitely strikes that chord between ultra-powerful and ultra-useful.

Friedmud

Re:I agree

[Abreu]

My "day-to-day" supercomputer is a 2048 processor machine made up of generic Intel cores all running a slightly modified version of Suse Linux.

We all envy you.

Re:I agree

[CronoCloud]

The thing about RoadRunner and others (such as Red Storm at Sandia) is that they are special pieces of hardware that run highly specialized operating systems.

from http://en.wikipedia.org/wiki/IBM_Roadrunner

The Roadrunner uses Red Hat Enterprise Linux along with Fedora as its operating systems and it's managed with xCAT distributed computing software.

Re:I agree

[blair1q]

Your users don't really need that kind of power, then. They could get by using much less of a computer, because they're spending less time coding and have time for running. In fact, I suspect about 70% of your users could get by on a 4x4 card if they ran it and went on vacation for the summer.

The really big HPCs can tolerate the really big software development efforts because the runtime saved by the specialized OS may more than discount the extra development effort.

Re:I agree

[SL Baur]

My "day-to-day" supercomputer is a 2048 processor machine made up of generic Intel cores all running a slightly modified version of Suse Linux. This is a great machine for development _and_ for execution. My users have no trouble using my software and the machine... because it's just Linux.

I am in awe of you, sir.

May I ask out of curiosity, how long it takes to compile the kernel on such a magnificent beastie?

Re:I agree

[jd]

If they'd finished the Berlin GUI, you could have played one hell of a game of Quake.

Re:I agree

[Profane MuthaFucka]

I envy his boss. He doesn't have to write any code, but he can go to supercomputer conferences and talk about all the neat things he's doing on his supercomputer.

Re:I agree

[Anonymous Coward]

Not as fast as you might think, and besides you'd have to submit it as a batch job. Relatively few CPUs are dedicated to interactive use on such systems (if any at all).

(Not the OP, but used to admin a very similar system.)

Re:I agree

[Anonymous Coward]

Berlin. Ha! That goes back to the furthest reaches of my memory.

But then I'm really drunk right now

Re:I agree

[Anonymous Coward]

Yes, but Roadrunner uses the new Cell SPUs - very different hardware from a typical x86-based cluster.

Red Storm, if I recall correctly, runs a minimalist Linux kernel - something called Catamount, or Compute Node Linux, or something along those lines. Basically, it lacks some features of a full Linux kernel, including various I/O capabilities, but frankly I haven't heard of people having a hard time using Red Storm.

Re:I agree

[speedingant]

Its only 3 in the afternoon!!

Re:I agree

[Anonymous Coward]

It takes about as long to compile a kernel as on any other system. You don't have to submit it as a batch job.

These machines, while having somewhat specialized hardware, are basically just lots and lots of computers with a fast interconnect.

You build a kernel and copy it out. It's not a big deal.

Re:I agree

[friedmud]

Not very long with distcc....

Re:I agree

[friedmud]

"he really big HPCs can tolerate the really big software development efforts because the runtime saved by the specialized OS may more than discount the extra development effort."

The thing is... that might be true for a very select number of projects... and that number of projects is actually smaller than most people think.

Yes, my code can run up to 3 or 4 thousand processors nicely... to get some really high fidelity engineering simulations... _but_ most people are running in the 64 to 128 processor range day to day and only very rarely need the kind of fidelity that comes from half a petaflop or more.

And therein lies the rub: While "potato-flop" computing sounds great... there really just aren't compelling reasons to move to it. The best justification is in analyzing natural phenomenon such as the weather. For engineering simulation (which is what LANL is all about) there are so many sources of error, that running these high fidelity simulations actually doesn't mean anything. A lot of the time the codes that "really make use of the supercomputer" are complete pieces of crap that take _many_ shortcuts in order to make sure that they execute efficiently on these oddball architectures.

I don't mean to sound all down on it... afterall this really is how I make my living. I just mean that from the outside looking in it's hard to see that Flops != good.

Friedmud

Re:I agree

[friedmud]

I will just say that it was about a year before any meaningful calculations were run using Redstorm...

Friedmud

Re:I agree

[SL Baur]

The only time I was employed doing kernel hacking was at NEC and the "big" machine they gave me for testing had ~1GB with 2 CPUS (in 2002). Sigh. Gone are the days...

Re:I agree

[I cant believe its n]

Its only 3 in the afternoon!!

Early bird catches the worm

Re:I agree

[Anonymous Coward]

You don't have to submit it as a batch job.

You do if you want to parallelize the compile (-j) using any significant number of CPUs. Or if your system allows people in interactive CPU sets to monopolize as many CPUs as they wish, it's probably horribly misconfigured. :)

But the point is that even if you specify a very large '-j', it still doesn't compile all that fast.

Re:I agree

[convolvatron]

this is another symptom of the kind of sleeze that drove intel out of the market for ascii red.

obviously there were some technical delays. cray screwed up the control system, and sandia's insistence on certain unfavorable design decisions increased the time from first boot to running application quite a bit (primarily in the communications implementation, the fabulous* catamount runtime, and the whole i/o fiasco)

but the real game that was going on was that by screwing the project schedule, sandia was able to threaten cray with breach of contract. given government contracting rules this would have been fatal for cray. by messing with the project they ensured that cray would be on the hook for 'additional consideration', which took the form of several upgrades. the largest of these was a whole fifth row.

then they got to turn around and claim to be heroes to the DOE

Non-story...

[Kjella]

...ever looked at gaming benchmarks? Server benchmarks? Productivity benchmarks? Rendering benchmarks? In fact, any kind of benchmark? Seen how they all differ depending on the product and test run? Same with supercomputers, you got some synthetic benchmarks, and you got some real world benchmarks. But the weather simulation may not be a relevant benchmark at all if you're doing nuke simulations or gene decoding or finite deformation or some other kind of simulation. Synthetics are the lowest common denominator - you'd rather see benchmarks in your field, and most of all benchmarks with your exact application. That doesn't change that those are individual wants and synthetic benchmarks are the only ones with any value to everyone.

Re:Non-story...

[blahplusplus]

"...ever looked at gaming benchmarks? Server benchmarks? Productivity benchmarks? Rendering benchmarks? In fact, any kind of benchmark? Seen how they all differ depending on the product and test run? Same with supercomputers, you got some synthetic benchmarks, and you got some real world benchmarks."

I hear what you're saying, but if we look at application complexity (modern games for instance) there really there isn't any such thing as "real world benchmarks" since each application is specific because of the design. Design is just as important as the hardware, you can have a really fast computer and a bunch of bloated code running on it. My guess is that a lot of modern apps are very bloated.

I'd love to see some supercomputing power dedicated to reducing bloat and optimization efficiency in code/languages/compilers, etc.

The Turk..

[stevedmc]

Isn't "The Turk" supposed to be the worlds most powerful computer?

Like any benchmark...

[Junta]

Just with a lot more dollars behind it...

Every one remotely engaged in Top500 systems knows how very specific the thing being measured is. It's most sensitive to the aggregate clock cycles and processor architecture, and not as sensitive to memory throughput/architecture or networking as many real world things are.

http://icl.cs.utk.edu/hpcc/

Is an attempt to be more comprehensive, at least, by specifying a whole suite of independently scored benchmarks to reflect the strengths and weaknesses of things in a more holistic way. Sure, it's still synthetic, but it can give a better 'at-a-glance' indicator of several generally important aspects of a supercomputer configuration.

The thing probably inhibiting acceptance of this is that very fact, that it is holistic and the winner 'depends' on how you sort the data. This is excellent for those wanting to more comprehensively understand their configurations standing in the scheme of things, but hard for vendors and facilities to use for marketing leverage. Being able to say 'we built *the* fastest supercomputer according to the list' is a lot stronger than 'depending on how you count, we could be considered number one. Vendors will aggressively pursue pricing knowing about the attached bragging rights, and facilities that receive research grant money similarly want the ability to make statements without disclaimers.

Rest assured, though, that more thorough evaluations are done and not every decision in the Top500 is just about that benchmark. For example, AMD platforms are doing more strongly than they would if only HPL score is counted. AMD's memory performance is still outrageously better than Intel and is good for many HPC applications, but Intel's current generation trounces AMD in HPL score. Of course, Intel did overwhelmingly become popular upon release of their 64-bit core architecture based systems, but still..

Is it *REALLY* the Top 500?

[Anonymous Coward]

Most of the locations listed are mostly educational institutions, r&d centers, and computer companies. The results were probably submitted unofficially. There are few exceptions, but they are just that--few. It makes you wonder what the Big Data companies (Google, Yahoo!, etc) actually have running. They have no reason to participate, after all...

Consider something like Yahoo!'s research cluster. Why isn't it on this list? Why don't they run the tests?

Thats not a "barrier"

[quenda]

computings Mount Everest - the petaflops barrier

Two bad cliched metaphors in one! Its not a peak, and its not a barrier, just another arbitrary milestone. Who writes this crap?
Oh ... a "professional" writer from an industry magazine. That figures.
This guy should enter the The Bulwer-Lytton Fiction Contest

Re:Thats not a "barrier"

[im_thatoneguy]

True. And if anything it should be the K2 of computing.

The real problem is once you've overcome the highest peak does that mean all computing is down hill! Hurray for bad analogies!

Re:Thats not a "barrier"

[JustOK]

you're making a mountain of a metaphor

Re:Thats not a "barrier"

[morgan_greywolf]

No kidding. A petaflop ought to be enough for anybody!

(SCNR)

How appropiate,

[Abreu]

That an article about featuring IBM supercomputers comes shortly after a few misguided individuals were posting that "IBM is no longer relevant, they are a OEM reseller nowadays" or that they "only make bloated, slow software"

Well, let's see

[Louis Savain]

It's about a half a petaflop... but guess what? It runs Linux!

This sounds kind of nice but why should this make it any easier to write parallel programs for it? You still have to manage hundreds if not thousands of threads, right? This will not magically turn it into a computer for the masses, I guarantee you that. I have said it elswhere but parallel computing will not come of age until they do away with multithreading and the traditional CPU core. There is a way to build and program parallel computers that does not involve the use of threads or CPUs. This is the only way to solve the parallel programming crisis. Until then, supercomputing will continue to be a curiosity that us mainstream programmers and users can only dream about.

Re:Well, let's see

[Varun Soundararajan]

There is a way to build and program parallel computers that does not involve the use of threads or CPUs.

Agreed about threads, but CPUs? Are you going to run your calculations on a bacteria?

Re:Well, let's see

[friedmud]

"Until then, supercomputing will continue to be a curiosity that us mainstream programmers and users can only dream about."

I'm not so sure that's a bad thing... most applications don't need the power of a super computer...

At the same time, I agree that I wish that desktop development tools made it easier to do threading for multi-core machines. Every new computer comes with more than one core... but the development tools (languages, compilers, IDE's, debuggers) simply aren't helping the everyday joe programmer out there make use of them...

Friedmud

Re:Well, let's see

[Louis Savain]

Agreed about threads, but CPUs? Are you going to run your calculations on a bacteria?

Funny but no. There are other types of processors besides the CPU. The GPU comes to mind. My personal favorite is a pure MIMD vector core but you're not going to find too many of those around.

Re:Well, let's see

[dkf]

This sounds kind of nice but why should this make it any easier to write parallel programs for it? You still have to manage hundreds if not thousands of threads, right? This will not magically turn it into a computer for the masses, I guarantee you that. I have said it elswhere but parallel computing will not come of age until they do away with multithreading and the traditional CPU core. There is a way to build and program parallel computers that does not involve the use of threads or CPUs. This is the only way to solve the parallel programming crisis. Until then, supercomputing will continue to be a curiosity that us mainstream programmers and users can only dream about.

You're wildly mistaken, but for interesting reasons. While SIMD architectures are indeed interesting and do very well on some types of problem (I believe that weather simulation is good) on other problems it is far easier to express them as either MIMD or something more exotic. A real system that needs to handle a mix of codes needs to be implemented as a hybrid: e.g. a cycle-scavenged Condor pool for high-availability, a traditional cluster with a better interconnect, and a supercomputer for those times when you really need the grunt-power. Sure that's not a one-size-fits-all situation, but since when did any reasonable software engineer ever expect that to be the case?

(I don't remember exactly where I read it, but there was a paper fairly recently that characterized 13 fundamentally different types of computation, from embarrassingly parallel to embarrassingly sequential. Hardware architectures that are tuned for one of those cases can do really badly on others.)

Re:Well, let's see

[jstott]

You still have to manage hundreds if not thousands of threads, right?

Don't think of these things as 2048-core CPU's sharing memory and disk space, think of them as clusters of 2048 separate [asnychronous!] PC's, each with its own memory and disk space. The actual architectures will vary, but this is usually the right paradigm (Crays and other big vector processors are the main exception to this rule). Think, then, of the program as if it were a bunch of network clients talking to each other over sockets. Because each CPU has its own memory, if you want to know what your neighbor is doing, you have to go ask him and get him to send you the data you're interested in over the network, so fast network interconnects are essential. Same goes for disk writes -- its a shared resource accessed over the network, so bandwidth and networking matter.

This is, in fact, how things like the MPI library are implemented (MPI is a very standard library for parallel programming, Apple's XGrid is a variation on this theme too). The library takes care of starting up processes on each node (using rsh/ssh) and all the network communications for you. It also implements primitives like scatter-gather, send-receive, broadcast, and barriers. The hard part is understanding parallel programming, the internal trade-offs of your hardware, and the available algorithms well enough to write a well-designed and efficient parallel implementation.

Having said that, you're right, these machines are not for the average user, nor should they be. The average user doesn't need a supercomputer, the average user (or web farm, for that matter) is best served by a computer designed for their needs, specifically, an off-the-shelf PC. Supercomputers are designed for solving large-scale computational problems that significantly exceed the capacity of a desktop computer and are properly matched to a given supercomputer architecture.

-JS

RoadRunner

[Anonymous Coward]

Is running relatively stock Fedora (the ppc distribution). True, it's ram hosted, but the OS is hardly specialized in terms of libraries and such. You could say the Cell SDK is a tad specialized, but the underlying platform is not so custom as implied.

In fact, every single Top500 system I've ever touched has been far more typical linux than most people ever expect.

In any event, the most compelling aspect of RoadRunner in my view is the flops/watt. Application developers who can leverage highly parallel clusters are those who have the best shot of taking adequate advantage of something like the cell architecture, which is admittedly a pain for those that are accustomed to no more than 1 or two concurrent heavy-loaded processes or threads.

BTW I still hate the Infiniband cabling with a passion. Even as they've made it less bad over time, it's still a huge connector. Nothing like Quadrics, mind you, but still reminiscent in bulk to 10base5.

Re:RoadRunner

[friedmud]

The specialization of the hardware / software combo is what I was referring to.

Have you ever coded for one of these special architectures? It really is a bitch. Yes, Redstorm is even worse (special OS that doesn't even allow dynamic linking!)... but the non-generality of the cell-processors is going to kill the real world impact of Roadrunner.

ASCII Purple was one of the previous machines at LANL that was a "one-off" build from IBM. It was a complete disaster. Porting code to the machine took much longer than usual and any person who could show that they were successfully running _anything_ on the machine got a pat on the back. I had the luxury of porting some software to it... good god, just thinking about it makes me want to blow my brains out...

I can't believe they've gone down that same path again.

Friedmud

Re:RoadRunner

[friedmud]

Sorry... got my supercomputers mixed up... ASCII Purple was at LANL...

I was thinking of ASCI Q, but that was made by HP...

Oh... just nevermind... I screwed it up well enough, just forget it ;-)

Need to get some sleep.

Friedmud

Re:RoadRunner

[joib]

Hasn't Red Storm switched to Linux yet? Seems most of the Cray XT sites have done so already. One reason for developing CNL (Compute Node Linux, the Linux kernel they run on the nodes) IIRC was that multicore support in Catamount was sort of a kludge. And OS buffering, and..

Then again, the catamount developers are at Sandia, so maybe they have some emotional attachment to it. :)

CNL still doesn't allow dynamic linking, though it's much less alien than catamount.

Re:RoadRunner

[Anonymous Coward]

Nothing like Quadrics

QsNet II cables are "chunky", yes. QsNet III will be much better.

Re:RoadRunner

[friedmud]

As far as I know (as of 3 months ago) they're still running Catamount at Sandia... and it's for the reason you state: they developed it.

Friedmud

Re:RoadRunner

[Anonymous Coward]

they wasted a huge amount of cray's time on that idiotic little pseudo-kernel. cray even had to give them a kickback by hiring them to port it (a task which cray had to do anyways)

when one of their own people discovered that gnu malloc was measurably faster than their special 'supercomputing malloc', the project switched libraries...but of course it was forced to switch back.

its just corruption

Uhh, do you have a model?

[Mike Rice]

"State-of-the-art systems today can simulate about five years per day of computer time, he says, but some climatologists yearn to simulate 100 years in a day."

IANAM (I am not a meteorologist) like Mr Loft, so excuse me please if I am wrong, but does not the current state of the art in weather modeling provide something like a 3 day preview of the future, with only 50% accuracy?

I submit that Mr Lofts complaints have much more to do with the current mathematical model limitations, than with the ability of current hardware.

This is not a hardware issue (yet). This is still a mathematical issue, and has not one iota to do with the prowess of any computational hardware.

When Mr Loft presents a 100 year weather model with 50% accuracy, I might begin to worry about whether our CPUs can handle it.

Until then I keep my eyes glued on the Weather Underground.

Re:Uhh, do you have a model?

[TapeCutter]

I just threw away a couple of mod points to bring you this announcement: Climate != weather, climate is the long term statistics of weather. Two different numerical analysis models, both computationally expensive.

Re:Uhh, do you have a model?

[Waffle Iron]

IANAM (I am not a meteorologist)

That's for sure.

Here's an analogy: Say you pour two different colored cans of paint into a bucket and start stirring. Weather is like predicting the exact patterns of swirls that you'll see as the colors mix. Very hard to do looking ahead more than a couple of stirs.

Climate is more like predicting the final color that will result after the mixing is done. Not nearly so intractable. The summary is talking about climate, not weather.

Re:Uhh, do you have a model?

[Anonymous Coward]

Here's an analogy: Say you pour two different colored cans of paint into a bucket and start stirring.

Red is like the color your ass is going to be when your dad finds out you've wasted two cans of paint. Period!

Re:Uhh, do you have a model?

[sumdumass]

The problem is that separating climate from weather doesn't mean much when doing predictions. You still have to predict the weather to provide data points for the climate. Otherwise, you can't have long term statistic of weather patterns in the future that were predictions of today. And you can't make claims about the weather in the future from the climate predictions.

So all future climate predictions will inherit inaccuracies from weather prediction. They are not separable even when you show how they aren't the same. Well, that is unless climate predictions throw the entire long term statistics of weather out the window but then it wouldn't be climate according to your own definition.

Re:Uhh, do you have a model?

[Anonymous Coward]

That's for sure.

Thanks for showing your superiority. That's fine and all but could you please rephrase in the form of a car analogy?

Re:Uhh, do you have a model?

[TapeCutter]

"And you can't make claims about the weather in the future from the climate predictions."

Can't I? - On average the weather will be colder in winter 4008 than it will be in summer 4008.

The rest of the logic in your post is upside down, however we have crossed swords before and I have (in the past) provided you with relevant links that you are still choosing to ignore.

Re:Uhh, do you have a model?

[sumdumass]

Can't I? - On average the weather will be colder in winter 4008 than it will be in summer 4008.

Are you sure about that? You see, you opened a hole so broad that your statement isn't accurate even today. It's always summer and always winter on the earth. So it could be the same temperature in summer and winter in 4008.

Also, you didn't explain what future climate predictions you came to that conclusion over. I might have not been specific enough for a fan boy like you, but I was making the claim to future weather predictions from climate predictions. Of course a prediction is an attempt to explain what the future will be like so it is in the future too. You can conceive the seasonal climate differences in a particular spot from historical reference but not future climate predictions. In other words, you can't claim the climate will be X in the future and then make a claim about the weather from X. Or do you know something the rest of us thinking individuals don't? I mean how do you come to the conclusion that summer will not be as cold as winter by using climate predictions alone?

The rest of the logic in your post is upside down, however we have crossed swords before and I have (in the past) provided you with relevant links that you are still choosing to ignore.

That's a cop out. I really wish you guys wouldn't get your panties in a knot when someone questions the premise of your faith. The only flaw in my logic is where it hampers with your beliefs.

And the site your talking about uses some false logic and logical fallacies in and of itself. I remember it, in one article, attempting to reference a claim that was recently refuted in order to refute the claim that just refuted the previous claim. Yes, your head should be spinning by now. It's like saying your wrong because of this stuff that your claiming is wrong shows something different. But that's what I would expect from a site pioneered by a NASA scientist who said he knew information he was using was flawed but "exaggeration by scientists had its place when it was necessary to mobilize public opinion." Of course the father of global warming also published his first climate model claiming global cooling was a threat in 1971. And to make things even worse is the political hijacking of the issue and almost all of it's purposed solutions. But like you've said, we have had this talk before.

Re:Uhh, do you have a model?

[TapeCutter]

Can't I? - On average the weather will be colder in winter 4008 than it will be in summer 4008.

Are you sure about that?

No, in fact my faith in science as the provider of the best available explaination for systematic observations of the natural world means that I'm not even sure the Sun will rise tomorrow BUT it my faith also tells me not worry about it.

Absolute certainty of future events is a sport played by politicians and opionion columnists and I suggest that is why you insist on using their stale arguments in every post you make concerning the climate. Perhaps you hadn't noticed but these people themselves have now largely abandoned those twisted factoids and begrudgingly accepted the two main points of the IPCC consensus. Your continued inability/refusal to look at the evidence in a non-political manner belies your ranting.

- You've been fed, now get off my lawn!

Re:Uhh, do you have a model?

[sumdumass]

No, in fact my faith in science as the provider of the best available explaination for systematic observations of the natural world means that I'm not even sure the Sun will rise tomorrow BUT it my faith also tells me not worry about it.

I wasn't talking about your faith in science. I am talking about your faith in global warming and how it has to be true.

Absolute certainty of future events is a sport played by politicians and opionion columnists and I suggest that is why you insist on using their stale arguments in every post you make concerning the climate.

I'm wasn't talking about absolute certainty. I was talking about the ability to predict climate independent of weather. It can't be done which was the point of my post and the reason why separating climate from weather has no bearing on the accuracy claims.

Perhaps you hadn't noticed but these people themselves have now largely abandoned those twisted factoids and begrudgingly accepted the two main points of the IPCC consensus. Your continued inability/refusal to look at the evidence in a non-political manner belies your ranting.

Actually, you don't have evidence in a non-political manner. That is the problem. Hansen fought forever to keep his evidence secrete. It wasn't until someone collected enough of their own to check and find something was wrong before they started opening it up. I have heard the arguments about peer reviewed and all but when a lowly blogger attempts to get the data, he runs into road block after road block, lost data sets, and all of the rest. You have heard the saying, Garbage in Garbage out, well we have no guarentee that there isn't garbage in and until we do, all the work from the political organization, the IPCC, has to be questioned. But, as you will notice by using your expert scientific mouse clicker and go back, I have said nothing disputing global warming now have I. But somehow, pointing out that separating climate from weather with a definition that locks them together and blasting a biased site who's main contributor claims it is perfectly fine to exaggerate because the ends justify the means, you now have me denying science altogether. Well, here is a hint for you, Not trusting the source of something say nothing about it at all. I don't get how if we don't just accept what your deity says without question, we are somehow non-believers and committing blasphemy. In fact, that is so wrong that I'm not sure you can even claim to be scientific in your beliefs. The basic premise of science is to question the answers and test everything.

Re:Uhh, do you have a model?

[Anonymous Coward]

> Climate is more like predicting the final color that will result after the mixing is done. Not nearly so
> intractable. The summary is talking about climate, not weather.

With the difference that you are red green blind, and there are actually 20 buckets in the mixing process of which you can only see two.

Re:Uhh, do you have a model?

[TapeCutter]

HaHa, I got you to copy my argument and mod yourself up with a puppet!!! Who's trolling who baby... ;)

Re:Uhh, do you have a model?

[sumdumass]

I don't have puppet accounts.

But I do find it odd that instead of addressing anything said, you jump to accusations. For all I know, you could have modded it up yourself with one of your puppet accounts just so you can escape the scrutiny I put forth.

But that's OK, We have had discussions in the past, I never have seen you act this way, but avoiding the issue while pointing to irrelevant information seems to be something we are used to. Don't worry, I don't hold it against you.

Accuracy?

[Hatta]

Number of simulated years per day isn't exactly the metric you want. I can simulate a million years in a minute on my home pc, just not very accurately. As you get more accurate, the sim years/CPU day will decrease.

So knowing the number of simulated years per cpu day doesn't tell you anything unless you know exactly what algorithm you're using.

Re:Accuracy?

[Anonymous Coward]

So knowing the number of simulated years per cpu day doesn't tell you anything unless you know exactly what algorithm you're using.

Which of course is current climate models. There are a number of "standard" models being used these days and that's what's being talked about here.

Cost per computing power

[pimpimpim]

Not just buying the thing, also the cost of maintaining it (standard hardware is most likely easier to maintain), and the power it and its cooling system uses, check green500 for that one. Actually, as a user, I have often found that most supercomputing clusters are inefficient for at least the first year-and-a-half due to imperfect queuing systems or network/filesystem incompatibilities. "Yeah, your run will likely crash every now and then but we don't know yet why". I do not blame the administrators, I blame the suppliers for not working on solutions to make it easier to successfully operate a cluster, e.g. via standardized methods .

As for top500: really, quit with this political joke benchmark. E.g. In molecular simulations alone you will spend on a computer which has either broad memory access for matrix inversions in quantum calculations. Or on a high clock speed, low bandwidth one for MD, which basically does nothing but floating point operations. The score in the top500 will give you 0 information about what machine to choose.

Something like

[Luthair]

90% of Roadrunner CPU time is reserved for the military as I recall.

Re:Something like

[Anonymous Coward]

90% of Roadrunner CPU time is reserved for the military as I recall.

Not quite. RoadRunner _is_ used mostly for nuclear weapons work (called "Stockpile Stewardship" in politico-speak) but the DoD has very little to do with it. They have their own supercomputers. Los Alamos is a Dept. of Energy lab.

#1 is IBM's Roadrunner

[Anonymous Coward]

Great!

But will it run Crysis?

Fortran?

[actionbastard]

Why don't they code LINPACK in COBOL?

I think it is in phases..

[Anonymous Coward]

IBM retains their engineering, but sometimes the business decision makers fail to understand the value of that, and try to profit by slapping IBM logo on other company's products.

Let's presume we start with an outsourced generation of products. IBM does it and gets slapped with warranty/service costs/get complaints from the services organizations saying they cannot build quality solutions on random white-box systems, and generally customers see little product differentiation from the companies whose products IBM are reselling at increased price.

Then they have a generation of good, IBM-engineered product that uses their in-house engineering teams to produce product. The products work well, and the customers and IBM services business can consistently build upon them.

Then, they decide it's going so well, if only they could cut costs by outsourcing, and the vicious cycle continues.

Re:Exactly. HP PA-RISC vs. 21364 Alpha vs. UltraSP

[ari_j]

I was actually referring to the difference between read as in the system call and read as in what you do with a book.

This is common knowledge

[Raul654]

It's fair to criticize Linpack for being a one-trick pony. It measures system performance for dense linear algebra, and nothing else. Jack Dongarra (the guy who wrote Linpack and maintains the top 500 lists) is quite up-front about Linpack's limitations, and he thinks that using a single number as the end-all-be-all of a computer's performance is a bad idea. It's a simple fact of life that certian kinds of computers do better on certain problems. The good guys out at Berkeley even sat down a couple years ago and enumerated all of the problems they found in real-world HPC applications (See the tables on pages 11-12). The real truth here is that people should stop treating Linpack like it's the final measure of system performance. If you are doing pure linear algebra problems, it's a pretty good measurement for your purposes; if you are not, then you use it at your own peril.

Oblig

[PietjeJantje]

They should just run Vista on them to use as a benchmark. That will effectively flood bottlenecks on all kinds of levels.

Re:Oblig

[setagllib]

What's even funnier is the reality that Windows can't even boot on that many CPUs, let alone scale to utilise them.

Please stop linking to Wikipedia

[Anonymous Coward]

We need a new moderation: -1 Wikipedia Googlebomb. Yes we know you can look things up in Wikipedia. But every time you make a link to Wikipedia from Slashdot, Wikipedia goes up in the Google Page Rankings. And then people act all surprised when Wikipedia is in the top ten for every Google search. Every time you link to Wikipedia, it gets a little bit more powerful.

So instead, why not link to some other relevant page? In this case, link to the owner of the Roadrunner supercomputer. You can probably even go to Wikipedia to get the link. If Wikipedia has a great page on something, don't link to it, just put the plaintext name in like this: "Search for IBM_Roadrunner on Wikipedia."

Please everybody, stop linking to Wikipedia. You're destroying the internet.

Re:Please stop linking to Wikipedia

[bilby727]

If Wikipedia has a great page on something, don't link to it

Tell people to search for a page instead of directly linking to it? Hyperlinks are a fundamental part of the web. Now who is "destroying the internet"?

Re:Please stop linking to Wikipedia

[cerberusss]

That's a bit ridiculous isn't it? The wikipedia page is very useful. Who cares if it's in the top ten, if it's not useful then:

a) Google will do something about it

b) People can look at the other nine results

You can simulate 100 years/day on your laptop

[HoneyBeeSpace]

The thing about climate models is that they get more complex and higher resolution as soon as the computers get faster. We will always take about 3 months to run a simulation. You can run it faster? Make it more detailed. It takes longer? Wait a few months to a year and it'll only take 3 months. Why 3 months? Not sure. Partly because that is about the length of a funding cycle of design experiment, run it, analyze, and write it up.

If you want to run 100 years per day, you can do so with an older model. The EdGCM project has wrapped a NASA global climate model (GCM) in a GUI (OS X and Win). You can add CO2 or turn the sun down by a few percent all with a checkbox and a slider. Supercomputers and advanced FORTRAN programmers are no longer necessary to run your own GCM.

Disclaimer: I'm the project developer.

Isn't Modeling Weather Futile?

[CleverMonkey]

I seem to recall a Nova special I watched many moons ago about "strange attractors" and "fractal behavior" that seemed to indicate that for a large class of complex-valued iterative functions there was a weird phenomenon called the "Butterfly Effect". Apparently... according to this show I saw 20 years ago (and I think that Mandelbrot mentioned it in a lecture I attended a few years later), initial variables which are as intertwined as the rational and irrational numbers can have drastically divergent outcomes in these situations.

It seems that the reason that this was called the Butterfly Effect was actually because the disturbance caused by a butterfly could be enough to change the track of a massive storm some days later. ( Reference)

The fact is that the weather forecasters on the local broadcast channel are less accurate than if they always predicted sun in one study:

"The graph above shows that stations get their precipitation predictions correct about 85 percent of the time one day out and decline to about 73 percent seven days out.

"On the surface, that would not seem too bad. But consider that if a meteorologist always predicted that it would never rain, they would be right 86.3 percent of the time. So if a viewer was looking for more certainty than just assuming it will not rain, a successful meteorologist would have to be better than 86.3 percent. Three of the forecasters were about 87 percent at one day out â" a hair over the threshold for success."

(ref: http://freakonomics.blogs.nytimes.com/2008/04/21/how-valid-are-tv-weather-forecasts/)

It's a wonderful idea that we can model the incredibly complex climate of our huge planet, but I'll believe it once I can trust the weekend forecast before Friday.

Any other ideas about useful purposes to put these huge computers to? Perhaps accounting and auditing for the new Emergency Financial Legislation?

Re:Isn't Modeling Weather Futile?

[azgard]

Well, it's actually quite possible to model climate but not weather. It's an issue of scales. I won't go into mathematical argument, because I don't understand it anymore, and you probably don't want to know that either.

But look at this thing: http://en.wikipedia.org/wiki/Lorentz_attractor - it is essentially an infinite line curved into a strange "two ears" shape. The line is defined by 3 ordinary differential equations, and it is completely defined by selected point and direction in space.

The interesting part is, if you start drawing the line from two very close points and directions, the lines that correspond to them will become completely separated after some time. For example, one will go in the left ear and one into the right ear.

However, what is even more interesting, no matter the starting point and direction, the resulting line will always give you the same ear shape.

So, to sum up - if you have a starting point, but not know quite exact position and direction, then, after some time, you are unable to predict in what ear the line will be. But you are able to predict the general shape of the thing. So here you have a system which is unpredictable on detailed scale but very predictable on larger scale.

With weather, it's a bit more complex, but very similar to this thing. We are able to model weather accurately on hours scale, and also we can model accurately the climate on tens of years scale (or lower - or lower scale, the climate tends to be very similar). But we are unable to extend this scale in weather to several years, and it is (probably) not possible to extend our understanding of climate to hundreds of years. The changes in climate are much slower, so we can model it on longer time scales.

Re:Exactly. HP PA-RISC vs. 21364 Alpha vs. UltraSP

[johanatan]

Way to clarify that for the AC out in left field!! [His plug for Transmeta was complete OT, but you managed to use it as a legitimate 'question'.] Congrats! ;-)

Re:Exactly. HP PA-RISC vs. 21364 Alpha vs. UltraSP

[ari_j]

Thanks ... I think. ;-)

ok you Apple fanboi.

[Anonymous Coward]

He had to plug Transmeta because he anticipated your plug for Apple's powerpc-based "I think." That's quite a crystal ball he has. I wonder if he can predict the next 128-bit Hitachi SuperH processor comme!@# oh he's good...
 
get back to work.

dont invoke threadfights.

[tempest69]

First off... I hate threads... they are a locking nightmare once things get complicated,, finding a halting problem isnt worth the effort.

Really I'm an LAM-MPI freak.. let all the processes talk, gather data and not share memory. it skips the pthreads issue, but some would call the mpirun a thread-launcher.

Nope, mpi wont make parallel code a computer for the masses... I dont have ANY clue as to how to pull the CPU out of the equation.

However there are a BUNCH of ways that parallel code can be commoditized. Sure there are limits to what can be parallelized.. but most modern code is begging to be transformed. The OS's need some upgrading to do a slicker job, and I think that the CPU could add some parallel functionality.

Anyway "only way to solve the parallel programming crisis" seems pretty bunk to me.. of course after reading some of your links... you have some "ideal parallel" concept where in the real world "fake parallelism" is quite preferable...

Storm

Sukky hardware

[Anonymous Coward]

I remember hearing all about Cell processors. Uh Huh. IBM loves them. But they are a very very specialized beast. I hoped to use one in a PS3 as a scan line renderer. Bad idea. IBM says 'Oh, that should be wonderful'. Except they keep the code to fire up the SPE's. Without it, the Cell runs like a Pentium4 at 1.8 GHz. Problem 2: The memory 'full load' is 256 MB. You can't upgrade (there is no manufacturer that makes bigger and the stock bus won't allow more). For large data sets (large images) its untenable. It cannot deal with that much data at a time, and the data must be dealt with as a single complete piece. The new line of multi-core processors that can use many gigabytes of memory per core are much better suited to this kind of application. IBM will yelp about subdividing the data, but doing a complete rewrite of the application is a lot more work than getting a processor chain suited to the job.

This just in....

[TimeZone]

Synthetic benchmarks aren't applicable to all users! Who knew?
TZ

Cluster

[RealRav]

Imagine a Beowulf cluster of these....

Colombia is still cooler looking.

[Anonymous Coward]

The Colombia Supacomputa is still much cooler looking than this.

This far and no HG2G references

[dkleinsc]

I'll save those huge computers a few million years: The answer is 42.

Now try producing that on a mere Milliard Gargantubrain.

Chasing your own tail

[KiwiCanuck]

Build a faster computer. User's expands simulation parameters (which bogs down the faster computer). Rinse. Repeat.

Is Tom Slick in the race?

[Anonymous Coward]

'Cause if the Thunderbolt Greaseslapper ain't in it, I ain't interested.

Leave it to IBM...

[the saltydog]

Someone finally comes up with a machine that can handle Vista Ultimate... who else did you think could do it - Gateway?!?

Software

[Danathar]

Software is the weak point in supercomputing nowadays. Funding for hardware SO outstrips the development of software that runs on the expensive hardware that it's becoming a SERIOUS problem.

winner

[BenphemeR]

contest over /
apple.com

Something Doesn't Mul-Add Up Here

[Nom du Keyboard]

State-of-the-art systems today can simulate about five years per day of computer time, he says, but some climatologists yearn to simulate 100 years in a day."

Something just doesn't jib here. So it takes you 20 days instead of 1 to compute the next hundred years. Why is this a problem? It's still a hundred years in 20 days. Don't you have even a little bit of patience?

The only reason to compute a hundred years in one day is if you're going to restart the computation each morning to see what the next hundred years is looking like. But that means you're throwing away the previous day's computation each morning and starting over. Which means you believe that the previous day's computation is bad. Which means why are you even running it in the first place?

If you can't compute the next hundred years accurately anyway, why don't you work in getting the next 5 years working instead. And hey, you've already got that amount of computing power. And when you do get the thing working right then I can wait an additional 19 days for the answers to the remaining 95 years.

In short, quit yer complaining and get your models running correctly. Then you'll only have to compute it once!

I remember when a megflop was FAST

[peter303]

Moving from PDPs to VAXen. Doesnt seem all that long ago.
About 13, 14 years for each new 1000x level.

Climate change by means of simulation...

[TheCaptain]

"State-of-the-art systems today can simulate about five years per day of computer time, he says, but some climatologists yearn to simulate 100 years in a day."

Strangely enough, I'm reminded of the Heisenberg uncertainty principle. Considering the power and infrastructure requirements to run a supercomputer with that much processing power today, you might change the climate by simulating it.

12 ways to fool the masses...

[Anonymous Coward]

A group at NASA Ames Research Center created a benchmark suite in the 90's as they measured the relative performance of parallel supercomputers. This benchmark suite is perhaps more useful than LINPACK, if the kernels represent YOUR application. However, like all benchmarks (including LINPACK) companies worked hard to show their results in the "best light", sometimes unrealistically so. After watching the tricks played in reporting benchmark results this paper was written called "Twelve Ways to Fool the Masses When Giving Performance Results on Parallel Computers" crd.lbl.gov/~dhbailey/dhbpapers/twelve-ways.pdf While the units are MFLOPS and not PFLOPS the points made about what people will do to make their machine look great in benchmark results is still true today.

Why does the yearning stop there?

[GPS Pilot]

some climatologists yearn to simulate 100 years in a day

Why does the yearning stop there? Why not yearn to simulate, say, 1000 years in three seconds?