Slashdot Mirror
Supercomputing and Climate Research
Mr. Obvious writes: "It must have already been submitted, since the article is over a day old (gasp!) but there's a good
round-up
on the state of the art in supercomputing, as it applies to modeling the weather --- that is to say, modeling the planet --- over at the NYTimes. They go into lots of interesting things concerning how hard it is, what progress has been made lately, why the US researchers feel themselves to be hamstringed in comparison to those in Europe or Japan, and even into some things you probably didn't know (I didn't, at least) about the weather."
(First, a preface. I'm a member of a computational astrophysics research group. We have ported our codes to the kinds of hybir d architectures of the machines discussed here, and have benchmarked their performances. Moreover, we have previously run on vector machines, so we have a fair idea of the pros and cons of the two approaches.)
While zavyman points out the basic problems inherent in parallelizing any discretized numerical model, the problem in obtaining good performance on hybrid architectures like the IBM SP-2s and SGI Origins which currently top out the top 500 list goes much deeper.
First, these machines are built around a hybrid architecture. Each node has a few processors (typically between 4 and 16, depending on the model), which utilize shared memory. These nodes connect to one another via an internode interconnect, with relatively modest bandwidth.
While this hybrid architecture allows supercomputer manufacturers like IBM and SGI to scale into the thousands of processors, it also introduces a substantial complexity into building of high-performance codes. Ideally, one would like to run threads-based parallelization on each node, and MPI between nodes, though the reality is that most codes in use use only MPI.
One can get decent scalability (into the hundreds of processors) when one runs physical models with limited communication -- ie, which simulate hyperbolic PDES like those of hydrodynamics (as zavyman describes above). However, things become more interesting when one considers more varied physics, such as that involved in solving elliptic PDEs (such as Poisson's equation for self-gravity or electrostatics). Because elliptic equations connect everything with everything else on the spatial domain, the communication costs ARE MUCH HIGHER. It is extremely challenging to build a multiphysics code with such varied parallelization demands. Indeed, it is a fair statement that no one has yet achieved excellent performance on anything close fo the thousands of processors available on these hybrid machines. For instance, another poster describes a climate model available from another research group. However, if you dig deeper, you find that they state,
"ForesightWX uses an IBM 12-node system with 52 processors working 24 hours a day. The cluster fits snuggly in a small room. A decade ago the same power would have filled the building."
52 processors is a far cry from the thousands of processors available to the users of these machines. Since each processor is slower than a vector processor like the Cray (by about a factor of 3 - 5), and assuming ideal speedup, such modest levels of parallelization lead to speedups of about 10-15 relative to a single Cray T90 processor. It is quite evident that there is little net gain over running the same simulation on 8-16 T90 nodes.
Moreover, due to the hardware constraints described above, IT MAY VERY WELL BE THE CASE WE NEVER SEE EXCELLENT MULTIPHYSICS PERFORMANCE ON THEM.
(One can get better parallel performance by increasing the problem size, but as the article points out, doubling the resolution of a simulation increases the cost by a factor of 16; hence, simply increasing the problem size may lead to unacceptably long computation times.)
I think massively parallel architectures will ultmately be the wave of the future, but there is little getting around the fact that the current generation of IBM-SP2s are dogs in the performance category.
Bob
Science, like Nature, must also be tamed, with a view turned towards its preservation.
This is a monte-carlo approach (hence the name Casino-21). Each machine will run a completely *independent* climate simulation with no interaction with anyone else's machine. The point being that each simulation is set of with a slightly different set of options on the "control dials" of the model. The big ensemble of results will then help scientists determine the sensitivity of the climate to different effects.
OK, Moore's law will solve a lot of the problems, but not all of them. One of the big problems is gathering input data! We have this huge system to model, and we only have datapoints every few hundred miles. The air column goes up 10s of thousands of feet. Even if the govt put a gound station on a grid of 10 miles on a side, you still have to send up weather ballons to get readings of the air column (Temp, humidity, winds aloft etc) all the way up.
So, there are HUGE holes in the data. Makes it hard to make a model
-- 73 de KG2V For the Children - RKBA! "You are what you do when it counts" - the Masso
This graph is one of the scariest things I have seen in a long time. It's a plot of the temperature variations and CO2 levels over the last 500,000 years measured from ice cores drilled out from Lake Vostok in the Antarctic. The two series track each other incredibly closely.
As we now have good models for why CO2 should cause temperature change, but not the other way round, it is something to take very seriously.
The figure was taken from The Economist magazine, a paper not usually associated with extreme anti-business views. Two recent articles gave good summaries of our present state of knowledge about global warming, and how both the data and the models have improved over the last ten years:
- The Reality of Global Warming (14 June 2001)
- The science and politics of global warming (16 Nov 2000)
(Titles given are those used in the magazine's index of its environmental stories online.)Article on the report for the Bush administration from the National Academy of Sciences.
Background piece before the last 'Kyoto' meeting in the Hague
One worrying new possibility is that there may be an abrupt change (bifurcation) in the ecosystem response as the temperature rises. At the moment about 50% of the manmade CO2 emissions are being absorbed by the Amazon rain forest. But the latest Hadley Centre models predict that if the temperature continues to rise, this greatly increases the frequency of much drier weather in this region, causing the forest to dry out, ultimately leading to uncontrollable forest fires. This would release vast amounts of more CO2 into the atmosphere if the whole lot went up -- perhaps ten times as much as human activities.
(And that is not the ultimate nightmare positive-feedback scenario, which is the enormous amounts of methane hydrate locked up at the bottom of the ocean in the arctic permafrost. The only thing that keeps it stable is the high pressure and low temperature. There is thought to have been a runaway destabilisation 55 million years ago, which raised the temperature 15 degrees C in less than 20 years).
I suppose somebody might come up with a techno-fix solution. But the complacency of gambling on that is like playing Russian roulette with five of the six chambers loaded.
I dount that the situation has changed remarkably since then. One thing that I'm sure hasn't changed is that there is no shortage of really solid data to support both sides: that the temperature really has risen, and that it really hasn't. There are thousands of temperature time series, some direct and some inferred, some are climbing, some are falling, and most aren't changing significantly after controlling for all the relevant sources of variance.
Globally it is likely that the 1990s was the warmest decade and 1998 the warmest year recorded (since 1861). Certainly this seems to be the case in the northern hemisphere not simply since 1861 but for the last ten centuries.
Yep, I hope so. We are still coming out of a little ice age, returning to the higher temperatures which were the norm when the Vikings grew grapes in Newfoundland. The scary thought is that we might find out, in 100 years, that the temperatures are really going down.
You point out that the EPA and UN-funded scientists have found evidence of global warming. Notice where their funding comes from. If Exxon was paying the bill, these same guys would no doubt have found the opposite. Government and industry researchers don't get tenure.
There are literally thousands of responsible scientists who work in these fields who believe that any sort of costly action to "avert global warming" is a bad, irresponsible idea. Some of them are Exxon employees, but certainly not all. Here and here (loosely related) are a couple of random links which might help make the point that it isn't a settled issue in the minds of people who understand it and aren't funded by the Government or Greenpeace (HINT: both these groups find it easier to get money from the public if they can claim that the sky is falling.)
In short, ad homenim arguments are less productive than usual here, since we see the usual suspects on each side of the issue. The energy companies are pushing their issue, Greenpeace is pushing theirs, and so on.
We need to consider the consequences of being wrong. Seeing the global temperature rise by 1 to 2 degrees C is probably going to make the world a better place to live in the long run. That's the maximum likelihood prediction from most of the models that folks on either side take seriously. The doomsday 5+degree C senarios have very low probabilities under most models.
Consider the cost of "taking action": Millions of people around the world, most of them already desperately poor, will die earlier and more miserably if we do anything to limit energy use. The only thing I can think of to reduce greenhouse gasses without causing disaster is replacing coal with nuclear power. That isn't going to happen anytime soon, unfortunately, because of the same agenda that is driving the "its getting hotter" side of the issue.
See what I've been reading.
I hate it when the press makes it sound like America is the jack-ass backwards donkey of the supercomputing world. This writer implies the Japanese and Europeans have vastly superior computing power. This is clearly the notion of a chucklehead. Take a look at The Top 500. By its (Linpack) metric, 8 of the top 10 machines are in America. Three of them are DEDICATED to weather or environmental work (Naval Oceanographic Office, National Centers for Environmental Prediction). A fourth one at NERSC is relatively open, compared to defense machines, and I'd be willing to bet weather code is running on it regularly. These are all teraflops machines. Japan has the other two in the top 10. Anybody know the job mix on those two? Europe's fastest machine is the Hitachi in Muenchen. The fastest dedicated European weather machiens are the T3Es at the Deutscher Wetterdienst and at the UK Meteorological Office.
I don't buy these whiny weathermen's complaints. The difference is that the American machines are all massively parallel machines (mostly IBM SP). The Japanese manufacturers all make vector machines, some of which the Europeans use. The Cray T3E is kind of a weird in-between architecture. It takes a good programmer to use a MPP to its full capability. The vector users, on the other hand, have 30 years of old code and practice which keeps them in the game. If the Americans would suck it up and learn to use their amazingly fast IBMs we would hear whining from the other side of both ponds. If you try to run your old code for the Cray C90 on an IBM SP, you are going to get terrible performance. If you rewrite the code, you may get great performance. But these guys aren't rewriting the code. Take for example the machines at NCEP. These create the daily production weather models used all over the US. They are IBMs which replaced a Cray that self-immolated about 1.5 years ago. When they brought the new machines up, I wonder if they rewrote the code beyond making it run? If you know, enlighten us!
- Kabauze
Nobody had the technology in 1950, and I doubt we do now, to measure the temperature around the globe and get a global, annual average accurate to a tenth of a degree. Forget it.
One other thing that bugs the crap out of me about "global warming". NOBODY EVER TALKS ABOUT CONCRETE, STEEL, AND ASPHALT!!! Hasn't ANYBODY ever noticed how hot a street or roof gets in the sun?!? I expect a few temperature measurements in growing cities would be more than enough to throw off their temperature measurements.
Then, there's the well-ignored fact that we're coming out of a mini ice-age, which peaked circa 1850. Greenland was green when it got its name, folks. The earth got colder since then and is warming back up, completely without our assistance.
And another thing... I saw just the other day that one of NASA's earth-monitoring has recorded a 30% increase in the levels of planktin in the oceans over the last 10 years. That's not a prediction, folks, that's a direct measurement. Concidering that planktin, not rain-forests as the greenies would like you to think, fix something like 70-80% of the CO2 in the atmosphere, it would appear that the earth is more than capable of absorbing whatever increase in CO2 we're providing.
Really, these global warming people sound about as rediculous as the Y2K people. The sky is falling! The sky is falling! Buy my book! I just lost a fortune in tech stocks and I need money!
If you're interested in lending a hand to such research into climate change, some folks at the Rutherford Appleton Laboratory would appreciate your help with their Casino-21 distributed client. Its still in the preparatory stages (ie client comming soon), and requires a significant investment in terms of commitment as compared to such things as SETI@home: "Casino-21 client will most likely require at least 128MB of memory, and 500MB of free disk space".
troodon.net
The article notes the objection of global warming skeptics as if there is scientific consensus that a) the build-up of so-called "greenhouses gases" causes the Greenhouse Effect (probably true) and b) that an increase in the concentration of greenhouse gases is anthropogenic (probably false):
Yes, the Earth is warming in some areas, e.g. Siberia. But, this is totally expected if you look on a geological timescale, vis-a-vis the Ice Age cycle. The debate is centered on whether or not man or natural processes (cycles of flora and fauna, volcanoes) are driving the current trend. I have not seen any convincing evidence to support the existence of anthropogenic phenomenon, and plenty to support the existence of natural phenomenon.
*** Proven iconoclast, aspiring epicurean ***
These crazy scientists are going to modify the weather pattern on Earth : as they progress in their weather simulations, they'll need more and more supercomputers, which in turn will run so hot they'll raise the temperature world-wide, which will make it harder for the scientists to simulate the weather, so they'll need more supercomputers ...etc... ARGHH, SOMEBODY STOP THEM !
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
If the Americans would suck it up and learn to use their amazingly fast IBMs we would hear whining from the other side of both ponds.
Great, what the hell do you think we are doing over at Argonne National Labs? I mean, have you tried to paralellize an atmospheric climate modeler? I don't think so. Coding for a vector-based machine is pretty straight forward. You concentrate on once machine as if it had one processor and one bank of memory, and code away, occasionally noting if your loops are not easily vectorized. The compiler magically does the rest, and your program runs really fast. That's why the Japanese machines are nice.
On the other hand, imagine designing an atmospheric climate modeler on a large cluster. The current paradigm being used and developed is MPI Let's see what you have to worry about. One, since the processors are not sharing memory, messages have to be passed between them to share memory. No biggie. But now consider that the whole atmosphere has to be broken up into pieces of a grid. On the boundaries, grid points must be shared by two or more processors. At each timestep, those points must be synchronized. Code must exist to know the processors that border one another and that know which points to share.
Now what happens if you want to combine different models together, as in the atmosphere, ocean, land, and sea-ice models. This is known as a climate coupler. Well, now you have differing grids for each of the models because they were developed independently. Now your program must handle interpolation of the grid points and must again know which processors border one another so that data is efficiently transfered. Finally, there must be decent load handling so that each processor is doing its fair share of the work.
I'm now working on the climate coupler project here at Argonne. Vector machines are quite easy to program, but we do know that they will lose out to massively parallelized clusters. It's just that the programming is much more difficult, since the messaging is always the bottle neck. Communications development follows a rate similar Moore's Law, but with a longer doubling time. For maximum efficiency, the programmer must handle the messaging model directly.
The modeling group here at Argonne understands the issue, and we are working on a general climate system to run quickly on parallized machines. No you know why you can't just do a simple code rewrite. You need to redesign the whole system.
Accelerated Climate Prediction Initiative.
as it applies to modeling the weather --- that is to say, modeling the planet
The article's a little misleading. It starts with a discussion of the weather, then moves on to discuss modelling of the climate. It's basically impossible to predict the weather -- meaning the exact temperature, rainfall, cloud cover, etc. -- more than a week in advance, because you have to specify the model with essentially infinite precision or chaotic effects take over. In fact weather prediction was one of the earliest manifestations of chaos theory. The climate -- meaning long term averages -- can exhibit stable behaviour that is possible to model in the long term. Don't look for this technology to dramatically improve weather prediction.
Toronto-area transit rider? Rate your ride.