Slashdot Mirror
Supercomputers Race to Predict Storms
pillageplunder writes "CNN has an interesting article on how different supercomputers from around the world are working to predict large storms tracks. The 3 days it takes now has been cut in half. Cool read."
Oh wait, that's the job of the Diebold supercomputer.
If you think
I'll be impressed when I see supercomputers chasing tornadoes around Kansas in rusty pickup trucks. Not before.
lysergically yours
Actually if you read the article you will realize that it only takes about an hour of number crunching, but that the three day storm path accuracy errors have been cut in half... and that 5-day forcast is getting much more accurate.
I guess we should read articles before submitting them...
D.O.U.O.S.V.A.V.V.M.
I like how the article says: Just as a 5-megapixel digital camera more accurately depicts reality than a 1-megapixel device, higher resolution grids can capture a better picture of the atmosphere and help produce accurate forecasts.
:)
Way to pitch to the high-tech crowd CNN
But....... imagine a beowulf cluster of these weather predicting supercomputers.
Chris
Yeah, the Earth Simulator http://www.es.jamstec.go.jp/ does try to do this, however it does a vast number of other things as well. The other systems focus on more specific incident (such as Ivan), thus more computing power is aim'd at a short-term problem.
Earth Simulator does - Atmosphere & Ocean Simulation, Solid Earth Simulation, Multiscale Simulation, and Advanced Precipitation Simulations. (And other cooperative projects).
I am
http://www.sgi.com/company_info/newsroom/press_rel eases/2004/june/fleet_numerical.html
Nope, the Earth Simulator is to predict overall climatic change, not specific weather conditions.
...but still not as fast as "nowcasting" (and yes, it's an actual meteorological term.) I've always wondered why the local news just has to tell us, "And in downtown it is currently raining at the moment." The people who go outside already know, and the rest of us don't care.
According to New Scientist 28/08/2004, it's a little more to do with long-term climate change, rather than predicting if you need your umbrella tomorrow in Bristol... Earthquakes and the Earth's magnetic field are also modelled too apparently...
A snip at $430 million...
No Norm, those are your safety glasses; I'll wear my own thanks...
The models -- actually complicated software written in a computer language called Fortran -- attempt to account for everything happening in the atmosphere on a global basis.
... actually I used to work for the National Weather Service ... C++, Tcl/Tk, and even Fortran ...
Well no wonder weather prediction is so off!
I kid, I kid
If easy-for-geeks-to-build home weather sensors were available, this would be a cool SETI-at-home-like project that would let hardware geeks have fun with distrubited computing too.
The storms will hit the Caribbean and Florida in September.
Next they'll have sensor strapped to the back of every butterfly on earth, increasing hurricane predictability 10 fold.
-Randy
I currently work for NOAA at a facility called GFDL. We house some of the super computers here. I currently operate and control the computers and its deffinitly a treat to be able to work with these fast machines. We have some of the worlds fastest computers here and they compete very well with the earth simulator. We also have some of the top hurricane guys working for us as well. It is good to see that the techonology that we use is getting publicity. It will inform everyone how things are done and where they get the information from.
You don't know too many scientists-turned-programmers do you? Fortran is still alive and well in scientific circles. Companies like IBM and SGI still write and optimize Fortran compilers for their newest CPUs. Even Intel recently released a major update to their P4 and Itanium2 Fortran compilers.
i) because most numerical weather codes are already written in Fortran. This means that people with the right scientific knowledge tend to be Fortran programmers, and makes porting a whole lot easier.
ii) Fortran compilers are the ones where the most work has gone into optimising the hardcore mathematical routines. Thus, the compiled code has traditionally been faster. This may no longer be true.
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
Fortan is just as scalable as C. And with modern supercomputing libraries and toolsets, it's probably even better suited to the task. Companies like IBM, SGI, and Intel continue to tune and tweak their Fortran compilers for the latest CPUs (R16K, Pentium 4, Power5, Itanium2, etc).
There are a lot of existing, hightly tuned fortan algorithms out there and plenty of scientists to keep the code running.
They are written in Fortran because that is what they have always been written in. Keep in mind that we are talking of at least tens of thousands of lines and in some cases a few hundred thousand lines of complicated math. This isn't one of those things you port to C or C++ in an afternoon. Also note that most of these models are written by scientists (physicists/chemists) not computer scientists. Most groups now have programmers on staff to help with problems but scientific programming isn't always about having elegant code; more often then not, we care about the output and Fortran does just fine. Yes, we do mix in C where appropriate.
You can see the current predictions by each model at any given time here:n g/at200406_model.html
http://www.weatherunderground.com/tropical/tracki
The NHC discussion of the model guidance for each storm is here, under 'discussion' for each storm:
http://www.nhc.noaa.gov/
They explain why they're agreeing with or discounting each model in their overall forecasts.
Generally, it's difficult to find much prediction of hurricane tracks that doesn't come somehow from the NHC. This isn't because there aren't independent analysists, but because they try not to send mixed signals, which might lead to people not evacuating when they should. The raw information from the computer models is the closest you get to dissenting opinions, afiak.
xkcd.com - a webcomic of mathematics, love, and language.
To a casual observer of the weather, like me, it seems that the paths of the hurricanes are little more than extrapolations of the current path with a slight bend to the east. For the hurricanes this year, it seems that time and again the models proved wrong for last minutes changes to the storm. I know from family who lives on the west coast of Florida that many people were caught off guard by Charlie. I really think that it is probably impossible to accurrately predict the path of a storm. I mean I could take a look at the motion of the storm and guess about as accurately as the models guess. My same family that was caught off guard by Charlie headed to Orlando when Ivan was about a week away, but the storm didn't land near their house. If you think about it, 3 days notice is not enough to have every person in a metropolis patch up their houses and move to higher ground. Some might say that everyone with the possibilty of getting hit by the storm should prepare, but imagine having to board your windows every 3 weeks or so only to be missed by the storm. It would be even worse if you evactuated on the same schedule. This would make it very difficult to live a normal life. Honestly, the prediction of storms like hurricanes needs to get much better, but I doubt that it ever will.
SIGFAULT
I saw in another post regarding my parent post that part of the reason is the tweaks for the fortran compilier being released.
I'm just trying to understand why they aren't moving to a more object oriented method of design for weather modeling. So they can drop in objects that don't require the entire code base to be recompiled.
Fortran is still the dominant language for programming high performance code. I'd still rather use C, but it's not really that different. When you're trying to optimize a piece of software for a machine architecture you need to use a language that is pretty low-level. The closer to assembly you are, the greater chance you have to best exploit the functionality of the hardware. C++/Java are right out.
As a resident of Florida (who's so far been pretty lucky with respect to the hurricanes), I've taken a keen interest in these models. The best place I've found to see them is at Weather Underground. Each listed storm has a "Computer Models" link at the end. See
Ivan
Jeanne.
Since the pages auto-refresh, I've just been leaving them up in a tab in Mozilla and checking them every once and a while. Though the models aren't always accurate and tend to change a lot, they kind of give you a feel for where the storm is probably going to go.
"Save the whales, feed the hungry, free the mallocs" -- author unknown
Living in Florida I've spent a whole lot of time looking at track modelling in the past month. For hurricane Frances at one point, the cone of error of eye movement was over 90 degrees, as in they had no clue which direction the storm would be moving an hour from when the model was made. In Charley, their predictions were over 100 miles off for the eye only a couple hours before landfall (which really sucked for me since the eye hit land only 30 miles away). And since that happened, the margins of error for Frances and Ivan have been much much larger as they realized their path predictions still sucked.
What's more, one of the local TV stations in-house track forcasting program gave a dead on accurate prediction several days before it hit while the National Hurricane Center was saying Tampa. With Ivan, at one point one of the major models (BAM Medium) predicted the storm would change directions from WNW to NE instantly at a point 3 days away.
Perhaps media forcasters should be evaluating which models are most accurate for the current storm instead of just reporting on a numerical average of computer model coordinates, since often outlying models pick up on something the other ones missed. A few places do that, and often predict the path with a much smaller error than the NHC, whose predictions are purely based on averages of models of which none are always accurate to begin with.
When your code runs for days, recompile time is pretty trivial. Fortran also has massive speed optimizations over Java. Most code like this has a really small inner loop that runs as fast as it possibly can, and even the smallest of performance hits from things like exceptions, object dereferencing, register loading of potentially aliased variables, and 1001 other minor things that goes on in the background that you don't see can increase run time by hours. Fortran is a least common denominator that lacks the flexibility of programming, but makes up for it by allowing the compiler to do all sorts of tricks like automatic parallelization of these inner loops. Java and C++ don't even come close to this, and hand coding such things is a gamble on efficiency for a good programmer, a sure loss for a mediocre programmer.
One lesson of object oriented is that you should let the language do the work for you. Sometimes this means that you shouldn't use object oriented languages.
Much of fortran code is written by scientist rather than programmers. HOWEVER, big hpc labs like this employ dozens of programmers (most with master or phds) to write the software for the scientists. I don't know about the navy labs in particular, but big DOD and DOE labs have optimization teams with several dozen programmers. These are the sort of people who present at the SC conference.
Even academic hpc facilities employ teams of experts to optimize code for the scientists.
You're right that there is a lot of inertia keeping people in the fortran fold. But the software vendors are also helping this by having really strong fortan libraries.
Not really. distributing the load like SETI isn't really an option in most of these simulations because of the large amount of comunication that needs to occur between the nodes. Problem is this isn't a bunch of independent processing tasks like in SETI. This is the same reason why fast interconnects like myrinet and infiniband are often used to connect the nodes in the clusters.
- dharhas
Fortran compilers are guaranteed that the programs do not try to do strange things behind their backs (such as pointer aliasing). Therefore they can make optimizations that would be almost impossible to prove valid in, say, C. Also, Fortran numerical libraries are of very high quality.
Finally! A year of moderation! Ready for 2019?
For straight up mathmatical processing, Fortrans about as fast as it gets(Disclaimer: Among major high-level languages, not counting assembly, something special cooked up for you, etc) C++ is fast, but it sacrifices a little bit of speed for flexibility. Java...well, Im anti-java, so lets not go there, but what else you gonna use, visual basic? The older languages are still quite good at what they were designed for.
Support more choices in goverment-Vote 3rd party.
Im guessing the choke point would be the drives read speed. I wonder if they run seti@home on these things. :-)
Support more choices in goverment-Vote 3rd party.
Oh, that's good. Screw around with mother nature some more.
Did you ever consider that things happen for a reason? Like balancing global heat loads and adjusting the water vapor cycle?
So instead of having a bunch of light to heavy storms, we'll end up with having ONE BIG MONSTER that we *can't* stop.
Predicting if you need your umbrella tomorrow in Bristol does not require any super computer... yes, you do.
Charley swerved just before landfall, Frances stopped dead in the water 60 miles off Florida, and Ivan "bounced" off Jamaica, shifting its path by 500 miles, none of which were predicted. Possibly, none of which are predictable. If you can't warn people where landfall will occur when it takes some non-obvious path, then what's really the point?
Weather simulation is not a tast you can cut up into a bunch of smaller tasks and farm out. If you cut the atmosphere up into lots of little chunks to model, after each step every chunk needs to know what the results from all the chunks around it where.
If you're waiting for those results at Internet (Latency: 100ms) speeds instead of intra-system speeds (latencyL 1 us) it takes you 100,000 times as long to get your data.
With SETI, all you do is get the data once, compute, and send back the answer.
paintball
This sort of code is probably heavily matrix oriented. Fortran handles matrix operations, making matrix operations as easy to program as simple arithmetic.
The place even looks cool http://www.ncar.ucar.edu/eo/what/arch1.html
Very cool place indeed, if you're in the area.