Slashdot Mirror
Supercomputers Race to Predict Storms
pillageplunder writes "CNN has an interesting article on how different supercomputers from around the world are working to predict large storms tracks. The 3 days it takes now has been cut in half. Cool read."
Isnt predicting the weather the job of that huge ass supercomputer in Japan? I think it is called Earth Simulator, but I am not sure... I remember a post about it awhile ago, and it showed all of the racks being 'crooked' because it allowed the heat from the processors to actually rise out of the case, and not stay in the rack. :P
-Bill
Oh wait, that's the job of the Diebold supercomputer.
If you think
I'll be impressed when I see supercomputers chasing tornadoes around Kansas in rusty pickup trucks. Not before.
lysergically yours
I could make a joke about this helping evacuation plans, but really it's just good news what with hurricanes pounding the southern part of the U.S and the Caribbean. A more accurate ETA of storms would be tremendously helpful to business and civilians alike.
-Teiresias
Actually if you read the article you will realize that it only takes about an hour of number crunching, but that the three day storm path accuracy errors have been cut in half... and that 5-day forcast is getting much more accurate.
I guess we should read articles before submitting them...
D.O.U.O.S.V.A.V.V.M.
I like how the article says: Just as a 5-megapixel digital camera more accurately depicts reality than a 1-megapixel device, higher resolution grids can capture a better picture of the atmosphere and help produce accurate forecasts.
:)
Way to pitch to the high-tech crowd CNN
But....... imagine a beowulf cluster of these weather predicting supercomputers.
Chris
http://www.sgi.com/company_info/newsroom/press_rel eases/2004/june/fleet_numerical.html
If we follow along this trail, we'll be able to predict the weather almost instantly with good precision... but there'll always be the "random factor" that nature always feels like putting in our way. :P
I don't think we will ever be able to predict the weather perfectly, without any errors, but I do believe we can get close.
...but still not as fast as "nowcasting" (and yes, it's an actual meteorological term.) I've always wondered why the local news just has to tell us, "And in downtown it is currently raining at the moment." The people who go outside already know, and the rest of us don't care.
Or, if you prefer, "error" was left out of the summary. According to the article, it is the amount of error in the three day forecast which has been halved since 1998.
The models -- actually complicated software written in a computer language called Fortran -- attempt to account for everything happening in the atmosphere on a global basis.
... actually I used to work for the National Weather Service ... C++, Tcl/Tk, and even Fortran ...
Well no wonder weather prediction is so off!
I kid, I kid
Why not just distribute the load(a la SETI). Seems like it would be a lot less costly and a lot more efficient.
If easy-for-geeks-to-build home weather sensors were available, this would be a cool SETI-at-home-like project that would let hardware geeks have fun with distrubited computing too.
The storms will hit the Caribbean and Florida in September.
The Army reading list
Next they'll have sensor strapped to the back of every butterfly on earth, increasing hurricane predictability 10 fold.
-Randy
"The models -- actually complicated software written in a computer language called Fortran -- attempt to account for everything happening in the atmosphere on a global basis." Now, this is a scary thought why on earth would they write the applications in Fortran? I would assume they would want to have this be a scalable and open platform.
http://pao.cnmoc.navy.mil/pao/n_online/archive/vol 22no1/articles/2001.htm
Wonderful journalistic number in the summary. Predicting storm paths now take only 1.5 days compared to 3 days before. As it happens I can predict where a storm will go in 24 hours in less time than 1.5 days (in fact, I might be able to tell several minutes before it even gets there!). Without any context that number is completely and in every way useless.
I currently work for NOAA at a facility called GFDL. We house some of the super computers here. I currently operate and control the computers and its deffinitly a treat to be able to work with these fast machines. We have some of the worlds fastest computers here and they compete very well with the earth simulator. We also have some of the top hurricane guys working for us as well. It is good to see that the techonology that we use is getting publicity. It will inform everyone how things are done and where they get the information from.
Ah, this must be one of its new predictions. :)
You don't know too many scientists-turned-programmers do you? Fortran is still alive and well in scientific circles. Companies like IBM and SGI still write and optimize Fortran compilers for their newest CPUs. Even Intel recently released a major update to their P4 and Itanium2 Fortran compilers.
John-John Mackey and the Stormtracker Accu-Cast
"He doesn't report the weather, he makes it his bitch"
You can see the current predictions by each model at any given time here:n g/at200406_model.html
http://www.weatherunderground.com/tropical/tracki
The NHC discussion of the model guidance for each storm is here, under 'discussion' for each storm:
http://www.nhc.noaa.gov/
They explain why they're agreeing with or discounting each model in their overall forecasts.
Generally, it's difficult to find much prediction of hurricane tracks that doesn't come somehow from the NHC. This isn't because there aren't independent analysists, but because they try not to send mixed signals, which might lead to people not evacuating when they should. The raw information from the computer models is the closest you get to dissenting opinions, afiak.
xkcd.com - a webcomic of mathematics, love, and language.
To a casual observer of the weather, like me, it seems that the paths of the hurricanes are little more than extrapolations of the current path with a slight bend to the east. For the hurricanes this year, it seems that time and again the models proved wrong for last minutes changes to the storm. I know from family who lives on the west coast of Florida that many people were caught off guard by Charlie. I really think that it is probably impossible to accurrately predict the path of a storm. I mean I could take a look at the motion of the storm and guess about as accurately as the models guess. My same family that was caught off guard by Charlie headed to Orlando when Ivan was about a week away, but the storm didn't land near their house. If you think about it, 3 days notice is not enough to have every person in a metropolis patch up their houses and move to higher ground. Some might say that everyone with the possibilty of getting hit by the storm should prepare, but imagine having to board your windows every 3 weeks or so only to be missed by the storm. It would be even worse if you evactuated on the same schedule. This would make it very difficult to live a normal life. Honestly, the prediction of storms like hurricanes needs to get much better, but I doubt that it ever will.
SIGFAULT
Almost every NOAA scientist that I know (NB: I know quite a few) is proficient in Fortran and IDL. This is the norm in atmospheric science.
Has anyone ever wondered why the weatherman is always wrong after just a few days? Chaos Theory dictates that supercomputers won't help, unless all the initial conditions are known. It has been said that if there were sensors spread accross the upper atmosphere spaced a yard apart, the data taken from an initial reading would break down in less than 6 hours. You can't predict the weather. But modeling it is cool as heck.
This summary is terrible. The article very clearly states that the *error* in the 3 day forecast has been cut in half -- not the computing time it takes to run the predictions. Geesh...
Fortran is still the dominant language for programming high performance code. I'd still rather use C, but it's not really that different. When you're trying to optimize a piece of software for a machine architecture you need to use a language that is pretty low-level. The closer to assembly you are, the greater chance you have to best exploit the functionality of the hardware. C++/Java are right out.
The military admits it can control the weather.a nge-Weapon s.htm
http://www.prisonplanet.com/articles/september2004 /130904hiltontranscript.htmn et.tv/articles/september2004/ 030904alexoncspan.htmp ?ARTI CLE_ID=38645h tmi cle= 44011&d=29&m=4&y=2004a .info/wecontrolamerica/ peakoil.html/ /www.surfingtheapocalypse.net/cgi-bin/forum. cgi?read=13928i llthomas.net/Investigations/Articles/ cellphones.htma l.com/excerpt.htmk e.net/emagazine/vol23/articles/ farley-23.html. htme smenw .geocities.com/Heartland/7006/psychopoli tics.htmlp ://www.hermes-press.com/brainwash1.htmw .ninehundred.net/control/d e/~aton/swfqw.htmlc om/v/babdoc.htmc ansvote.org/home1.asp
h ttp://www.lp.org/ bg1.htmll 1 1l iti on.html
N ewsl l y2004/07020 4highwaywatch.htm_ state.html
http://www.mindfully.org/Air/Climate-Ch
http://www.freedomdomain.com/weather.html
http://www.prisonpla
Truth about oil
http://www.worldnetdaily.com/news/article.as
http://www.unlearning.org/editor30.
http://www.arabnews.com/?page=6§ion=0&art
http://joevialls.altermedi
http://www.gasresources.net/
http:
Cell phones
http://rfsafe.com/index.php
http://www.w
Fluoride
http://www.silentbetray
Chemtrails
http://www.davidic
http://www.carnicom.com/contrails
http://omega.twoday.net/stories/296251/
Bon
http://www.parascope.com/
Psych
http://ww
http://www.raven1.net/patents.htm
htt
http://ww
http://phoenix.akasha.
http://www.mindcontrolforums.
Vote
http://boeing.helpingameri
http://www.badnarik.org/
Elites
http://www.infowars.com
http://www.sovereignty.net/timeline.htm
http://www.willthomas.net/911/Bush/index.htm
9
http://letsroll911.org/articles/controlleddemo
http://www.rbnlive.com/northwoods.html
http://www.davidicke.com/icke/headlines.shtm
Gestapo
http://www.prisonplanet.tv/articles/ju
http://www.infowars.com/police
http://tinyurl.com/globalwarmingisascam
As a resident of Florida (who's so far been pretty lucky with respect to the hurricanes), I've taken a keen interest in these models. The best place I've found to see them is at Weather Underground. Each listed storm has a "Computer Models" link at the end. See
Ivan
Jeanne.
Since the pages auto-refresh, I've just been leaving them up in a tab in Mozilla and checking them every once and a while. Though the models aren't always accurate and tend to change a lot, they kind of give you a feel for where the storm is probably going to go.
"Save the whales, feed the hungry, free the mallocs" -- author unknown
i live in south louisiana and the hurricane is little to the east of me. i bet they had it bad in the old days when all they knew was the wind is picking up then buy then it is way to late.
www.angelfire.com/dc2/stockman/index.html http://www.FreeFlatScreens.com/default.aspx?refer
Living in Florida I've spent a whole lot of time looking at track modelling in the past month. For hurricane Frances at one point, the cone of error of eye movement was over 90 degrees, as in they had no clue which direction the storm would be moving an hour from when the model was made. In Charley, their predictions were over 100 miles off for the eye only a couple hours before landfall (which really sucked for me since the eye hit land only 30 miles away). And since that happened, the margins of error for Frances and Ivan have been much much larger as they realized their path predictions still sucked.
What's more, one of the local TV stations in-house track forcasting program gave a dead on accurate prediction several days before it hit while the National Hurricane Center was saying Tampa. With Ivan, at one point one of the major models (BAM Medium) predicted the storm would change directions from WNW to NE instantly at a point 3 days away.
Perhaps media forcasters should be evaluating which models are most accurate for the current storm instead of just reporting on a numerical average of computer model coordinates, since often outlying models pick up on something the other ones missed. A few places do that, and often predict the path with a much smaller error than the NHC, whose predictions are purely based on averages of models of which none are always accurate to begin with.
Technically speaking... everyone controls the weather... everyone and everything that moves, no matter how insignificant.
The New York Times had a similar story two days ago. Ironic that CNN would take two extra days to get a story about forecasts being extended out by two days.
Have you read my blog lately?
Fortran compilers are guaranteed that the programs do not try to do strange things behind their backs (such as pointer aliasing). Therefore they can make optimizations that would be almost impossible to prove valid in, say, C. Also, Fortran numerical libraries are of very high quality.
Finally! A year of moderation! Ready for 2019?
All this technology for storm tracking is useless when the real problem is the fact that each storm, when it touches land, causes damage into the tens of billions of dollars, not to mention the large death toll each storm brings with it when it touches land. We should care more about making technology to fight storms, reduce their impact on civilization.
It would be interseting to see a figure releating 1)real-world sample density and 2)computer power in flops and 3)choice of algorithm to 4) prediction accuracy.
Ceci n'est pas une signature.
Im guessing the choke point would be the drives read speed. I wonder if they run seti@home on these things. :-)
Support more choices in goverment-Vote 3rd party.
You'd still have the issue of reliability. It isn't ok if your hurricane forecasting network suddenly craps out because the people you told to evacuate took their computers with them. :D Or whatever... blackouts elsewhere, long weekends.
Oh, that's good. Screw around with mother nature some more.
Did you ever consider that things happen for a reason? Like balancing global heat loads and adjusting the water vapor cycle?
So instead of having a bunch of light to heavy storms, we'll end up with having ONE BIG MONSTER that we *can't* stop.
3 moab's on top, 3 moab's at the bottom.
No more Hurricane.
Charley swerved just before landfall, Frances stopped dead in the water 60 miles off Florida, and Ivan "bounced" off Jamaica, shifting its path by 500 miles, none of which were predicted. Possibly, none of which are predictable. If you can't warn people where landfall will occur when it takes some non-obvious path, then what's really the point?
Weather simulation is not a tast you can cut up into a bunch of smaller tasks and farm out. If you cut the atmosphere up into lots of little chunks to model, after each step every chunk needs to know what the results from all the chunks around it where.
If you're waiting for those results at Internet (Latency: 100ms) speeds instead of intra-system speeds (latencyL 1 us) it takes you 100,000 times as long to get your data.
With SETI, all you do is get the data once, compute, and send back the answer.
paintball
but I guess since their functions are flawed too, there's a possibility that bad data in a bad model will cancel each other out.
We've heard about this before. We predict when/where the storm will happen, then we prevent it and put the storm in jail, but then sometimes there's a minority report and it turns out we have to let all the storms go free and the supercomputers retire to a nice little pastoral island somewhere in Maine or something.
I think the public has a right to know who was the higher biddder to swing the election results. Time to include Diebold in disclosure laws!
I'd like a little warning if Michael Eisner or Bill Gates is going to be our next president through a suprize number of "write-ins".
"There is more worth loving than we have strength to love." - Brian Jay Stanley
This sort of code is probably heavily matrix oriented. Fortran handles matrix operations, making matrix operations as easy to program as simple arithmetic.
"Why fix something when it ain't broke?"
/ browse? s=e&p=30
Pure and simple, as had been taught in my CS minor, OO-programming is very good. Except in numerical calculations, because you'll sit there and try to figure out these cutesy algorithms and objects and stuff that makes everything more complicated than it needs to be.
You're talking about, at its simplest level, integrating the following groups of equations in time:
http://amsglossary.allenpress.com/glossary
It's a straightforward problem that really doesn't need to complicated by object orientation. Why do something in a language that does more than you need it to?
Trust me, as an atmospheric science grad student, I don't like programming in FORTRAN any more than I have to, but it's one of the most terrific tools for high-speed pure numerical calculation. It's simple to learn, very fast, and well-tested. It has, like any other language, its quirks, but you quickly learn what to do about those and work with them. It's a beautifully direct and useful language. I never understood why FORTRAN programmers are given the disdainful look down the nose by many other programmers.
(There's this argument, as well as the FORTRAN optimization argument and the FORTRAN inertia argument. FORTRAN 90, also, can have very object-oriented portions of code... the same with analysis programming languages like IDL or MATLAB.)
-Jellisky
The real problem in predicting these problems is the size of the scales involved in the solution. A weather simulation is a numerical solution to the full Navier-Stokes equations, which are a (very) non-linear system of partial differential equations. They are the same equations that describe most (newtonian) fluid motion. They can be solved directly (google Direct Numerical Simulation) but the resolution of the grid required scales with Reynolds Number to the 9/5 power. For a model of, say, the US, you could only imagine that the length scale (and hence Reynolds number) would be HUGE, and the number of grid points would be astronomical. What next?
The solution that has been most often used is to simulate some of the non-linear behavior with models. As any scientist can tell you, all models are wrong (just some are better than others). The last few decades have been great for the compuatational fluid dynamics community since faster and larger computers allow us to solve more difficult problems. Right now people are running DNS models of simple flows, but large and complex flows are not yet viable. Until things get a lot faster, we'll be stuck with bad predictions.
bueller...bueller...bueller
Just type in "city,state" or a ZIP code, and you're good to go.
how different supercomputers from around the world are working to predict large storms tracks
Additionally, the scientists discovered that all supercomputers predicted different tracks for the same storm..
From what I remember from reading the optimization guidelines for a SGI O3400 box (a few years ago when it wa brand new), Fortran code is easier to automatically optimize then C code is. I believe it has something to do with the use of pointers in C.
I think the major reason Fortran is used is because of legacy algorithms (and legacy programmers!)
What we should be concentrating on is technology to remove a storm from existance as soon as it begins to form.
:-(
I was wondering how long it would be until somebody suggested this.
Aside from the fact that we'd be incredibly stupid to even try, we just couldn't even begin to have a significant impact on these storms. It's hard to believe how much energy they contain.
More info from NOAA:
http://www.aoml.noaa.gov/hrd/tcfaq/C5c.html
There is nothing in the architecture of the ES to prevent it from performing the calculations for modeling hurricanes. The only possible problem might be that the software for doing the modeling has not yet been ported to the ES.
Anyhow, what is interesting to note is that the modeling of hurricanes is a critical operation of the supercomputers. Another critical operation is command control on the battlefield. In such critical operations, which supercomputer would you use?
At the end of the CNN article, it implies that the new system for modeling hurricances is a super-multiprocessor powered by the Power4+ from IBM. In a sense this deployment of the IBM supercomputer says a lot about the Power4 and IBM technology. When your life or the lives of millions of people depend on the calculations performed by a supercomputer, which supercomputer manufacturer and processor would you choose? Apparently, the clear choice is IBM and Power4.
There is no mention of Pentium IV or UltraSPARC. Not surprising, really.
Totally off-topic but when I saw the article I was picturing a pop-up box telling me to stop playing Starcraft before I am banished to the couch by the missus.
And yes by playing Starcraft, I acknowledge that I'm old.
I think they were called "shutters". :)
"We're sorry, but the website you're trying to reach has been disconnected."
I'm modeling supercells that produce tornadoes
[Pardon me for taking advantage of your expertise]...So two questions have bugged me for a while.
Q1: Once I read where there seemed to be the possibility that tornadoes could be driven by magnetohydrodynamic forces [Nalivkin, 1963]. Is that plausible, credible, or are density variations due to thermal buoyancy enough to account for the observable physics of tornadoes?
Q2: What is it - really - that causes lightning? I've heard hand waving arguments about ice particles and triboelectric phenomena - but do people really know what causes charge separation in clouds and between clouds and ground?
"Provided by the management for your protection."
Many of the most famous Fortran computational libraries have long since been translated into C and other languages, but when you have millions of lines of perfectly functional (and reasonably complicated) Fortran production code, moving to something like C is a difficult decision, especially when you have limited value in the move.
As someone else pointed out: C is close to the machine, Fortran is close to the math. Where Fortran falls down is building interesting data structures and producing pretty output. Truth of the matter is that C isn't that much better as complex structures, and if you want pretty output you can always pipe the fortran output thru a perl filter. -- besides, these days the output is usually turned into a graphics image, so pretty numerical output isn't so important any more.
Besides: Just because the modelling is being done in FORTRAN doesn't mean that other parts of the process aren't done in languages appropriate to those tasks.
Fortran does just fine, thank you for numeric computation. Just because I wouldn't use it for OS building doesn't mean that I'm willing to throw out this baby with the bathwater.
(( Using COBOL on the other hand, seems lik cruel and unusual punishment for almost any task. ))
Sometimes boldness is in fashion. Sometimes only the brave will be bold.
Mother nature is going to do whatever the @#$%@#$ she wants to do, no matter how hard we try to say what we think she is going to do. All credit is due, however, to the people who by making their best guesses with the equipment and data they have ARE SAVING LIVES. Good on them, I say!
"...Two roads diverged in a wood, and I-- I took the one less traveled by, And that has made all the difference."
Many of you are familiar with Seti@Home to look at ET. They moved to their new infrastructure called BOINC. Boinc supports multiple concurrent project.
Well there is a weather simulation project called ClimatePrediction.net where your computer simulate 15 years of the earth climate while you get a cool looking screen saver with the simulated weather. They are calibrating with simulation of past weather. With the calibrated models they will then forecast the next 50 years and hopefully have a best model of global warming.
Join in numbers and help clear up doubt about the future climate. You can share the same machine with SETI@Home if you want.