Slashdot Mirror
New Weather Computer
Sarah writes "It seems that the National Weather Service has a brand-new computer which will allow them to predict the weather earlier and more accurately. If I were a kid, I could now plan my snow days in advance..." Yeah, but the teachers would give you enough homework to last you through the day.
If the computer controled the weather then I would rule the world.
Real men dump cores! Read my journal, I am neat.
8th?
The older Cray C-90 computer had been in use since 1994 and was to be offered to other government agencies when replaced, but it was destroyed in a fire last September.
What a tragic waste of a computer. Oh, the humanity!
Yet another expensive way to completely miss the weather forcast.
kwsNI
Can it run Linux?
Criminalize spam and telemarketing!
This makes me think... Now since we have a "suprise proof weather predition system" they can warn us about every drop of rain. Next time they say the snow word, hundreds of morons run to the store and buy all the bread and bottled water they can. this is insanity!
government agencies when replaced
In related news, Commander Taco announced the newest Andover.net venture : www.slashdot.gov.
Jack Valenti and the MPAA are to technology as the Boston strangler is to the woman home alone
Posted by NJViking:
The computer sounds very impressive. They will be able to make more long-range forecast as well as predict more factors sooner thereby giving people more warning.
I would think this would also help them with tornado predictions as well.
NJV
Anyone have any experience with weather modeling?
I don't know about you guys, but I'm glad to see any kind of advancement toward better long-range prediction of the weather. I'm tired of the weatherpeople being wrong about the third and fourth days of the forecast. It was going to be a sunny 53 degrees F today, according to Sunday's forecast, but according to the news this morning, it's going to stay below freezing all day, and it's supposed to sleet this evening.
XenoWolf
XenoWolf The Original - Since 1993
"with a warning that major East Coast cities face the threat of snow and severe cold late this week"
Ummmm.... I think it's a little late for that. The cold already arrived, and has been here the past few days (Is 0 degrees in Connecticut severe enough, yet? With a bad wind chill, nonetheless!)
I could've forecast that!
On the other hand, this new computer sounds pretty good. Forecasts over a week in advance? Great!
(Now, what they need to do is make a distributed client, Weather Channel@home, ala SetiAtHome or Distributed.net, to push it to a month in advance.)
I'm not a meteorologist (but my sister is...) With computing resources like this, I hope that top management in NWS does not carry the belief that more computing power alone will improve forecasting. You need to have good models behind it.
My sister once told me that met. is "sloppy physics", mostly because many of the variables for their equations aren't measurable, thus they often need to extrapolate or even guess them. Stuff like speed of vertical motion of airmasses, which as I understand aren't measurable via radar (but I could be wrong)
e to the i pi equals negative one
The main benefit we should see from new NOAA computers is more efficient operations. We should be able to analyze storm tracks more quickly in cases of major weather events such as hurricane or tornado. Long range forcasting will still have the same level of best guess scenaria it had last week.
"For every complex problem, there is a solution that is simple, neat, and wrong."
My office has been taken over by iPod people.
Now if they would adopt a distibuted computing model for generating weather predictions, I think lots of people would join immediately. Imagine being able to say "I want my CPU power to help calculate the weather for my hometown." That would be cool.
Boy, could we use one of them here i Oz.
It's Uccellini's center that uses the new 786 processor IBM SP computer located in Bowie, Md.
...hmmm... 786 processors in this upgrade from the Cray C-90 ... me thinks someone over at the NWS has a sense of humor, and is punning off of Intel's x86 line...
Your Brain + EEG + LEGO Robots = Brainstorms
What about a Beouwolf cluster of these?
--
then it comes to be that the soothing light at the end of your tunnel is just a freight train coming your way
This is something about which I remember reading last year some time. Now, before anyone starts flaming that it's still news, yada-yada-yada, let me point something out. I am NOT saying that Slashdot should not have posted it ... I'm simply pointing out that this has been planned for ages (In other words, it's an "FYI.").
One particular aspect of the story I read about this last year, was that they announced their plans during one of those wonderful storms in Florida. Of course, at the time, it didn't do Florida any good, did it? (rhetorical question, BTW.)
The computer is a great step forward for NOAA. I used to work for them, my wife still does -- both trained meteorologists.
The whole "Sloppy physics" argument gives short shrift to the endeavor. The physics are very complex, and still not completely understood. They are also incredibly complex -- meteorology encompasses advanced physics and chemistry, along with the ungodly math that goes along with it. Only Theoretical physics will have more computers dedicated to it on the top 100 list of supercomputers.
Still, forecasting is as much art as science -- Truly good forecasters rely on intuition and experience to interpret output from several different models (both graphic and numeric) and put together a forecast. Statistical methods are also used to compare with similar events from the past. It is very easy to forecast -- it is extremely time consuming and difficult to forecast WELL.
Many TV stations' on-air people are not meteorlogists (in training or temperment)-- in fact many of the people on the weather channel are communications majors (at least they have a room full of metos telling them what's going on.)
The theoretical limits on forecast ability come from a number of factors. The reliability and the density of data points. There are relatively few datapoints for upper-air data (release a balloon, etc...)- on the order of a few per state - and those soundings happen only twice per day (except perhaps in extremely active severe weather environments). Even automated senesing stations are few and far between. Data then has to be interpolated for intermediate points and then stuffed into the model. Most models are then run on a 64-km grid and interpolated down. Finer mesh models (32 km ETA, et. al.) are being developed, but when all the models get run on the same machine, sacrifices in the name of efficiency must be made. Additionally, we still just don't know how it all works exactly. The effects of small scale things like the "heat-island" effect of large paved areas, pollution, solar activity, etc. are still being teased out.
Anyway, it's good to see them get the new machine (actually it wasn't so much the fire as the SPRINKLER SYSTEM that killed the old one). Give them a break. Their mission isn't to tell you what sort of coat to plan on for the morning, it's to save lives and property, and on that count, they do a hell of a job.
In a previous life I was a meteorology graduate from Rutgers University (1988). While its nice that they have a bigger and better computer, unless and until thay have better input/initialization data to feed it, I can't see how the forecasts will get any better.
Twice a day (0Z and 12 Z) the main prediction models are initialized with data from all over the world. Not only surface data, but "upper air" data as well. Upper air data come from sparsely located stations that actually have the ability to send up and record data from weather balloons.
To give you an idea of how sparse these stations are, near my house in New Jersey (USA), the closest upper air observation sites are from nearby:
Albany NY
Pittsburgh, PA
Wallops Island Virgina.
Every gridpoint in between, no matter how many there are, is interpolated/guessed at as initialization for the various numerical models that depend on that data.
Click here for a complete list of NWS stations that are included in the national upper air data collection network .
So while they might have the ability to have more gridpoints, and they can have the capability of modeling the interactions between more gridpoints, the initialization data is still the same. It seems to me that they also need to spend more money on getting more data.
I remember the Olympics in Atlanta. IBM setup a very sophisticated weather observing system that allowed the NWS to predict weather at each individual venue. They were able to do this because they had upper air data every 10 or so miles a *few* (i.e. more than twice) times each day.
Click here if you would like to see the current output of these models. This will lead to a whole set of links for the various models. Some sites are better than others (the Unisys site and the Uinversity of Wisconsin site are the best.
The current models of choice at the NWS are the ETA and AVN. The NGM is an older model they still run and is referenced in many of their discussions. They run it as an internal consistency check to make sure the other models did't get caught in a chaos loop somewhere.
I'm still working on a clever footer.
See i think this is a good thing. About a month ago I remember seeing a chart listing the most powerful computers in the world. The first 20 or so from the US belonged to the "defense" dept. while virtually every european country had their number one acting as a weather computer. The US' number one weather computer was ranked something below 50. It seems we had our priorities a little twisted on the super computer front...or maybe that's the way it's supposed to be, being the bullies we are.
Actually due to calculational constraints I doubt the models themselves are anywhere near perfect. The physics of fluid dynamics involves vast reams of very messy non-linear differential equations IIRC and these require complex numerical methods to solve even approximately. Even if they had the data at every single point (impossible I know) the current models would still fail in the long-term / small scale.
Yah - Linux runs on a few PPC RS/6000's but not on an SP since the real complexity is getting the interprocessor switch backplane to work. Remember that an SP is not a single system image machine. It is a group of processor complexes and for every processor complex you have another instance of AIX. Each SP is rack made up of processor complexes each one of which is analogous to an indivdual SMP RS/6000 and the switch backplane is what holds the whole shebang together. Each complex is called a node and can hold 2-12 (or even more possibly) PPC CPU's. A rack holds a bunch of nodes so if you have 12 CPU's in each node a 768 processor machine is made up of 64 nodes mounted in 8 racks or so depending on the packaging.
I agree that this is one gnarly piece of big iron, but how much time and money is being devoted to the refinement of the computer models used forcast the weather? A weather forecaster friend mentioned in conversation that most of the models are pretty good at predicting summertime trends, but in the winter(the NGM and ETA especially), most fail miserably.
Here in the Ohio valley, weather is unpredictable enough, but winter weather is especially bizarre. It would be nice if the local forcasters had higher quality data and had to rely on their "gut feeling" less. It really sucks to wake up to a 1/2 inch of sleet frozen to the road when the forecaster assured the city we would only see light flurries!
If nothing else, imagine the babes you could get with that kind of computing power at your disposal. "Hey baby, wanna check out my gigaflop"?
~Any apparent grammatical or typographic errors are caused by defects in your display device.
Wouldn't it be cool if they would open source their forecasting code? It isn't like anyone is threatening to take over the job of actually doing the forecasts. Most of us don't have the computing horsepower. And who else has the up-to-date data sources? But I think some of us might take a look under the hood to see how it all works. And if they're lucky, they might get a couple of good patches that would get the a little more speed or fix a bug or two.
The net will not be what we demand, but what we make it. Build it well.
Think of a simple system, like a round-bottomed bowl turned curved side up. Put a marble on the top and record its path. The get the marble and put it close to the original starting point. It could, if you're sufficiently close to the centre of the bowl, end up going in any direction. That's the butterfly effect.
Since measuring instruments can't measure every contributing factor to the weather (temperature, pressure, moisture, wind) to arbitrary levels at a sufficient number of points to form an accurate and complete initial condition from which to predict the weather, it'll go close for a while (the better the measurements, the closer), but within a couple of weeks the values just diverge.
If people are interested in reading a bit more about this stuff, there are a few good books of introduction, like "Chaos" by James Gleich or "Does God play dice?" by an author I can't remember. A good article as a lead-in is here.
Dave Neary.
Often, when pondering for no reason, I wonder how many "state of the art" programming techniques that have existed in CS (fuzzy logic, neural networks, hello world in every language) have been utilized by the meteorlogical sciences people?
I'm not knocking their abilities for development of software to solve their problems, but I always come to a single issue: they are primarily meteorologists who have learned to program as opposed to program and who's primary focus is meteorology. What if they had an influx of people who's background is entirely programming? People who program because they focus on programming.
I believe that the two disciplines working together could better attack the problems at hand. As an industry we are happiest when we have a nice fat dataset to analyze that we can sink our algorithmic teeth into...
e to the i pi equals negative one
Often, when pondering for no reason, I wonder how many "state of the art" programming techniques that have existed in CS (fuzzy logic, neural networks, hello world in every language) have been utilized by the meteorlogical sciences people?
I'm not knocking their abilities for development of software to solve their problems, but I always come to a single issue: they are primarily meteorologists who have learned to program as opposed to program and who's primary focus is meteorology. What if they had an influx of people who's background is entirely programming? People who program because they focus on programming.
I believe that the two disciplines working together could better attack the problems at hand. As an industry we are happiest when we have a nice fat dataset to analyze that we can sink our algorithmic teeth into...
e to the i pi equals negative one
>>(actually it wasn't so much the fire as the >>SPRINKLER SYSTEM that killed the old one).
A $$gazillion invested in a Cray super computer, and they didn't spring for a Halon system to protect the room? Ouch!
You missed Upton NY - Read Stony Brook, where the local NWS center is - the grounds of the Old Camp Upton - now known as Brookhaven National Labs
-- 73 de KG2V For the Children - RKBA! "You are what you do when it counts" - the Masso
I thought there was one on Long island! I was looking for others in NY but didn't realize Upton was the one.
I'm still working on a clever footer.
There are TVs scattered through the hallways where I work, switching back and forth between CNN and an internal USAF news network. On the CNN report I just watched that covers this story, there's a brief snapshot of one of the NWS scientists hacking away at a workstation running CDE.
That's why there's a two-week limit to the forecasting times. After that, CDE has exhausted the swap space.
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
did anyone see ABC news covering this story? they showed the monitor: right smack in the middle, "Segmentation Fault".
MM5: http://www.mmm.ucar.edu/mm5/mm 5-home.html
This is the primary research model used in the met. community and is generally used for short range prediction (out to ~48 hours). Fairly easy to work with though getting all of your data set up can be a bit of a hassle.
ARPS: http://www.caps.ou.edu:80/ARPS/
The ARPS model is being worked on by the Center for Analysis and Prediction of Storms (CAPS) at the Univ. of Oklahoma. The goal of CAPS is to provide short term predictions of hazardous weather. Everything but the kitchen sink in the code. Not the fastest code out there for sure.
WRF: http://wrf.fsl.noaa.gov/
The NWS also makes the source code available for the Eta model as well (try rooting around at the National Centers for Environmental Prediction web site. This version of the code will more than likely be the old version of the parallelized version of the code, not the new version that's been changed for the distributed nature of the SP2.The Weather Research and Forecasting (WRF) model is the next generation community model that is currently being developed. This model will be used both for research as well as operational forecasting. This is the successor to MM5. The NWS will begin to run this model operationally at some point once development gets far enough along.
-mike
I mean, 'chaos theory' largely came out of wx prediction, specifically the 'butterfly effect' where a very small change in the initial conditions can vary the outcome wildly - what I'm saying is isn't wx naturally pretty 'random and chaotic' (like life!) and that there are some kind of natural theoretical limitations on just how much CAN be predicted, like predicting the toss of a die, no matter how much cpu horsepower you have - kinda like an Uncertainty Principle of Meterology?
The Scarlet Pimpernel
try { do() || do_not(); } catch (JediException err) { yoda(err); }
just noticed someone already said the same damn thing :)
(read before posting? Naaaa!)
The Scarlet Pimpernel
try { do() || do_not(); } catch (JediException err) { yoda(err); }
in part because the upper air network is so sparse (and so infrequently sampled) compared to the surface network. There are also all sorts of boundary layer and topographical effects that cause the surface to be not representative of the entire atmospheric column. This, of course, cuts both ways -- local microclimates can be very important.
The classic example is a deep, bowl-shaped valley -- on clear, calm nights it will typically be sharply colder than on surrounding hills. On nights with a light wind, if the air in the valley decouples from the light breeze aloft, the difference might be very sharp indeed.
However, all in all I think it would be more cost effective to upgrade the upper air network, particularly over the oceans and Asia.
check out iwin.nws.noaa.gov. I use the text interface; look under State Data, and then under most of the states you'll find Forecast Discussion. Depending upon who's on duty and how interesting the weather is, you'll get anything from "Will continue current forecast" to a long discussion of factors influencing the weather and local effects. Walt Drag of the Boston (actually Taunton), MA office is legendary for his discussions, which recently have sometimes filled two full screens. He's clearly a big-time snow buff.
l is the current and last several discussions; they're usually updated about every 6 hours (again, except for Walt, who likes reissuing them for changing conditions). The current one (as of noon EST on Wednesday, January 19) is really juicy. Assuming he's still on duty this afternoon (I'm not certain how the shifts run), his afternoon discussion will be even better. It's interesting to read, to see how these folks interpret the data and the forecast models.
http://iwin.nws.noaa.gov/iwin/ma/discussion.htm
DNA is a Turing machine. You, however, being dynamic and emergent, are not.
Five times faster than the Cray C-90 it replaces, the new IBM can make 690 billion calculations per second. By September it will be speeded up to 2.5 trillion calculations per second... Yeah, it sounds sort c00l, but can it run Linux? ;)
On the other hand, another major deficiency in today's meteorological models is getting the land-surface right (e.g., just how wet is the soil? and how much of the sun's incoming energy shows up as conducted heat, and how much serves to evaporate water?) Present models aren't very good at this; we're working on it here at NCSC (see URL http://envpro.ncsc.org/projects/dashmm but it winds up very computational -- you have to use resolutions well under 1 KM in order to get the terrain-slope/drainage effects right. And you need to do that globally! (Satellite data aren't very useful; statellite radar generally doesn't penetrate beyond about 1 cm. And there's no funding to put up long-wave-radar interferometry equipment that might do better.)
"My opinions are my own, and I've got *lots* of them!"
And who's going to be uploading this weather information ahead of time? Why, you guessed it! Good old Bill Gates! That's right, we all know he controls the weather! (By the way Bill, thanks for all the snow that's comin to us down here in the mid-east! Woohoo!)
Bring on the karma-toilet if you think this isn't funny! hehehehe!
............ no.
Why not use distributed.net to help predict the weather? 786 processors... ha! We could have so many more...
I could not justify my existence if I were a turkey farmer. Would I terminate myself? Undoubtably, yes.