Slashdot Mirror
New Weather Computer
Sarah writes "It seems that the National Weather Service has a brand-new computer which will allow them to predict the weather earlier and more accurately. If I were a kid, I could now plan my snow days in advance..." Yeah, but the teachers would give you enough homework to last you through the day.
Anyone have any experience with weather modeling?
I'm not a meteorologist (but my sister is...) With computing resources like this, I hope that top management in NWS does not carry the belief that more computing power alone will improve forecasting. You need to have good models behind it.
My sister once told me that met. is "sloppy physics", mostly because many of the variables for their equations aren't measurable, thus they often need to extrapolate or even guess them. Stuff like speed of vertical motion of airmasses, which as I understand aren't measurable via radar (but I could be wrong)
e to the i pi equals negative one
The main benefit we should see from new NOAA computers is more efficient operations. We should be able to analyze storm tracks more quickly in cases of major weather events such as hurricane or tornado. Long range forcasting will still have the same level of best guess scenaria it had last week.
"For every complex problem, there is a solution that is simple, neat, and wrong."
My office has been taken over by iPod people.
Now if they would adopt a distibuted computing model for generating weather predictions, I think lots of people would join immediately. Imagine being able to say "I want my CPU power to help calculate the weather for my hometown." That would be cool.
It's Uccellini's center that uses the new 786 processor IBM SP computer located in Bowie, Md.
...hmmm... 786 processors in this upgrade from the Cray C-90 ... me thinks someone over at the NWS has a sense of humor, and is punning off of Intel's x86 line...
Your Brain + EEG + LEGO Robots = Brainstorms
It could, it if genuinely is "suprise proof" lead to fewer warnings. I rather doubt that even this machine is really "surprise proof" with something as annoyingly comlpex as weather prediction. The real test, and likely downfall, will be to predict truly nasty weather - genuinely severe storms that spawn tornadoes, for example. While these may be somewhat predictable, I rather doubt we'll see forecasts of which paths to get clear of before the twister shreds it. That would be "surprise proof" prediction. Who really cares if the forecast high or low is off by a couple degrees?
The real hope is that there will be sufficiently increased accuracy so that needless warnings are not issued. Hopefully the reverse will not happen and overconfidence cause a failure to issue a warning that deserves to be issued.
As for runs on stores due to snow, maybe that situation happens in places where snow is an unusual event. I happen to be in a place where it is possible, in the colder months, to walk across lakes without getting wet. It is snowing right now and nothing much is unusual. Folks still go to work and all, and there's no rush to stores as this is just like a rain shower here, except when it's over the result can (or has to) be pushed aside.
I don't subscribe to RMS's GNUtopian vision.
The computer is a great step forward for NOAA. I used to work for them, my wife still does -- both trained meteorologists.
The whole "Sloppy physics" argument gives short shrift to the endeavor. The physics are very complex, and still not completely understood. They are also incredibly complex -- meteorology encompasses advanced physics and chemistry, along with the ungodly math that goes along with it. Only Theoretical physics will have more computers dedicated to it on the top 100 list of supercomputers.
Still, forecasting is as much art as science -- Truly good forecasters rely on intuition and experience to interpret output from several different models (both graphic and numeric) and put together a forecast. Statistical methods are also used to compare with similar events from the past. It is very easy to forecast -- it is extremely time consuming and difficult to forecast WELL.
Many TV stations' on-air people are not meteorlogists (in training or temperment)-- in fact many of the people on the weather channel are communications majors (at least they have a room full of metos telling them what's going on.)
The theoretical limits on forecast ability come from a number of factors. The reliability and the density of data points. There are relatively few datapoints for upper-air data (release a balloon, etc...)- on the order of a few per state - and those soundings happen only twice per day (except perhaps in extremely active severe weather environments). Even automated senesing stations are few and far between. Data then has to be interpolated for intermediate points and then stuffed into the model. Most models are then run on a 64-km grid and interpolated down. Finer mesh models (32 km ETA, et. al.) are being developed, but when all the models get run on the same machine, sacrifices in the name of efficiency must be made. Additionally, we still just don't know how it all works exactly. The effects of small scale things like the "heat-island" effect of large paved areas, pollution, solar activity, etc. are still being teased out.
Anyway, it's good to see them get the new machine (actually it wasn't so much the fire as the SPRINKLER SYSTEM that killed the old one). Give them a break. Their mission isn't to tell you what sort of coat to plan on for the morning, it's to save lives and property, and on that count, they do a hell of a job.
In a previous life I was a meteorology graduate from Rutgers University (1988). While its nice that they have a bigger and better computer, unless and until thay have better input/initialization data to feed it, I can't see how the forecasts will get any better.
Twice a day (0Z and 12 Z) the main prediction models are initialized with data from all over the world. Not only surface data, but "upper air" data as well. Upper air data come from sparsely located stations that actually have the ability to send up and record data from weather balloons.
To give you an idea of how sparse these stations are, near my house in New Jersey (USA), the closest upper air observation sites are from nearby:
Albany NY
Pittsburgh, PA
Wallops Island Virgina.
Every gridpoint in between, no matter how many there are, is interpolated/guessed at as initialization for the various numerical models that depend on that data.
Click here for a complete list of NWS stations that are included in the national upper air data collection network .
So while they might have the ability to have more gridpoints, and they can have the capability of modeling the interactions between more gridpoints, the initialization data is still the same. It seems to me that they also need to spend more money on getting more data.
I remember the Olympics in Atlanta. IBM setup a very sophisticated weather observing system that allowed the NWS to predict weather at each individual venue. They were able to do this because they had upper air data every 10 or so miles a *few* (i.e. more than twice) times each day.
Click here if you would like to see the current output of these models. This will lead to a whole set of links for the various models. Some sites are better than others (the Unisys site and the Uinversity of Wisconsin site are the best.
The current models of choice at the NWS are the ETA and AVN. The NGM is an older model they still run and is referenced in many of their discussions. They run it as an internal consistency check to make sure the other models did't get caught in a chaos loop somewhere.
I'm still working on a clever footer.
Yah - Linux runs on a few PPC RS/6000's but not on an SP since the real complexity is getting the interprocessor switch backplane to work. Remember that an SP is not a single system image machine. It is a group of processor complexes and for every processor complex you have another instance of AIX. Each SP is rack made up of processor complexes each one of which is analogous to an indivdual SMP RS/6000 and the switch backplane is what holds the whole shebang together. Each complex is called a node and can hold 2-12 (or even more possibly) PPC CPU's. A rack holds a bunch of nodes so if you have 12 CPU's in each node a 768 processor machine is made up of 64 nodes mounted in 8 racks or so depending on the packaging.
Wouldn't it be cool if they would open source their forecasting code? It isn't like anyone is threatening to take over the job of actually doing the forecasts. Most of us don't have the computing horsepower. And who else has the up-to-date data sources? But I think some of us might take a look under the hood to see how it all works. And if they're lucky, they might get a couple of good patches that would get the a little more speed or fix a bug or two.
The net will not be what we demand, but what we make it. Build it well.
Think of a simple system, like a round-bottomed bowl turned curved side up. Put a marble on the top and record its path. The get the marble and put it close to the original starting point. It could, if you're sufficiently close to the centre of the bowl, end up going in any direction. That's the butterfly effect.
Since measuring instruments can't measure every contributing factor to the weather (temperature, pressure, moisture, wind) to arbitrary levels at a sufficient number of points to form an accurate and complete initial condition from which to predict the weather, it'll go close for a while (the better the measurements, the closer), but within a couple of weeks the values just diverge.
If people are interested in reading a bit more about this stuff, there are a few good books of introduction, like "Chaos" by James Gleich or "Does God play dice?" by an author I can't remember. A good article as a lead-in is here.
Dave Neary.
There are TVs scattered through the hallways where I work, switching back and forth between CNN and an internal USAF news network. On the CNN report I just watched that covers this story, there's a brief snapshot of one of the NWS scientists hacking away at a workstation running CDE.
That's why there's a two-week limit to the forecasting times. After that, CDE has exhausted the swap space.
You cannot apply a technological solution to a sociological problem. (Edwards' Law)
In fact, for a long time, it had none. Last year it was shut down/reduced to low availablility for quite some time when the crew installing sprinklers in the room found asbestos and it had to be quaratined while they cleaned up.
I mean, 'chaos theory' largely came out of wx prediction, specifically the 'butterfly effect' where a very small change in the initial conditions can vary the outcome wildly - what I'm saying is isn't wx naturally pretty 'random and chaotic' (like life!) and that there are some kind of natural theoretical limitations on just how much CAN be predicted, like predicting the toss of a die, no matter how much cpu horsepower you have - kinda like an Uncertainty Principle of Meterology?
The Scarlet Pimpernel
try { do() || do_not(); } catch (JediException err) { yoda(err); }
just noticed someone already said the same damn thing :)
(read before posting? Naaaa!)
The Scarlet Pimpernel
try { do() || do_not(); } catch (JediException err) { yoda(err); }
Rlk dun said:
Another meterologist who consistently puts out educational (and damned funny!) forecast discussions is "I-Sixtyfive" down at the Birmingham, AL WFO. (His specialty is severe weather as well as taking a decided sense of humour--among other things, his forecast discussions have at times imitated hit parades, at one point actually spoofed Star Wars...it's obvious he has quite a bit of fun at his job :)
Seriously, though...reading forecast discussions is a good way to at least start to learn about meterology...you learn what forecast models are used, how they get the data, what makes a good or bad forecast, etc. (And just to note--a lot of forecasting ain't computer models so much as someone who's been in the area for years and knows how the weather patterns work. I've seen this many a time with the SDF (Louisville) NWSFO--Ohio Valley weather is living proof that weather is chaotic ;) and many times what makes or breaks a forecast is being familiar with how weather tends to "behave" (or, rather, misbehave) in the area.)
-Windigo The Feral (NYAR!)
Pyramid dun said:
That's because Ohio Valley weather is more chaotic than chaos itself. :) We don't even need the damn butterfly--all we need is Tom Wills[*] to blow his nose, and within the span of a week we will get F6 tornados, blizzards that dump two feet of snow and six inches of ice, AND a flood to boot--and nobody will see it coming till it brews up on top of them. ;)
Yes, I'm exaggerating. Not by much, though--to give an example of wonderful Ohio Valley winter weather--a few days before Christmas we get five inches of snow where maybe a "light dusting" was predicted. January 3rd, we get tornadoes because the weather is unusually springlike. Yesterday, snow was predicted but we got freezing rain and sleet instead. We are supposed to get freezing rain and snow tonight, but I will not be one bit surprised to wake up tomorrow to see a foot of snow on the ground and the city of Louisville entirely shut down because people do not know how to manage more than four inches of snow at a time. :)
As a minor aside--I remember reading that, largely because standard models cannot predict Ohio Valley winter weather worth a damn, the Louisville NWSFO is working on a new weather model specifically meant to predict Ohio Valley snowstorms...I wish them good luck, especially knowing our weird and wonderful weather (don't like it, wait fifteen minutes...it'll change...it might put the fear of God into you in the process, but it'll change, trust me :)
-Windigo The Feral (NYAR!)