Perfect Weather on the Net

ctwxman writes "Here in the East, we're having a heck of a weekend. The snow has been flying, accompanied by wind, accompanied by people on TV saying, 'stay home.' I'm one of those people. I forecast the weather for a living. It's a great job... a magnificent physics puzzle, solvable with high level math and some acquired skill. And, there's a new puzzle every day, tenuously linked to the one you solved yesterday. When I started doing this over 20 years ago, the data to accomplish the task was tough to come by. I remember how excited I got when I first was able to get data at 300 baud on a request/reply basis (I guess we call that interactive now). Now, nearly everything you need to forecast the weather is here, on the net. Yes, there are a zillion sites that already have the finished forecast, but this is Slashdot. We don't need no stinkin' forecasters!" Read on for ctwxman's notes on do-it-yourself weather forecasting.

ctwxman continues:

I always like to start at NCEP (National Centers for Environmental Prediction) for a look at the dynamic models. Each is run using somewhat different equations, making them often come up with different, quite contrary solutions. Some of these models, like the GFS are worldwide in their coverage and forecast out an amazing 16 days (note: the word accurately was not used in the last sentence). Once the dynamic models are through, we can massage them against past performance under similar circumstances at specific places. These are the statistical models, referred to as MOS (Model Output Statistic) models. Again, there are somewhat different solutions from different models. If none of these work for you, run your own. There are programs available to allow you to run your own model, specifying the domain, grid spacing, time interval, etc. The most commonly used research model of this type is the MM5, produced at Penn State University. Run it on your PC! Of course, it's freely available and supported. Sometimes, the data you want already exists, but not in the form you'd like to see it. That's where software like GrADS comes in. Put out by the Institute of Global Atmosphere and Society's Center for Ocean-Land-Atmosphere Studies, GrADS claims to be an 'interactive desktop tool that is used for easy access, manipulation, and visualization of earth science data.' I agree with all except 'easy.' I run a version of GrADS on my server in order to produce localized forecast graphics like this that wouldn't otherwise be available. Yes, looking at satellite imagery and radar is a lot of fun... but the real fun is knowing what will be there before you look. And, astoundingly enough, we are significantly more accurate (and I get assaulted significantly less often at the grocery store) than even a few years ago.

Comments

Run your OWN weather station

[caferace]

I bet I'm not alone doing this. Decent weather stations are cheap, and it's simple enough to hook it into your box and keep statistical models on a running basis. Using wx200d and an Oregon Scientific WM-918 (sold under other names as well) is a great solution for less than USD $200.

Granted, it's not true forecasting, but you can easily add your data to aggregate with other users at Weather Underground and pull radar data from just about anywhere.

Me? I just like to know what's happening NOW, but it's also pretty handy to know what the temp is in your home "server room".

Tying all this historical data back into longer range forecasts would be fun. I've found TV forecasting to be pretty stale and inaccurate. How many of them have real meteorological degrees anyway?

Old wives tails

[rf0]

Red sky at night shepards delight. Red sky in the morning shepards warning.

Lot of truth in that saying

Rus

Where's the raw data?

[Quixote]

I work in machine learning, and weather forecasting has always seemed like an interesting problem to explore. However, the lack of accurate raw data (temperature, dewpoints, pressure, humidity, precipitation, etc.) going back 30-40 years is a handicap. Where can one get such raw data, in a machine-readable format?

Also, I have been toying with the idea of writing a script to automatically grade the predictions put out by Wunderground and Weather.com, to see how accurate they are. It would be nice to see if it is really worth it to rely on their 5-day forecast.

On this note, but something completely different..

[cshuttle]

Since I was real little, I'd always been interested in the weather and forecasting and all that stuff. Now that toasters and other such standalone devices are becoming so prevalent, I'm looking for the simplest thing that apparently doesn't exist:

How about an 802.11 weather station?

I'm just looking for something that sits outside my house, collects weather data and other such simple stuff, and relays that data back to a server to build a web page with or whatever.

True, there are some devices like this available, but they all require a dedicated machine to log the data, and some really hard work to make them operate properly over a network (why would I possibly want a 1-wire data transmission solution, or even phone-line communication when I've got 802.11 right here?)

Have I simply missed the magic google search that has the toaster I'm looking for?

Re:If Only it was right!

[gurustu]

Seattle (and West Coast) accuracy is always going to be lower than you'd like (and lower than the East Coast's, for example) simply as a result of fewer observations.

The rest of the nation gets to see weather data from thousands of weather stations as patterns move from West to East, while Seattle gets its (relatively) skimpy data from satellites and scattered ocean stations.

Re:There are only 10 types of people

[AlphaOne]

BZZT. Try again.

On the 17th, at 09:14 GMT, winds were from 030 at 21 knots, gusting to 32 knots. Visibility was six statute miles with light rain and mist. The sky was overcast at 1200 feet.

Temporarily, for less than two hours, between 0900 to 1300 GMT, visibility will drop to 3 statue miles with moderate rain and mist and the ceiling will drop to 800 feet overcast.

From 14:00 to 21:00 GMT, winds will shift to be from 020 at 13 knots, gusting to 20 knots. Visibility will rise to 5 statute miles with light rain and mist. Sky will be overcast at 1200 feet. ... etc. They're actually easy to read if you do it often enough. I can usually decode METAR faster than I can read an ordinary plain-english forecast.

Re:A great job...

[GuidoDEV]

I've got an AthlonXP 2100+ and can run an 84-hour forecast using the MM5 in 2 1/2 hours. The domain is the continental US (plus some buffering around the CONUS, such as southern canada, parts of the atlantic/pacific, etc.), with a 40km gridpoint spacing and 30 vertical levels.

By contrast, the Eta model run by NCEP currently runs at 12km grid spacing with around 100 vertical levels (I believe), but the key is that the forecast from a 40km model run and a 12km model run usually differ very little, though the 12km run will have a little better resolution and may be a little more useful as a result. If the model runs have a resolution much better than 12km, however, you start running into problems where events like individual thunderstorms are explicitly resolved in the models' physics, as opposed to simply being "parameterized" (i.e., triggered in the model when certain larger-scale, favorable features are present). This sounds all well and good, but unfortunately the models are notoriously bad at explicitly resolving small-scale features like thunderstorms--much work needs to be done in this area.

In short, the average person can run their own weather model on their PC with no problems whatsoever, it no longer requires a supercomputer to do so. Good luck getting very skillful results beyond days 5-7 in your forecast, though.

Close, but no cigar

[Jerry]

As Dr Lorenz demonstrated in 1963, long range weather forecasting to any degree of accuracy is doomed to failure, even using several different mathematical models. Even accurate predictions no farther out than 5 days are limited to wind directions, barometric pressures, and the high and low temperatures. Precipitation, like thunderstorms, is 'predicted' in terms of percent probability that your area will get rained on, but even 0% or 100% predictions have often failed. Five years ago, on Oct 14th, none of the 14 inches of heavy wet snow we received was predicted. It downed over 50,000 trees in the city. Our award winning meteorologist spends about half of each broadcast, following a weather event, apologizing for mis-forecasting the previous day's weather. As Dr. Lorenz pointed out, the various runs of the model 'look similar' but that's not the same as a prediction, nor proof that the weather will even obey the model, regardless of what the model says.

Weather prediction is a standing joke. You are use to it. I am used to it. That's about best that can be done, despite all the high powered computers, mathematical models and and their theories. That's the nature of Chaos. Even when a thunderstorm is raging in the next county and heading in your direction there is no model that will predict if and when it will arrive. The 'meteorologists' at most TV stations use composite radar to 'predict' where storms are heading and when they will get there, and they make their predictions only minutes before hand, not hours or days, weeks or months ahead. I find that I can do exactly the same for the Lincoln area, with exactly the same accuracy, using the Omaha composite radar at www.crh.noaa.gov/radar/loop/DS.p37cr/si.koax.shtml

The best predictor for bad weather on the NOAA website I gave is the one-hour rain loop. But even when it shows a steadily advancing area of wetness, the "Great Wall of Lincoln" has unpredictable effects in diverting or suppressing rainfall. Ditto for snow and tornadoes.

The really arrogant folks are those who use models to predict global weather 50 years from now, even when they limit their 'predictions' to general high temperature 'averages' for regions like North America or Africa. Such dire 'Global Warming' predictions are fueled not by valid math models, because none exist, but by their political agenda. Those kinds of 'predictions' can only be classed as flagrant propaganda, and people willing to fabricate 'scientific' evidence for their political agendas scare me, just as much as folks who pass laws destroying my Constitutional Rights, while claiming to protect those freedoms from the actions of terrorists. They are from the same mold.

Here is a nice java applet demonstrating the Lorenz Attractor.
http://www.exploratorium.edu/complexit y/java/loren z.html