Slashdot Mirror
Statistical Accuracy of Internet Weather Forecasts
markmcb writes "Brandon Hansen considers the statistical accuracy of popular on-line weather forecast sources and shows who's on target, and on who you probably shouldn't rely. Motivated by a trip to a water park that was spoiled with hail despite a 'clear sky' forecast, he does a nice job of depicting deviations, averages, and overall accuracy in a manner that stats junkies are sure to love."
(I work at the Canadian Meteorological Centre, but I am not a meteorologist myself)
:-).
One thing that struck me is the 'abnormal diversity' of weather information sources. In Canada, weather models are computed in one place, a ~1000 processors computer in a basement which does only one thing: forecasting weather (the constant real-world observations that are ingested are used to adjust the models). Only one 'real' source (of course, there's the american, british, french, etc. official forecasting models to which we compare 'scores' on a daily basis). However, there's plenty of other canadian websites which will give you weather forecasts (one example). From what I know, these "other websites" have a significantly smaller workforce of meteorologists to interpret the models results than the Meteorological Service of Canada (the CMC is part of the MSC). That's why I would favor the 'original' source instead of a 'second-hand' source. I must however admit, commercial online sources of weather forecasting sometimes offer value-added products, such as the number of ski trails opened, offer general weather information capsules, etc.
And by the way, the official Environment Canada weather website is the most visited website in Canada (or at least, that's what they tell us, the employees!
Animoog.org
The National Weather Service collects all the weather data used by forecasters, they also provide the 1st forecast. AccuWeather and others take the National Weather Service forecast then watch the new data (using National Weather Service provided data) to offer a refined forecast a few hours latter. Who do you think is going to be the most accurate the guy who provides the first forecast or the guy who waits for more data and then refines the for cast? AccuWeather's has statistics that show they are more accurate then the National Weather Service but if you used the AccuWeather forecast then waited for the next National Weather Service update I bet National Weather Service would be more accurate.
I am surprised that this guy used the weather.com and not the National Weather Service for the actually temp for all his calculations. (It doesn't matter b/c I am sure weather.com is right from National Weather Service data). He did point out that AccuWeather is the only one who provides forecasts > 10 days in advance.
My preference for weather forecasts is:
National Weather Service
AccuWeather (easy to understand graphics and 2 week forecasts)
The Weather Underground (Years ago they were the1st to provided free access to hurricane computer models)
In the forecasting vernacular, that method describes forecasting based on "persistence".
Persistence is the yardstick all forecasters use to determine if they should find another line of work (or be asked to do so by others). If you can't demonstrate an understanding of the processes and data such that extend beyond the data source everyone else has (ie., the weather their experiencing), it's just snake oil.
(IAAM)
The little guy just ain't getting it, is he?
persistence is 75% reliable.
On any significant sample, weather reports were never worse than this.
Currently, models are able to make 85% or a little more accuracy.
This may sound paltry, but where this really works out is in longer term forecasts. At 75% you are probably wrong at 3 days forecast. Even if you take the assumption that forecasts are independent from day to day, 85% means you are probablt wrong after 5 days.
The extra two days you can predict for is what the money is going towards.
Geographically large metro areas -- especially those with hills or large bodies of water -- make a weather forecaster's job all the more difficult. The chance of rain may be higher on one end of town, but it's difficult for a TV or radio announcer (or a newspaper spread, for that matter) to present the distinctions clearly and quickly.
Too long ago, when I was an undergraduate taking Meteorology, we visited the weather department in a Twin Cities (MN) television station. The anchor on duty was pretty blunt: if there's a 100% chance of rain on one end of town and a 10% chance on the other end, the broadcast would distill that as a 55% chance of rain. He argued that it was the best his department could offer given the commercial realities of limited airtime and the mandate to serve the entire metro area.