Slashdot Mirror
Statistical Accuracy of Internet Weather Forecasts
markmcb writes "Brandon Hansen considers the statistical accuracy of popular on-line weather forecast sources and shows who's on target, and on who you probably shouldn't rely. Motivated by a trip to a water park that was spoiled with hail despite a 'clear sky' forecast, he does a nice job of depicting deviations, averages, and overall accuracy in a manner that stats junkies are sure to love."
(I work at the Canadian Meteorological Centre, but I am not a meteorologist myself)
:-).
One thing that struck me is the 'abnormal diversity' of weather information sources. In Canada, weather models are computed in one place, a ~1000 processors computer in a basement which does only one thing: forecasting weather (the constant real-world observations that are ingested are used to adjust the models). Only one 'real' source (of course, there's the american, british, french, etc. official forecasting models to which we compare 'scores' on a daily basis). However, there's plenty of other canadian websites which will give you weather forecasts (one example). From what I know, these "other websites" have a significantly smaller workforce of meteorologists to interpret the models results than the Meteorological Service of Canada (the CMC is part of the MSC). That's why I would favor the 'original' source instead of a 'second-hand' source. I must however admit, commercial online sources of weather forecasting sometimes offer value-added products, such as the number of ski trails opened, offer general weather information capsules, etc.
And by the way, the official Environment Canada weather website is the most visited website in Canada (or at least, that's what they tell us, the employees!
Animoog.org
I remember some years ago a radio presenter saying that you could achieve greater accuracy than supposed weather forcasters simply by using the assertion: today's weather will be the same as yesterday. Have we moved on from this position?
Any sufficiently advanced bug is indistinguishable from a feature.
The National Weather Service collects all the weather data used by forecasters, they also provide the 1st forecast. AccuWeather and others take the National Weather Service forecast then watch the new data (using National Weather Service provided data) to offer a refined forecast a few hours latter. Who do you think is going to be the most accurate the guy who provides the first forecast or the guy who waits for more data and then refines the for cast? AccuWeather's has statistics that show they are more accurate then the National Weather Service but if you used the AccuWeather forecast then waited for the next National Weather Service update I bet National Weather Service would be more accurate.
I am surprised that this guy used the weather.com and not the National Weather Service for the actually temp for all his calculations. (It doesn't matter b/c I am sure weather.com is right from National Weather Service data). He did point out that AccuWeather is the only one who provides forecasts > 10 days in advance.
My preference for weather forecasts is:
National Weather Service
AccuWeather (easy to understand graphics and 2 week forecasts)
The Weather Underground (Years ago they were the1st to provided free access to hurricane computer models)
I am not a meteorologist, but I have worked with them a few times.
Generally the competing weather models will show a range of possible outcomes with various probabilities. You can average across all scenarios and come up with a 60% probability of rain, but the more days out you go more the scenarios diverge, so the less useful a single average will be.
Most people would not find it useful to hear that "there will be probably be thunder on Wednesday if it remains hot enough, but if it cools down on Tuesday then the thunderstorm will be off to the north somewhere"
Additionally, a lot of weather conditions are influenced by thin layers of cloud high up, so thin that precise measurements are critical so precise forecasts in one location more than 3 days out are difficult.
What I find funny is that the entire practice of weather prediction is based on a logically fallacy. They take the data from previous years and say, ok, last time conditions looked like this x happened, so we predict x will happen again. Anybody who's taken an introductory logic class knows that you can't correlation does not equal causation.
And anyone whose understanding of correlation goes beyond "an introductory logic class" knows that in fact, as long as you're very careful about what you're doing, you can in fact very often use observed correlations to make valid predictions.
There's this whole field of study called "statistics," see. Not the "X% of people surveyed believe Y" type of thing you hear on the news, but an actual science, grounded in rigorous mathematical theory and growing more sophisticated all the time at producing useful knowledge from mountains of data. People get PhD's in it and stuff. Really. Maybe you ought to read about it some time. Maybe even take a class.
Or perhaps you'd rather remain secure in your prejudices, repeating "correlation does not equal causation" like a mantra, snickering at people whose knowledge you choose not to understand.
The correlation between ignorance of statistics and using "correlation is not causation" as an argument is close to 1.