A British Supercomputer Can Predict Winter Weather a Year In Advance (thestack.com)

The national weather service of the U.K. claims it can now predict the weather up to a year in advance. An anonymous reader quotes The Stack: The development has been made possible thanks to supercomputer technology granted by the UK Government in 2014. The £97 million high-performance computing facility has allowed researchers to increase the resolution of climate models and to test the retrospective skill of forecasts over a 35-year period starting from 1980... The forecasters claim that new supercomputer-powered techniques have helped them develop a system to accurately predict North Atlantic Oscillation -- the climatic phenomenon which heavily impacts winters in the U.K.
The researchers apparently tested their supercomputer on 36 years worth of data, and reported proudly that they could predict winter weather a year in advance -- with 62% accuracy.

Comments

No it can't

[Mr D from 63]

I call BS on the headline. Let the damn thing prove it can do it before we claim it can. And doing regression model tweaking doesn't prove anything.

Re:No it can't

[michelcolman]

Hey, but 62% accuracy... that's 12% better than me!

Re:No it can't

[Mr D from 63]

I wonder if they did better than the Farmers Almanac?

Re:No it can't

[Anonymous Coward]

I call BS on the headline. Let the damn thing prove it can do it before we claim it can. And doing regression model tweaking doesn't prove anything.

Why? Predicting the winter weather in Britain is pretty simple. This little program will get it right about 90% of the time:

#include <stdio.h>
#include <string.h>
#include <time.h>
#include <unistd.h>

int main()
{
char date[32];
time_t rawtime;

time (&rawtime);
struct tm *timeinfo = localtime (&rawtime);
strftime(date, sizeof(date)-1, "%d.%m.%y_%H:%M:%S", timeinfo);

printf("[%s] Weather prediction: Precipitation\n", date);
sleep(86400);
}

So can I

[blogagog]

66% of the days in London contain some form of precipitation. So, I predict rain every day. I'm right 66% of the time. Wow, I'm smarter than a supercomputer!

Re:fallacy

[Calydor]

No, what they mean is they test it by feeding it the data from 1995, then comparing its predictions to what the weather was actually like in 1996. They are doing exactly what you say is the only way to test the validity of the data - they just started collecting data long ago.

THAT SAID, 62% correct doesn't seem all that awesome unless they use very tight margins. Does the computer say it'll be -10C and then count it as a fail if it's actually -11C? -15C? Does it say 'Good enough' if it says "Rain and 5C" and instead we get "Snow and -2C"?

Re:fallacy

[WhiplashII]

The problem with that approach is that you will tweak the algorithm until it works in 1996.

In other words, you will incorporate 1996 into the test set.

This is the big problem with almost all climate studies, and the reason why people that understand statistics really hate the current climate "science" as it is done. You really do need to make a prediction, and then test the prediction. If you get it wrong, you cannot re-try against the same data set until it works.

Re:fallacy

[rubycodez]

There are an infinite number of functions that can go through the data points of the past. I could make you 1,000 perfect stock predictors for past data.

Ask yourself, how did they refine and improve this model over time? It's nothing but a pile of cooked books

Re:fallacy

[phantomfive]

No, not at all, because doing what you described is incorporating brand new data every year.

They kept adjusting the algorithm over and over until they got the right answer from 1980 onwards. The huge risk with that method is overfitting, and if you develop an algorithm this way, it's important to also show that you've managed to avoid overfitting.

You can do the same thing with stock market data: adjust it until you get nearly 90% correct returns on a test interval, then you will find that the next year, the model is completely wrong because of overfit. Even if you incorporate the next years data, you will still get incorrect results because the nature of the stock market is chaotic and also random.

Really ?

[ctrl-alt-canc]

Arnold in one of his textbooxs demonstrated that, to make a weather prediction one month in advance, you need to measure pressure, temperature, wind speed and humidity with at least five significative decimals. He used sound mathematical methods based upon a theorem by Poincaré. With all the respect for technical skills and competence of people at Met Office, I trust more what Arnold demonstrated using nothing but paper and pencil. Good math is never overcome by brute force computation.

Re:Really ?

[ctrl-alt-canc]

When quoting Arnold I have been a little incorrect, since five figures of precision in the measurement of physical variables actually give you a two months forecast. But I studied this about thirty years ago...
If you want to estimate the error, if n is the number of months of the forecast and eps is the measurement precision, the error is given by:
10^(2.5n) times epsilon. As you can see the error rapidly increases, although the formula I transcribed from Arnold's textbook is quite rough (toroidal Earth, steady flux and negligible viscosity). Not a bad approximation for estimating trade winds flux, however.
People at MET probably took care of the propagation of numerical errors in the calculation, by increasing the grid density and maybe setting up a system capable of working with quadruple precision. However the problem again is the needed precision of input data, that increases exponentially with the time forecasted.

Re:62% is fail

[religionofpeas]

That's only true if there are two types of weather to choose from.

Re: fallacy

[Anonymous Coward]

And more ignorant nonsense gets modded Informative. The anti-science here is getting worse. Posters like you not only drastically overestimate your own knowledge of unfamiliar fields, you then insist to others it must all be a scam.

Weather and climate models aren't some arbitrary curve-fitting; they're physically based using ridiculously detailed physical simulations of air movements and ocean currents, starting from an observed state and running the simulation forward. Read up a little, and maybe you'll learn how to learn again.