Slashdot Mirror
A British Supercomputer Can Predict Winter Weather a Year In Advance (thestack.com)
The national weather service of the U.K. claims it can now predict the weather up to a year in advance.
An anonymous reader quotes The Stack: The development has been made possible thanks to supercomputer technology granted by the UK Government in 2014. The £97 million high-performance computing facility has allowed researchers to increase the resolution of climate models and to test the retrospective skill of forecasts over a 35-year period starting from 1980... The forecasters claim that new supercomputer-powered techniques have helped them develop a system to accurately predict North Atlantic Oscillation -- the climatic phenomenon which heavily impacts winters in the U.K.
The researchers apparently tested their supercomputer on 36 years worth of data, and reported proudly that they could predict winter weather a year in advance -- with 62% accuracy.
The researchers apparently tested their supercomputer on 36 years worth of data, and reported proudly that they could predict winter weather a year in advance -- with 62% accuracy.
I call BS on the headline. Let the damn thing prove it can do it before we claim it can. And doing regression model tweaking doesn't prove anything.
It is predicting climate, not weather.
I'm a good cook. I'm a fantastic eater. - Steven Brust
66% of the days in London contain some form of precipitation. So, I predict rain every day. I'm right 66% of the time. Wow, I'm smarter than a supercomputer!
No, what they mean is they test it by feeding it the data from 1995, then comparing its predictions to what the weather was actually like in 1996. They are doing exactly what you say is the only way to test the validity of the data - they just started collecting data long ago.
THAT SAID, 62% correct doesn't seem all that awesome unless they use very tight margins. Does the computer say it'll be -10C and then count it as a fail if it's actually -11C? -15C? Does it say 'Good enough' if it says "Rain and 5C" and instead we get "Snow and -2C"?
-=This sig has nothing to do with my comment. Move along now=-
Or just ask an indian. When asked how he could tell how cold the winters would be, one old chief just said, "I watch how much firewood the white man splits."
Have gnu, will travel.
The problem with that approach is that you will tweak the algorithm until it works in 1996.
In other words, you will incorporate 1996 into the test set.
This is the big problem with almost all climate studies, and the reason why people that understand statistics really hate the current climate "science" as it is done. You really do need to make a prediction, and then test the prediction. If you get it wrong, you cannot re-try against the same data set until it works.
while (sig==sig) sig=!sig;
There are an infinite number of functions that can go through the data points of the past. I could make you 1,000 perfect stock predictors for past data.
Ask yourself, how did they refine and improve this model over time? It's nothing but a pile of cooked books
Did they access to the 1996 data when they developed the model?
Well of course they did. How else could they test the 1996 predictions the model made from the 1995 (and earlier) data?
You build models by using prior data to adjust the model's parameters to "predict" new data, until the accuracy of the prediction is optimal.
You seem to imply that they cheated somehow. Generally, scientists are honest, with the exception of a small minority who are discovered by their peers and vilified.
If it weren't for deadlines, nothing would be late.
No, not at all, because doing what you described is incorporating brand new data every year.
They kept adjusting the algorithm over and over until they got the right answer from 1980 onwards. The huge risk with that method is overfitting, and if you develop an algorithm this way, it's important to also show that you've managed to avoid overfitting.
You can do the same thing with stock market data: adjust it until you get nearly 90% correct returns on a test interval, then you will find that the next year, the model is completely wrong because of overfit. Even if you incorporate the next years data, you will still get incorrect results because the nature of the stock market is chaotic and also random.
"First they came for the slanderers and i said nothing."
to predict British weather.
Arnold in one of his textbooxs demonstrated that, to make a weather prediction one month in advance, you need to measure pressure, temperature, wind speed and humidity with at least five significative decimals. He used sound mathematical methods based upon a theorem by Poincaré. With all the respect for technical skills and competence of people at Met Office, I trust more what Arnold demonstrated using nothing but paper and pencil. Good math is never overcome by brute force computation.
Because the human mind is incapable of bias and groups of minds are incapable of systematic bias?! There's a reason we say a real test of a prediction requires waiting for the real future. And this should be obvious to everyone. And before anyone tells "troll", smart intelligent honest people are as subject to bias as anyone, except because they know they are smart and honest, they are also subject to what's called "expert bias". It's just one more thing to be aware of as we pursue greater knowledge and insight. And it is unfortunate that many will dismiss experts purely because the experts say something inconvenient to various selfish interests and ignorance, but that's also just one more thing to bear in mind. Gaining knowledge is hard.
... and also impacts winters in the northeast USA.
That's only true if there are two types of weather to choose from.
And more ignorant nonsense gets modded Informative. The anti-science here is getting worse. Posters like you not only drastically overestimate your own knowledge of unfamiliar fields, you then insist to others it must all be a scam.
Weather and climate models aren't some arbitrary curve-fitting; they're physically based using ridiculously detailed physical simulations of air movements and ocean currents, starting from an observed state and running the simulation forward. Read up a little, and maybe you'll learn how to learn again.
Your claims of overfitting would mean something if they used a purely statistical model, but it's not - it's a physical simulation, constrained by laws of thermodynamics.
It amazes me that people say how trivial it is to fit statistical models perfectly to any random data (the stock market always gets mentioned here), yet don't think to wonder why they "only" got 62% accuracy. You'd think that this would be a huge red flag that your assumptions are wrong, but instead it's waved away as them all being dumber than a high school stats student - or more commonly a scam, with the researchers clearly hoping that no meddling high school stats students would notice.
And you would know, being so obviously well-informed about weather simulations.
If the person who wrote the summary knew anything about "weather simulations" they would be aware that climate is not weather!!!
And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
When you're pushing the boundaries, anything over 50% is good.
Is it? It depends on the data, the model, the thresholds for "correct forecast," etc. There are lots of places in the world where a "persistence" forecast (i.e., today will be the same as yesterday) will net you a greater than 50% accuracy within a reasonable margin of error. And one should also always consider forecasting models against general predicted climate averages. Again, taking those into account, a forecast system just using climate averages might do pretty well too.
It really depends on what the percentage "accuracy" means in this case and how it was measured. I'm guessing they wouldn't bother reporting it if it weren't significant, but just how significant is difficult to tell without the details (and it seems the full research paper is behind a paywall).
Otherwise citing a number like "62% accuracy" is utterly meaningless. If you had a task like, "Guess how tall the next person to walk into the building will be," and I achieved 62% accuracy, that could be remarkable and improbable if the margin of error was 1/8 of an inch. But if I instead was guessing "Taller than 1 foot or shorter than 1 foot," then 62% accuracy might mean I'm mentally retarded.
You're talking about overfitting.
The thing is they aren't doing that regression-and- frigging-the-coefficients thing. It's a physics based, bottom up, method.
Nice armchair.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
The headline is completely wrong. They can predict the NAO with much more accuracy, and as that has such a big effect on the winter weather in the UK, that lets them say (with only 62% accuracy) whether it'll be mild winter or a cold one. That's all. They can NOT, of course, predict the weather anything like that in advance. They didn't claim to, either - lazy new editors and "journalists" have mangled it into the bullshit we read here.