Slashdot Mirror
Distributed Computing World Climate Simulation
Burnt Offerings writes: "The BBC reports that scientists at climateprediction.com are nearing the completion and public release in late summer of a distributed computing project that simulates the world's climate from 1950-2050 AD. It seems that each user's simulation will have different initial conditions built into their runtime simulation and a single completed simulation from 1950-2050 AD takes on average eight-months (Doh!), assuming average household computing power. The results will be sent back to the project's team, where they will select the models that resulted in the 'real' climate patterns that have occured since 1950-2000. I presume they will then use these validated models to help extrapolate the world's climate from 2000-2050. Pretty cool (or should I say warm? or hot?)."
... I hope that they save state information for those occasional reboots ...
doesn't a year have 12, not 8 months?
Slashdot Hypocrisy at work?
... for me. I couldn't vote on the latest Megahertz poll, my stately 33 MHz 486 didn't even have a category in which to put it.
For me, a single completed simulation from 1950-2050 AD should take a little over 100 years. Can't wait to get started.
With luck, however, I should get the right answer.
The end result of the project:
"On 1st January, 2050, it will start rather cloudy with outbreaks of rain, mainly in the north. These will clear up by late afternoon, leaving it warm with mild breezes in most of the country."
graspee
On this day in 1950, it was raining. The rain was as pure as Evian.
On this day in 1980, it was raining. The rain was as pure as the innards of a Duracell battery.
If it wasn't, we'd have accurate forecasts up a few months in advance. As it is, I find forecasts are routinely wrong about even tomorrow's weather. What happened to the hole "butterfly flapping its wings in Singapore affects the weather in Kansas" thing? I don't see how initial conditions would tell them much, I bet even random quantum events have a very strong influence on weather models over 50 years. I'd put the odds of success for this distributed computing project around the same as SETI.
Websurfing done right! StumbleUpon
Blame the climate changes from 1950 to 2000 on the expanded use of the automobile and unregular industrial waste. Do you think any scientist in 1950 could have known about our current situation? How can we in 2000 know about the new problems that'll creep up between now and 2050?
Spend your extra CPU cycles computing the cure for cancer or finding ET. I doubt this will prove useful.
Now they will be able to prove with computer simulations what the weather will be like tommorow and still be wrong... Gosh what a marvle of modern technology. I wouldn't be such a cynic except there are so many variables that go into weather predictions that any attempt is still a guess...
I like replies better than Karma, even if they are flames, because that tells me I got someone thinking.
I'm always a little sketchy on joining these distributed computing projects.
Something about my computer time being put to work so that a bunch of scientists can invent a new drug and make lots of money; or put out a new study and get some fame. It just doesn't seem right.
I'm not going to reprint the page ,HadSM3,HadCM3,HadCM3L)
unless it get's slashdotted, but none of the models (HadAM3
in the simulation take into account the biological factor.
It has been said, that both termites, cars, factories, cows, and Taco
Bell produce huge amounts of greenhouse gas which do attribute to global
warming. How can this lead to an accurate prediction model if these factors
aren't accounted for?
My understanding of the el nino weather patern is simply "something pretty weird is going on lately" causing droughts were there shouldnt be and ditto with floods. given that this odity will be in only say 40% (or the last 20 years) of the data they are fitting wont any extrapolations be pointless.Add that to global warming and i am seeing wasted CPU cycles
The Borg assimilated my race & all I got was this lousy T-shirt
So, let me get this straight: they're going to pick generated results that most closely resemble real, measured results, and then adjust their model to compensate.
Those models wouldn't be "validated" as the poster claims, or would they? It seems to me that without identifying the reasons the computed models differed from the measured results, the selection is damn near arbitrary -- the difference may be something the scientists never considered.
I've been wrong before.... once.
No doubt the odd geek has a room full of Alphas to add to the cause.
Protoplasm. Quiet Protoplasm. I like quiet protoplasm.
Last frost
It seems like there is a bit of professional dueling going on between this project and Seti@home looking at their FAQ and the quote by Dr Meyers Allen saying about their project "It's not a stripped down 'toy' version, so the runs take time"
My favorite quote from their FAQ was in response to the possible affect the computers running the client might have on the environment:
"To travel the paths of human imagination you have to be willing to unlearn all you know"
If this thing takes eight months to complete, I sure hope they plan on storing periodic checkpoints of progress for each test in a central location. What happens if my machine gets hosed at four months? Is all that data lost?
This is all well and good, but a much more effective distributed computing project would be one that would help the National Weather Service (where most meterological outlets still get their info) predict the weather for the next few days. They use a series of computers for simulation and then make an educated guess based on the runs of those models. Imagine, however, if instead of four or five test runs, they've got thousands to choose from.....
Just a thought.
The information on their website says the time step is 30 minutes and that their box is 3.75 degrees longitude by 2.25 degrees latitude (or visa versa: BIG, in any event).
Therefore, how do they expect this to work -- additionally absent any outside changes in the environment?
What I mean is, how do they know if they did a good job? Perhaps if the results are all very close to the current day climate, I'd buy that they got it right, but if they have a reasonable distribution of results, how do you decide? I mean, we've been clear-cutting the hell out of forests left and right for years: do they somehow takes this into account? Heck, how do they present the geographic information about the Earth: this bit has forest, this bit is desert. I would think that this would make quite a bit of difference in results (changes in albedo, for instance).
I certainly wish them luck, but they're not getting my PC for that long without something more detailed , informationwise.
Weather is chaotic, but climate is ... well, ok, climate might be chaotic, but we really don't know -- and if it is chaotic, it is still only chaotic on timescales of more than 50 years.
Predicting climate 50 years in the future is a computationally difficult task, but it isn't impossible the way that predicting weather would be.
Tarsnap: Online backups for the truly paranoid
Global warming accelerated by CPU heat as weather enthusiasts simulate climate with computer. Temperature for the next 2 years will rise by 2 degrees
This is cool. Beyond being used to understand the current climate change that is happening, obscure weather phenom could be modeled on a larger scale for a longer time.
Perfect example would be an article out of the latest AMS Bulletin of the American Meteorological Society Earth Interactions that discusses plane contrails. It seems that the lack of air traffic after 9/11 allowed the meteorlogist to work on a long held theory that plane contrails affect weather. Only problem was that the dataset was only over three days, which was just a small time sample.
Using a system such as this, those weather conditions could be recreated over a longer period of time and the results could be realized. Too cool.
Bryan R.
The price of freedom is eternal vigilance, or $12.50 as seen on eBay.....
Pretty cool (or should I say warm? or hot?)
You should wait until the results come in.
Got friends?
Pretty cool (or should I say warm? or hot?)
Say "cool", global warming could lead to an ice age. One theory predicts that warming can lead to too much fresh water being introduced into the north atlantic and decrease salinity levels beyond a key threshold. This in turn "shuts off" a descending (vertical) current, which in turn disrupts the gulf stream (horizontal) that currently sends warm water north, which ultimately results in cooling in north america and europe.
FWIW, there is evidence that the above occurs fairly regularly on a geological time scale. Man's efforts may or may not have much of an impact, it may or may not be egotistical to think we can change weather patterns with our SUVs. Perhaps if we have an impact the system was teetering on the edge in the first place. Not that this justifies a push over the edge.
In simulation A we set the Funding Amount variable to 0$ and the Donating Corporation to NULL. Their results was intense global warming in 2050.
In simulation B we set the Funding Amount variable to 200,000$ and the Donating Corporation to Exxon Mobile. Their result was no global warming at all in 2050.
In simulation C we set the Funding Amount variable to 300,000$ and the Donating Corporation to Amazon Lumber Harvesters. Their result was an actual decrease in green house gases by the year 2050 due to deforestation.
In simulation D...
Outdoor digital photography, mostly in New Engl
Pretty cool (or should I say warm? or hot?)
Wait eight months and tell us!
I like this idea. I'm no expert in meteorology, but it seems that the models for predicting weather are refined enough to do long range predictions, but the models are extremely sensitive to initial conditions (i.e., chaotic in the mathematical sense). Rather than go out and measure the initial conditions, they will guess the initial conditions by trying lots of them and finding the right set (from 1950) that correctly predicts the weather.
What is climate but (basically dumbing it down) taking the average of the last x number of years of weather to define the norm. So, to define what the climate is fifty years into the future, one would have to take a look at the weather for each of those years. I agree that is no small task.
I must take issue with the parent post, though. I agree that weather is a choatic system, very much so. But, all aspects of weather can be parameterized, even the most chaotic ones. The key here is a matter of scale. The mesoscale type systems are extremely hard to model, but you take a global system (long wave patterns), and you will have a much better time of modeling them. How? You throw out the small scale stuff like your butterfly and such. On a global scale, something like that would quickly disappear into the larger scale. That is why global models (like the MRF, NOGAPS, and such) work better out farther (those models run out to 384 hours as opposed to smaller scale models that run out 84). Verification rates are acceptable for those models out that far (numbers I cannot quote off the top of my head). They could do better, but they would require more time to process and would not be useful to the operational meteorologist.
This distributed system will be over eight months and on such a large scale, the results will be useful.
Bryan R.
The price of freedom is eternal vigilance, or $12.50 as seen on eBay.....
They haven't taken into consideration the existence of AMD CPUs :-)
I suffer from attention surplus disorder.
So will it be sunny when the time_t's wrap around?
It's generally regarded as a Bayesian technique. Actually, there's far more to Bayesian statistics that bootstrapping, but it's the part I spend a lot of time working with. In fact, I suppose that bootstrapping isn't fundamentally a Bayesian process, but it is highly empirical so it appeals to the same "crowd" as more decidedly Bayesian techniques. By the by, "Bayesian" statistics are statistics that make heavy use of Bayes' Rule to incorporate prior knowledge not included in your measured data.
My background - you develop a program to predict something biological. Let us say, to pick a problem on the same order of difficulty as predicting the weather, that you're trying to predict the three dimensional confirmation that proteins assume, based on their sequence.
Now, okay, you have a bunch of known sequences, which other people (personally, I do both the data mining and some crystalography) have attached to known structures. So, what do you do?
Well, you could fiddle with your program until it predicts really well on those sequences, and announce that it was good. This is "Bad Science", as the parent-poster points out, since the criterion are arbitrary - you have a tendency to "discover" random noise in the data, and you have no way of validating your results.
So, second option. Instead, you split the data in half at random (actually into more than 2 pieces, but conceptually in half.) You take one half, and you make the model predict as well as you can on that data. Then, you VALIDATE ON THE OTHER HALF OF THE DATA. You *never* change the model on the basis of the second half of the data - that is arbitrary/bad/cheating. This is called "bootstrapping". It has nothing to do with compiler installation.
So, as far as most scientists (as opposed to mathematicians) are concerned, the important question is - does this work? In the biological sciences, I can say categorically, yes, this bootstrapping technique has a proven track record. It does work. Obviously, you can screw up (using non-representative data is a good start) but the technique, when properly applied, is sound.
So, I assume it would work for predicting the weather, as well. By work I mean - you would know how well your software predicted the weather. Bootstrapping is not a means of predicting the weather in and of itself, merely of honestly evaluating the effectiveness of a weather prediction mechanism you already have.
The good and new comes from no quarter where it is looked for, and is always something different from what is expected.
Climate - The condition of a place in relation to various phenomena of the atmosphere, as temperature, moisture, etc., especially as they affect animal or vegetable life.
Sorry to fall back to dictionary definitions, but this sure sounds like weather to me. Maybe averaged on a longer time scale, but it's still quite obviously a chaotic system. We've found loose correlations with sunspots, deforestation, etc.. but even very large trends like the "little ice age" of 1500AD are unexplained and most likely chaotic. If we can't explain hundreds of years of pronounced trends, I don't see how we can do anything with the relatively uneventful last 50 years.
Websurfing: The Next Generation - StumbleUpon
Microsoft has developed a very similar distributed simulation software package. Last I heard, it would only take 3-5 months to recieve the results, too. A savings of 3 or more months. Rumor is, they plan on using it so that a person can run Office XP. Finally enough cpu power to run it quickly... I'm sorta jealous.
That's what makes weather choatic. If you start with the exact same conditions, you still don't get the same answer at the end. That's why the weather predictions for tomorrow are so often wrong. (i remember when they said 'there's an 80% chance of rain tomorrow', but now they just tell us it'll rain tomorrow).
Now this is a climate model and not a weather model, but I fail to see how the hell that's anything more than a labeling difference.
Okay I am not first to say this. But 8 months?!?! What are the odds that I will run it for 8 months... continously... without being distracted by another DC project?
/. Now imagine that your PC generated exact results from 1950-2000. A perfect model.
Also these people are entirely too green and liberal for my tastes. At first it is a very thought provoking idea. But these people already have preconcevied conclusions... and that isn't very good science.
Try this little thought experiment.
You can get the gist of what they are doing by reading
And then its results said that from 2000-2050 the Earth would cool down to 1950 levels.
Nother words no global warming took place.
Now go and read there website....
What do you think they would think of those results?
That isn't good science.
I mean these people have yet to generate a product and they are already tossing about the possibility of Oil Companies messing with there results.
Hello... McFly???
You know what I would really like to see?
Someone that doesn't have a chip on there shoulder take this project over.
Recode it so it doesn't take no 8 monhts.
1 month is probably the maximum it should take.
And cut all the Green Liberal Save the Whales crap out of it.
Then I might be interested.
Coincidence? I think not!
...
Seriously, this sort of modeling will take less time as processors scale bigger and Internet connectivity proliferates. I would like to participate, but it would be nice if I didn't have to run an MS OS to do so. I can, do and probably will, but if they would just release the source
Disk space: Need 600MB free to allocate to this experiment.
Wow.
They're starting with different initial conditions and hoping that some subset results in 50 years of weather?
Shouldn't they use the last 50 years of weather as initial conditions and vary parameters of the model instead?
What they're doing is like flipping an imaginary coin 500 times hoping to match the first 250 flips of a real coin to predict the the last 250 flips (albeit in a system with non-independent trials). But then they're taking those 500 flips and matching the first 250 to weather reports (might as well be coin flips) and then imagining the next 250 flips will approximate the future weather reports. What they need to do is fix the initial conditions and modify the model (coin flips vs. rolls of the die vs. LCRNG, etc.) to find a model that approximates the dynamics of the system.
Am I making sense here? How are these bozos not just going to apply their effective innumeracy to waste a few trillion CPU hours that could otherwise have been used to do protein folding or cancer-killing molecule matching?
--Blair
I think a potential problem with their energy use calculation of machines running the software is that it does not include monitors' use of energy. Most people have a screensaver running rather than having monitor set to sleep. Don't monitors use as much or more energy as CPUs?
/^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$/i
i think this would be more useful predicting the big picture, droughts, blizzards, hurricanes,and the like. Living in the Virgin Islands I pay quite a bit of attention to Dr. Greys hurricane reports. He uses computer simulations from data of years past to come up with a reasonable estamate of the next hurricane season. Most years he's close, a few years he's been dead on. Now what can we do about it..well nothing.. i've been through 7 hurricanes. one of witch was 200+ M.P.H. winds. and you can't do anything about it.. but if this type of predictions can be used for droughts or any other farming purpose it would greatly increce the world food supply...Think of the children..yea i know a bit sappy but hey anything that helps us lowley Humans get along better in our enviroment i'm all for it....
Just Limin' Mon
My CPU hours will remain dedicated to searching for a cure for diseases. If you would like to help check out the Folding@Home project that uses distributed computing to model protein folding to find possible cures.
http://www.kubuntu.org/
Why don't we quit wasting time trying to predict major climate change and start taking action to clean up our act?
Have you ever thought of how much garbage the world population puts out, trees we cut down, pollutants we flush, and general mayhem we induce?
Maybe we should be using our excess computing time into working on projects that actually might affect our environment in a positive way, rather than saying we should see what it is going to be like down the road...we all know what is going on here, and I'm not talking about global warming.
Its not the effect of global warming that is our problem right now, but the effect of our blatant misuse of resources and obvious disregard for the earth. Do we not live on this planet with the environment we are destroying...I don't think you need to be a very good scientist to realize that when the environment is decimated, we will be hard pressed to survive...
I guess everyone has some idea that God is going to come and fix everything for us, so we don't have to worry about cleaning up...hey, why don't we all call our mommys and see if they will do our work for us...why don't we own up and say, "Holy shit, I don't want to take the chance that my children are not going to grow up because I ruined their world for them." What is our general purpose in life besides taking up space, making money, and destroying the environment?
The world is a big place, but eventually our actions are going to reach around to spank us, just like our mom's did when we were bad...except it won't be a spanking we live through:/
I invite everyone to spend their 8 months attempting to exact reform in our environmental policies and personal resource use, rather than hoping your computer will somehow figure it out for you.
--"It's Bradford Company, slash your last name, dot your first name"
"And it's predicted that the polar ice caps will melt by 2030, resulting in... Oh.. Wait. That's right... We're missing the data from User #62280122 because he disconnected... Let's try this again..."
You need a FREE iPod Nano
No.
No you shouldn't.
...this is a climate predictor, not a weather predictor. No criticism necessary. :P
I question how good the extrapolation is going to be. Their using 50 years of fit data to extrapolate 50 years into the future. Normally, I would think thhat you would need a much larger set of fit data to extrapolate out acurately. Why can't they go back to when acurate records of the climate started being made, like 200-300 years ago? Then the R^2 value of the fit would be higher and a more acurate model would result.
I wonder if their simulations will take into account the heat produced by the computers running the simulations... since this is a distributed computing project, there can potentially be thousands of computers or more, and the heat produced will add up. However insignificant that may seem, I'm willing to bet that the weather will somehow be affected.
Also, whatever result they got from the simulations, however accurate it is in simulating the past 50 years, will not be accurate enough to be used to extrapolate the next 50 years. It won't be able to predict any future technology that could either pollute the planet even more, or clean up the planet (i.e. some kind of artificial and energy efficient carbon sink). Moreover, it won't be able to predict future natural or man-made disasters, some of which are unrelated to the weather, for example, meteor impacts. And lastly, they can't predict human behavior... someone can go out in the middle of nowhere and set an entire forest on fire, which will surely affect the weather.
It will be great if they can predict tomorrow's weather to a 99.99% accuracy, but knowing the weather for the next 50 years is simply impractical, not to mention impossible.
I do not see a real point to this application, as it has the potential to give a lot of misinformation. I think if the analysis that they are doing is on a very basic level and not taken too seriously that is OK, but I wouldn't take it as gospel.
It seems that this could make for a real headache, splitting the workload up onto all of these different computers. It's not data like Seti@home where you can distribute out data pieces, is it? All of the information needs to be there to simulate the planet. It sounds like it would be more effective to just get the fastest supercomputer they can get their hands on and start work on a more thorough level, like Japan is doing. Otherwise...
;)
"How's the global climate simulation going?"
"We're still waiting on the data from Australia. We sent it out to 5 people but we haven't gotten anything back yet."
In the meantime, the Earth's atmosphere bursts into flames and makes the whole point moot.
Remember "Bring 'em on"? *sigh
To be able to install and run such simulation on your PC, we recommend the following minimum system specifications:
-
Operating System: Windows 95, Windows 98, Windows ME, Windows 2000
Professional, or Windows NT 4.0
-
Processor: Athlon, PII, PIII, PIV
-
Speed: Min 450MHz. Preferably 700+
-
Memory: Min 128 MB at this stage. (64MB may be enough but we havn't
tested) for use
----- Yep, there you have it...As one poster has pointed out, weather is a chaotic system (and climate is also chaotic by definition).
Chaos is gravely misunderstood though so let me real quick through in my explaination for why this experiment will just generate FUD.
Chaotic equations are chaotic not because of the number of variables involved but because of the interdependency on themselves (each iteration requires the former iteration). This leads to extreme sensitive dependency on initial conditions (a.k.a. the Butterfly Effect). I should have probably emphasized the word extreme because even the slightly deviation will produce dramatically different results.
Even the best climate prediction algorithm would be crap if the initial condition was off by 10^(-20). The fact that we cannot measure temperatures exactly means that we could never feed a perfect initial condition.
Chaotic equations do have a given period before divergence gets extreme when initial conditions are altered. The original equations that Lorenz used (the pioneer of weather forecasting and the father of Chaos theory) showed divergence after about three days (which is why five-day forecasts still suck to this day).
I find it very hard to believe that these folks have developed an equation that doesn't show divergence for 100 years. Not to mention the fact that the number of initial conditions are much larger than the project makes them out to be.
Summary: Some PhD is looking for research money and figures that mixing "scientific" proof for global warming, chaos, and SETI-style distributed computer has to be good for a couple million at least.
int func(int a);
func((b += 3, b));
... or at least the best science has come up with so far, are downloadable from the Intergovernmental Panel on Climate Change (IPCC).
I'd start with the Summaries for Policy Makers, as a way of becoming very well infomrmed in just ~20pp.
AFAIK: It's a UN organization that is the center of research. Their reports are a consensus of almost all the leading scientists from every country on the globe, and their policy statements are approved line-by-line by governments. Even with all that, there are pretty strong statements.Here's better background.
goddamn thats an annoying motd
"The reader this message encounters not failing to understand is cursed."
You know where you are? You're in the $PATH, baby. You're gonna get executed!
Yet another attempt to model a multi-billion year old climate based on a short data stream.
Let's estimate the average income of everyone in the US over time by looking at people in Rhode Island for the last three days. Same sampling scale, or close.
Useless experiment to hype up the global warming debate again. Gee, I wonder if they'll pick any of the initial conditions that say "things aren't so bad after all". Nope, the only starting conditions that will ever see the light of day are the ones that back up their theory.
Not that the science on the other side is any better. I'm getting tired of the entire debate because, guess what kids, this is supposed to be SCIENCE. Not prognositcation. There is a difference. Come up with a theory, build a series of experiments to prove it, and see if it sticks to the fridge or not. All I'm seeing here is "come up with a theory, pick the data points that will support it, and then publish it in the NY Times".
Actually this system can produce a 100% accurate weather forecast for the following day. Unfortunately it takes 48 hours to calculate.
Thanks for the explaination!
I'm currently working on an application that monitors seemingly random data -- the stock market. I never stopped to consider that there may be statistical techniques above and beyond the standard technincal indicators.
Food for thought!
"...and by 'country' we mean Antarctica."
Snarkiness is inversely proportional to wisdom because it emphasizes feeling right rather than being right.
This silly experiment is a waste of time. Everyone with a time machine already knows that my massive Weather Altering Device (WAD) will come online in 2008 with the sole purpose of ruining the results of this trial...
------
Today's Top Deals
This is silly, Boundary Conditions are not useful unless you have the right Partial Differential Equation.
-Jay
http://www.uiuc.edu/~jthomas2
Hey, someone tell me what data is being sent to redsherrif.com.
The BBC site runs some java app that then send data....
anyone give any info on this?
with the genetic algorithm approach to selecting the fittest weather prediction.
Elementary, my dear sensitive dependence...
?sp
Presumably they are using this to determine various model parameters and test different models. This will be different from a lot of drug trials since a particularly bad kind of extrapolation is needed. We want to apply the model using high levels of CO2, but none of our data can test the validity of the model in the high CO2 region. However there is no way to overcome this problem, so we may as well give it a shot. Some prediction is better than none.
Also: as you say, it is better to use some data for "training" (ie., estimating model parameters) and other data for validation (estimation of error). It would seem that this study uses the same data for both. This will bias the estimated error to be too small. However since this is time series data I don't know a better way of doing it. (I wish I knew more statistics.) But I'm sure people have already worked out better, unbiased methods.
So although this study is an interesting effort I personally will not have great confidence in the results, especially if they disagree with previous results.
Having lived through some hoaxes already (Can you say Arab Oil Embargo?) I'm hesitant to jump on board this sky-is-falling craze.
Did anyone think to notice that the measuring stations over the last 100 years have become increasingly surrounded by the city's concrete?
Has anyone been at 30,000 feet lately to see just how tiny man's domain is compared to nature?
I keep hearing about global warming, but shouldn't something signalling the end of human life as we know it be **somewhat** universally accepted by scientists across the board? (See also: Ozone Hole, the Bermuda Triangle)
--- For a good time mail uce@ftc.gov
Yeah I know exactly what you mean. God forbid that the economic incentive from those $10 pills should ever motivate a corporation to pour all the resources and talent necessary into curing any specific disease. We sure as hell don't want any diseases cured if someone's gonna get rich from it!
I mean, all diseases are as easy and simple to cure as polio or tuberculosis right? There can't be any hardy viruses or bacteria that require more research then your run of the mill government agency/lab could provide.
Mac OS X and Windows XP working side by side to fight back the night.
global cooling, or global warming?
I'm glad they're running a lot of different models.
It will be interesting to see how divergent the predictions for the next 50 years are from the best fits to the past 50 years.
It will also be interesting to see how badly the best fits for the next 50 years fit the past 50 years. (There's gotta be a better way to phrase that)
There's also the long term effects that we have no good means to capture, like what turns off and on the various ocean currents.
cos there are a lot of them out there sadly but i never get y thies ppl that do the all this distrupited computing dont open sourse there code so if some once wants to port it to a os they can insted of wating for there slow brains to figer out we arnt all stuck behing the fences and walls
maja
people have pointed out that you can't extrapolate a 50 year prediction from 50 years of data. I think its more likely that they're using the data from the last 1000 years ("The increase in surface temperature over the 20th century for the Northern Hemisphere is likely to have been greater than that for any other century in the last thousand years" from http://www.ipcc.ch/pub/tar/syr/004.htm).
So, they're PRETENDING like their in 1950, then using all that data to predict 50 years into the future (which, we know of course), then, based on whatever prediction is most accurate, using THAT to predicate the NEXT 50 years (which we don't know)
its a little vauge, but i think that might be what they're doing.
...especially considering the nature of distributed computing where participants might sign up on a whim and then drop out a little bit later because they have to reinstall everything or upgrade their system or change work or simply gets tired of the project or it conflicts with some other program or gives their system performance degradation.
I don't know how much amount of immediate data that needs to be stored, but there definitely should still be a mechanism for periodically sending up progress-dumps so that somebody else can take over from wherever you were. This could at least shorten the time for having all the data run since you would notice participant drop-outs earlier and could hand over the rest of the calculations to another participant.
It could also be used to sort out really bad seeds at an earlier stage where the system, for example already after 10 or 20 years discover that you are way off and could hand you another seed instead.
This is actually a fairly normal technique.
I am doing agent-based modeling right now involving artificial societies: we create a set of assumptions (which is what the designers of this experiment have done and subtly varied) on the data that we have, then we run the simulation (in our case with variable starting conditions) and see if the end results come out within tolerances for what we know about real societies. (Heavily simplified, but you get the idea)
This is a fairly common technique for verifying a model.
Integrate Keynote and LaTeX
Well, the problem is that they are actually using non-representative data. 1950-2000 is a too small sample by far to even begin forming a model for climate variation, something which varies over periods of centuries or millenia.
They will probably get some form of result. It wont be valid, but it will nonetheless be a result which matches the earlier period.
Of course, this will start breaking down as soon as natural climate variation changes cycle. Likely it would be invalidated even faster if they try to apply the model to known data from the last 20k years (altho if they could get the model to account for the earlier climate variations that far back, I'd tend to accept it as more valid).
Your understanding of chaos is *wrong*. Initial conditions *do* matter, it's just that the interactions are incredibly complex and therefore even more incredibly hard to predict. But if initial conditions don't matter, you'd best reconsider any use of the old "cause and effect" idea.
another example of usage genetic algorithm is image recognition. take a 1 bln PCs, run it for 1000 years and choose one which gives correct answer "It's a dog on the picture". Do you think that the next it's answer will be right ?
I'm really quite worried about the vulnerability of this study to sabotage, be it from the mind-boggling resources and total lack of ethics of Intergloms & Megacorps; our illustrious philosopher king, Jr.; or by teenagers with poor social skills.
;) ?
What got me really worried was the website's amateurish appearance and claim that they were protected 'pretty well' from this sort or attack, but that this project was much harder to protect from or detect attacks than say SETI@Home. (8 months a run, no redundancy. SETI responds with the locations of hits which they can go back to the original data to have a closer look at. These guys need to do a 8 month run just to verify one overly 'sunny' result..)
IIRC, SETI had to restart from scratch (changed study parameters; not redundancy) after people started feeding them bad info in attempts to make themselves top the 'best triplet' high score list on S@H's website.
This is a critically important problem for peoplekind to solve-- which is why I am so worried that a semi-organized effort by Big-Oil or whoever could easily plug in enough random error as to cast doubt over the results of the whole project. Doesn't take much. Worse, seeding in bad data which doesn't get flagged, and bad results get published driving future policy.
These guys need serious crypto help, and now! before they get started full bore.
-posting AC so They don't come and get me.
seriously though, I don't think I'm being too paranoid (about the scuttling/attempt to skew results, that is. 'They' already know where to get me. The tinfoil hat is easily detectable from the new "NASA" hydro-sphere satellite.)
You can describe the weather 1950 - 2000 and then get it wrong afterwards, if your model just summarizes the data 1950 - 2000 instead of getting the underlying trends, which will recur after 2000. Knowing the stability of the system and what future shocks might invalidate your analysis would be nice though,. Kind of similar to:d =3322 825
http://slashdot.org/comments.pl?sid=30887&ci
[Is using someone elses post to back up your own:
a/a breech of copyrite
b/cowardly
c/cheating
d/Cowboy neal
?]
Fresh 'raw' data is your friend. See for reference 'news'.
Be Free: Free Software Tuition
Now if setiathome would just find some ET intelligence, I would have some processor time available for this... But until then... seti stays on my pc.
btw... 8 month average ? That would cost MY computer so much time, we will be able to walk outside to watch/feel the climate it is trying to predict LOL...
a) what makes you think TB is easy and simple to cure? TB is resurgent across much of the world.
b) while your point about the value of commercial research is true, it is nonetheless the case that many drugs originated in government or other publicly funded labs and were brought to market by drugcos. the pharmacos provided the deep pockets to run large scale phase III trials and to market the drugs. of course, many other drugs have been developed commercially, and many have been derived from natural materials.
At Last, the latest quake engine will not just show you some dodgy sky but the actual weather that is outside, woo hoo
Climate is not necessary chaotic if it is considered to be a moving average of weather. It is entirely possible and indeed quite likely that the non-linear fluctuations which make weather prediction so difficult to predict are in fact damped out over longer time periods. Or to put it in chaos terms, that the fractal dimension of the attractor for weather varies inversely with the sampling frequency.
-- the most controversial site on the Web
Well, again it's a contest for x86-platforms with Windows. Guess I'll keep feeding RC5-64 with 9 Mkeys/sec then with my G4/867 on OS X. After that I'll have S@H and F@H to feed my CPU's hunger.
"Honey, I feel a certain distance between us..." "Really? A 31ms ping ain't that bad..."
I wish I could remember the exact details, but this was the basic idea:
Some branch of the US military was trying to train a neural network to look at a photograph and recognize whether or not there was a tank there.
The people designing the system had pictures of scenes without tanks, and pictures of scenes with tanks. Half of the pictures were sealed away in a safe for later testing. Then, a neural net was trained on the first half of the pictures until it could, with 100% accuracy, correctly identify if there was a tank, or not, in the picture. Finally, the second half of the pictures were presented to the algorithm, and it also correctly identified those pictures as tank/not-tank.
However, when it was tried on another series of pictures, the neural net could only accurately identify about 50% - no better than chance. The engineers who trained the net were dumbfounded, so they went back and started studying exactly what the neural net was trying to use to recognize a tank.
Finally, they found the answer - all the pictures with tanks were taken on an overcast day, and all the pictures without tanks were taken on a sunny day. The million dollar neural net had been trained to differentiate between blue and grey skies! Back to the drawing board...
"I have never let my schooling interfere with my education." - Mark Twain
The technique these people are using is a very simple brute force technique to acsertain the various possible states of the global climate. Essentially, you figure out all of the 'possible' climates by twidling the knobs of your model, and running the thing thousands of times. Presumably one of your tweeks will produce a hindcast (1950-present) that looks like the 'real' earth. This will also, presumably, give you the best forcast (present-2050).
There are other, more graceful techniques used by weather prediction people which involve creating an 'inverse' model. This ammounts to a gigantic least-squares fit of some parameters of your model to the data. Unfortunatly, this would involve finding the inverse of a matrix that is the number of cells in your model times the number of timesteps. For a model like this, this number is very, very large. In order to do this, you need to make some simplifying assumptions, and find approximations to your inverse matrix. You can do this by running forward and backward models itteratively, until an acceptable fit is found.
I'm not sure what the advantages of using the brute force technique are. There are a number of problems developing these inverse (or backward) models, and perhaps they just didn't want to deal with the work. Also, these backward models don't seem to work to well on very non-linear systems, in particular, rapid state changes.
It will be interesting to see if this approach works. If so, I have some runs for some of you with spare cycles...
...that all the additional heat generated by the involved CPU's will add to global warming and invalidate the experiment. Or maybe they took that into account... :-)
Murphy was an optimist.
A serious model is initial conditions + equations.
Takes a lot of compute power, given non-linear feedback, etc.
Chaos ('butterfly effect') says you have to have initial conditions to extremely high accuracy.
We don't have that information, and we don't have the equations relating chemistry, absorption/reflection of atmosphere at different wavelengths, interaction with surface conditions (water, land, rain, surface evaporation),
Perhaps we can learn about the models. But we won't learn anything about climate.
Lew
"The Constitution, the WHOLE Constitution, and nothing but the CONSTITUTION."
it allways seems strange to me that thies didtrubited computing things are colse code. i mean if you whant more ppl to use it y not let them port it insted of haveing to wait around for the slow ppl in charge to do it
maja
Now - I'm going to be an AC troll here - but what you are describing here is 'Cross-Validation', not 'bootstrapping', no? ('Bootstrapping' is when you fake more data by resampling the data you have available in order to fulfill conditions for the calculation of asymptotic statistics..)
A site for nerds should not be furthering this kind of junk science. There is no more basis for predicting climate than the stock market. In fact, I'd say climate has a greater number of more complicated and less known variables. Curve fitting an equation to the climate of the last 50 years is a trivial operation. Attempts to use the fitted curve to predict the future are, however, as worthless as the same attempts have been on the stock market. No one has succeeded in predicting either short or long term stock market trends with any reliability. If they had, the market would be broke.
Or you could just look up what the weather will be like on a given date in 2050 in the Old Farmer's Almanac. They seem to have a better handle on things than the million bucks worth of equipment the local weather guy has. All you hafta do is remember that peepers peep through ice thrice.
Given the arithmatic errata most desktop processors have and the cross-platform nature of distributed computing, I'm wondering how anyone can possibly hope to gain accurate results - especially if there's any floating point math involved.
And with this specific project - isn't the earth's climate largely dependent on the amount of solar output, and isn't that amount relatively variable? How are they gonna know the slight variations in solar output over the next 50 years?
These are my friends, See how they glisten. See this one shine, how he smiles in the light.
All the "science" aside - the last page update on the site is claimed to be 6/27/2001 -- the last (only) item in What's New that is actually dated is 12-10-1999, with a claim that the software is coming soon. All of the articles mentioned on the site were written in 1999-2000. My question is "Is anyone actually still working on this project/site?"
Yes. I was gonna describe bootstrapping, and then decided to start by explaining cross validation, and then I cut out my explanation of bootstrapping because it was too complicated. I should've fixed my original sbuject (and some other things) but I didn't. Sorry, and good job calling me on it.
The good and new comes from no quarter where it is looked for, and is always something different from what is expected.
choas is a relatively complex behavior which is strictly governed by a mathematical algorithm, but, is nonetheless unpredictable due to sensitivity to initial conditions.
I'll give you that, but in weather it still doesn't matter. Given the uncertainty priciple it's impossible to know the inital condtion for a system such as the weather. So even if they have the right mathematically model (which I doubt) this is still all futile.
As the disclaimer in all mutual fund ads goes, "Past performance does not guarantee future results."
Bootstraping makes the assumption, valid in your biological examples, that the thing you're modeling doesn't change over time. Protein folding certainly doesn't over the time frame form which we have data on protein folding.
I doubt that's true of climate and I'm sure it isn't true of the stock market.
++PLS
That may have been the most intelligent post I've ever read on Slashdot (other than my own posts. :)
And the "Weather != Climate" post is running a strong second place, as far as I can remember.
I love this thread!
I feel amazingly enlightened.