Slashdot Mirror
Temperature Data Wants To Be Free
An anonymous reader writes "The UK's Met Office Hadley Centre and University of East Anglia have been refusing access to the data used for their global climate averages and scientific studies. A copy of the data has leaked, and attempts continue to accomplish the release of the data by whoever maintains it. Excuses have included confidentiality agreements which cannot be verified because no records were kept, mention of the source has been removed from the Met Office web site, and IPCC records were destroyed."
There are lots of data sources which are perfectly reasonable to use. NOAA's data being probably the best and most comprehensive.
Yes, the UK is turning into a strange parody of itself with its attempts to close the government to the public on the one hand and monitor citizens very closely on the other. But it's not the only game in town. Despite my own country's recent 8 year slump towards the same type of fascist state as Britain, the US scientific community is still one of the best and most open in the world.
So come and get your data from us, ya'll.
This may be a very important story, but it references as evidence two websites which are used by conspiracy nuts, one of which appears to be broken - not /.ed, just broken - and no independent confirmation of the claims. Can anybody give any links to any mainstream news or science sites which are reporting this?
From scarped cliff or quarried stone she cries "A thousand types are gone, I care for nothing, no not one."
Because it's not science if you hide your data and methods and only release them to people you like. This isn't about misrepresentation or whatever dirt you want to throw - this is about access and transparency. Anything less is not science. It's as simple as that.
A key ingredient of the scientific method is exposing your methods to the cold hard light of day and making sure they withstand scrutiny. If you only show it to people you like, that is hardly serious scrutiny. It needs to stand up to scrutiny from people you don't like.
When you "BELIEVE" science you're just another religion.
In fact, open source science is the BEST and ONLY WAY to avoid science from becoming the new religion as it has, for example, in the climate debates.
The scientific method is the tool for vetting the works of science and if the work of science is closed and secret and kept close to the scientists chests by refusals to share their data, methods, source codes, procedures, etc... then their work can't be verified and might as well be works of fiction just like those of any religious cleric or priest or nutter.
If you can't take others vetting your scientific work then maybe you don't belong in science?
Open Source Science raises the bar and will in the long run improve the quality of the science that is done. Some progress is being made, much more needs to be done.
Let's see, we're supposed to spend literally trillions of dollars to fix global warming, yet we can't see the raw data the hysteria is based on?
WTF!?!?!
Along the same lines, when is the source code used for the climate models going to be published and thoroughly reviewed?
If AGW is in fact true, it can withstand the scrutiny.
The important question which I've never seen the math for is how much CO2 is output by random natural events during a certain time period versus how much we output currently.
We are taking a few hundred million years worth of biomass and burning it up in a about a hundred and fifty. Perhaps this has no effect on the environment, but I think it's prudent to make sure that we don't send the climate into a self-feedback loop that destroys our way of life. It's not as if riding around in traffic or having an iPod is worth giving up food and water.
It is always a real red flag when data is withheld. The core of science is that "ideas are tested by experiment." Ok well that means that, for science to work, others have to be able to check your work. You have an idea and say "Here's my idea and here's my support." Ok well your support needs to include ALL your data, your methods and so on. Why? So that others can check your work. Only then, after they've repeated and independently verified your results, can we start to feel confident your idea might be correct. To me, hiding data says one of three things is going on:
1) You are dealing with something commercial, that is being held secret so you can market it. Ok well that shouldn't be the case here.
2) The data in fact does NOT support your conclusion, however you don't want to admit you are wrong and thus are trying to suppress it. Perhaps you are worried you'll lose grants.
3) You suck at the science. You think that science is a process where you, the scientist make a claim and the rest of the world just has to listen to you.
4) You are a charlatan, a con man, and you are trying to convince people of something that isn't real, you are trying to sell them snake oil as it were.
I just can't see any legit reason in a pure scientific study why all the data wouldn't be made available for all to see. That it isn't really sets off warning bells in my head. I've read papers like this in the behavioral sciences and always what I see happening is that their experiment was basically a bust, it falsified their hypothesis, or simply produced inconsistent results. However they don't want to admit it, so they find a way to tweak the numbers and then refuse to release full methodology and results.
So this worries me. If climate change is truly a threat to humans, then it should be in the interests of everyone that all the data is made available, unedited, unhindered, so that the theories can be checked and rechecked. Science should be allowed to proceed with as little barriers as possible so that it can proceed as rapidly as possible because the matter is of such importance.
What you describe as real is climate change on a short timescale - such fluctuations aren't extraordinary and the claims of "climate change" are for the most part suggesting a mostly permanent change in climate, brought about by man-made influence.
Even the changes you describe are hard to judge and have varied greatly just year to year - here in Ireland this year everything (plants/animals) was more a stereotypical Spring/Summer - albeit extra plant growth, insects and birds because the sun/rain in Spring were in just the right order for optimal conditions (one particular week of heavy rain, one particular week of strong sunshine, and a lot of other "nice" conditions besides).
I'm a skeptic in the true sense - I'm skeptical about the climate change hysteria, but not convinced either that there is no merit in the "man-made permanent climate change" argument, and certainly I think it's a good idea to cut back on pollution (although the exclusive focus on carbon/CO2 may need more justification). I don't think we have enough to go on either way and some policies seem very knee jerk and may be counter productive. Plus most policies that are happening as opposed to mere proposals are often due to other interests (ways to make money from it, keep certain section of voters happy, skew competition, raise tax, etc.)
Here in Ireland there is as much talk as anywhere else about carbon taxes etc. yet there is still next to no enforcement of building standards for example to ensure new houses are properly insulated, pathetic planning that nevermind about transport emissions - makes equal (or even poor) delivery of services across the country very expensive. Too sparse population in rural areas for all kinds of services never mind private car use problems - too unplanned and fast-increasing population in the capital for services needed for such an amount of people - traffic problems and not enough money for public transport due to cost of supporting rural area. Our poor planning also means developers are allowed to put up crummy buildings that last as little as 10 years before being redeveloped - regardless of climate change or CO2 or anything else it's obvious that such things are grossly wasteful.
All in all, I'd like to see common-sense policies while we continue to research the "big picture" rather than random ideologically-driven hypotheses being put into action where politically convenient.
-- *~()____) This message will self-destruct in 5 seconds...
No, basically nothing in your post is "true" in any scientific version of that word :) We do not have excellent data (gas diffusion in ice cores is a bitch!) and current models lack incredible amounts of algorithmic data which is instead made up as we see politically fit at the moment :) (for example, the influence of clouds)
We do not have an unusual trend of global warming at all. On the contrary, there might not be a trend to speak of when removing measurement uncertainties. (http://surfacestations.org should scare anyone who believes the tempereature data we're soon basing our whole economy on)
The best fit for the temperature changes over the last century is not with CO2 levels but with ocean cycles btw.
On gas diffusion: http://www.ingentaconnect.com/content/igsoc/jog/2008/00000054/00000187/art00012
On ocean cycles: http://atmoz.org/blog/2008/05/14/timescale-of-the-pdo-nao-and-amo/
Why are you not interested in doing actual science? We simply don't have data to support Hansen's and Gore's wild accusations.
it's in my head
Denalists? So basically when you don't like someone's opinion, you make up a new, derogatory term to try and marginalize them? That isn't science, that is marketing. In particular, it is the kind of marketing con men do. When people question their products/methods, they shout down the critics, they deride them, they call them names. They basically try to make it look like you must be retarded if you don't agree with them.
You are also pulling another con man trick: The appeal to authority. That a site is run by "climate scientists" or is not, doesn't matter. Science isn't about who has the authority in a certain area, it is a process for finding out about the world. So trying to say "Well this site is run by climate scientists, this one isn't," doesn't strengthen your argument. That is along the same likes of "4 out of 5 dentists agree!" Ok well so what? Maybe 4 out of 5 dentists are mediocre, and the excellent 20% realize that it doesn't matter?
There is also the matter of what is a climate scientist? This isn't a degree listed at most universities, and didn't exist at all until recently. If you look at the people who run realclimate you find their PhDs are Applied Mathematics, Geology, Oceanography, and such. None of them have a degree in "climate science." So what a climate scientist is, is simply someone who studies the climate. Ok, fair enough, however that does mean it isn't an exclusive club that only certain people can be members of. For that matter, Watts is a meteorologist, which is also on the topic of climate studies.
None of that means a given person is right or wrong, but it is incorrect to appeal to authority and try and claim that "Oh realclimate is run by climate scientists so they are the only place you can trust." No, that's not the case. Science doesn't work like that.
When you pull shit like this, it really doesn't help your case. If you disagree with the theory someone is putting forth, or their criticisms of a theory, deal with that. Don't play salesman/con man tricks. To me, it makes it look as though you've something to hide.
Some of the BADC data sets are restricted to non-commercial use only, so you need to flash your 'Academic Investigator' magic card at them to get it. These guys keep good metadata and license agreements and all that stuff. There's even some datasets from CRU, unrestricted (registration required).
Where does that leave the hobbyist researchers then?
Today most household computers are potent enough to be able to sift through amounts of data that we only could dream about a few years ago.
Don't forget that the collection of the raw data has been done through the money of the tax payers. It is of course possible to have a reasonable fee for obtaining a copy in some cases, but it may as well be put on the web these days.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
I'll quote Feynman, since he put it really well:
-----
It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated.
Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can--if you know anything at all wrong, or possibly wrong--to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.
-----
Remember: In science, we don't prove things true, we show them to be not false. Same thing? Not hardly. For a complete discussion on the topic, read the Logic of Scientific Discovery by Karl Popper. However what it comes down to is you do not do a test, and then prove a theory true. That can't be done. What you do is come up with a way to falsify your theory, that is to say you come up with a test that says "If things don't come out this way, we know this theory is wrong." You run the test, things come out that way. You have failed to falsify the theory, and we are now more certain it is true. The more than is done, the more certain we are a theory is correct. Each time we attempt to falsify the theory and fail, we are more sure it must be the truth.
If we do then falsify it, the theory has to be redone. That doesn't mean you toss the whole thing out, it may just mean some refinement is needed. For example you have a theory that predicts when X happens Y will results. In 400 tests, this is the case, however 3 new tests show it isn't. What you discover is that in all those tests, A was also present. You the refine your theory: Y will result from X, except in cases where A is present. Your theory is now a little more narrow in application, and fits with the evidence. Perhaps later you find out what A does, and incorporate that in to a more general theory.
The point of all this is that real science is all about trying to prove your theory wrong. You do everything you can to prove it wrong, then have other people do what they can to prove it wrong. When all of you fail at doing that, when the theory has been refined such that it fits all the evidence and you can't figure out how else to test it, then it is most likely the truth. THAT is what scientific rigor is about. It isn't about coming up with a theory, ignoring data you don't like, showing it to a few people who agree with you, and saying "Ok, we proved this true and nobody else can look at it."
The "bunch of scientists" you should be paying attention to aren't the half-dozen public figures engaging the quacks, but the ten thousand quietly publishing the research which led to the concensus in the first place. The handful of public scientists who can't whip out smoking-gun data like characters in a Roland Emmerich movie aren't the people who hold the actual science.
Your note indicates you understand research has to be replicated to be science. This is very good step in the right direction. Do you also understand that you cannot just turn the data over to those who agree with you? People who disagree with your conclusions have to have access to the data, otherwise it is just pseudoscience.
"Where does that leave the hobbyist researchers then?"
You have to get out of the basement, go outside and talk to people. Same as all the professional researchers. If you show up at the appropriate guy's office at the nearest university, tell him you want to collaborate and do some research, for free, he'll likely be happy to get you your data in exchange for some input and his name on the paper.
Same with scientific journals. Free access equal to the level of any academic is as far away as the nearest university library. A little slower access is probably closer - the nearest local library.