Slashdot Mirror
Temperature Data Wants To Be Free
An anonymous reader writes "The UK's Met Office Hadley Centre and University of East Anglia have been refusing access to the data used for their global climate averages and scientific studies. A copy of the data has leaked, and attempts continue to accomplish the release of the data by whoever maintains it. Excuses have included confidentiality agreements which cannot be verified because no records were kept, mention of the source has been removed from the Met Office web site, and IPCC records were destroyed."
There are lots of data sources which are perfectly reasonable to use. NOAA's data being probably the best and most comprehensive.
Yes, the UK is turning into a strange parody of itself with its attempts to close the government to the public on the one hand and monitor citizens very closely on the other. But it's not the only game in town. Despite my own country's recent 8 year slump towards the same type of fascist state as Britain, the US scientific community is still one of the best and most open in the world.
So come and get your data from us, ya'll.
Data from a particular NERC (UK Research Council) project I'm involved with are allowed to be kept by the researchers for a certain amount of time (18 months, maybe?) but then have to be released to the BADC: http://badc.nerc.ac.uk/home/index.html - this gives the researchers time to do some analysis and get some papers out on all the hard work they've done, but obliges them to release the data to the community.
Some of the BADC data sets are restricted to non-commercial use only, so you need to flash your 'Academic Investigator' magic card at them to get it. These guys keep good metadata and license agreements and all that stuff. There's even some datasets from CRU, unrestricted (registration required).
Because it's not science if you hide your data and methods and only release them to people you like. This isn't about misrepresentation or whatever dirt you want to throw - this is about access and transparency. Anything less is not science. It's as simple as that.
A key ingredient of the scientific method is exposing your methods to the cold hard light of day and making sure they withstand scrutiny. If you only show it to people you like, that is hardly serious scrutiny. It needs to stand up to scrutiny from people you don't like.
When you "BELIEVE" science you're just another religion.
In fact, open source science is the BEST and ONLY WAY to avoid science from becoming the new religion as it has, for example, in the climate debates.
The scientific method is the tool for vetting the works of science and if the work of science is closed and secret and kept close to the scientists chests by refusals to share their data, methods, source codes, procedures, etc... then their work can't be verified and might as well be works of fiction just like those of any religious cleric or priest or nutter.
If you can't take others vetting your scientific work then maybe you don't belong in science?
Open Source Science raises the bar and will in the long run improve the quality of the science that is done. Some progress is being made, much more needs to be done.
Let's see, we're supposed to spend literally trillions of dollars to fix global warming, yet we can't see the raw data the hysteria is based on?
WTF!?!?!
Along the same lines, when is the source code used for the climate models going to be published and thoroughly reviewed?
If AGW is in fact true, it can withstand the scrutiny.
The troll from ClimateAudit not being able to get free access to the source data isn't the same as there not being data. No one will give him data unless they have to because he dishonestly misrepresents it. And why the hell the UK government should spend ANY UK taxpayers money to even consider his request is beyond me. He's not British.
I would also note that the owner of one of the two sites mentioned in your story (Anthony Watts) has just employed the DMCA to have a video criticizing his Surface Stations project pulled from Youtube.
What's missing here are some additional context facts; recognizing that the data are not UK data per se. Data from many countries has been collected and collated at the CRU (Univ. of East Anglia) and which feeds into some of the UK Met Office work. Some of this data were collected under the arrangement that the source data were not to be made public because of commercial or other interests. Outside of the USA this is quite common -- that national meteorological services (tasked with maintaining a national observing system and archive) treat their data as a commercial product -- and so they will not release it to just anyone. The fact that I and others think this is wrong and inhibits science is not the issue, the reality is that many countries are not willing to freely release their data. So the CRU and Met Office are between the rock and a hard place; publicize the data and risk ruining their relationships with the data sources, or hold onto the data so that they can keep the data stream flowing and be able to produce the valuable derivative products.
It is that submitter, or Slashdot itself, linked to it through nyud.net. Apparently the site doesn't allow that. Just take that out of the URL, it works fine. The site in question is run by Steve McIntyre. While certianly not a disinterested party (then again people who are involved in something are rarely disinterested) he does have some credibility. He was one of two people who worked on the whole "hockey stick controversy" in terms of showing that the model used to generate the graph was flawed (the model generated a similar shape graph with random inputs).
You're completely wrong. We have excellent data on global climate back about 850ky, good data back to 60mya, and some data back as far as bya. There is something unusual about the climate today.
The idea that there is a stable weather utopia circa 1988 is CRAZY, and you're the one bringing it up as a straw man. Current models account for solar cycles (Milankovitch and others)--- currently, the sun is currently as a period of low output, actually, based on sunspot activity. These are well understood cycles. In spite of that, we have an overall trend of global warming. When you try to account for that data, the best fit to that data is easily the increased CO2 in the atmosphere.
The thing I cannot understand is this. We have a bunch of scientists, lots of them. Starting with Michael Mann in front of Wegman, but including Jones, Thompson, lots of really well known and respected people. They have all done work which supposedly proves that the human race on Earth is facing catastrophe. They supposedly have decisive evidence for this, in the form of data and code.
We then have a lot of sceptics who allege that the data does not exist, is not as described, and the code used to process it does not do what it is said to do, and that there is no such threat as described, or at leas that there is no evidence for one.
You would expect the scientists to immediately produce their evidence and their code and to silence debate once and for all. It would be so simple, it would just be end of story, and now lets focus on what to do about it all. But they do not. Instead they refuse to reveal anything. Jones, for instance, refused to even reveal the names of the stations in China on which his study was based. Mann would not reveal the algorithm which generated the hockey stick to a Congressional Committee. Thompson is silent. Yet supposedly this secret evidence proves decisively, contrary to the claims of sceptics, that the future of the human race is under severe and imminent threat?
It makes absolutely no sense. They never give any reasons for refusing that make any sense either. Sometimes it is commercial considerations. What commercial considerations can there be that outweigh the possible extinction of humanity? Sometimes it is, as Jones once is reported to have said, that they do not want people trying to poke holes in it. WTF??? Sometimes, as with Thompson's ice core data, there is just silence.
It is very hard to believe that this wonderful evidence really exists, and really is as represented. Or maybe it is, and they really do not want to convince everyone of the threat? I don't know, but the story as told makes absolutely no sense. Something is not right here.
I'll quote Feynman, since he put it really well:
-----
It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty--a kind of leaning over backwards. For example, if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated.
Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can--if you know anything at all wrong, or possibly wrong--to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.
-----
Remember: In science, we don't prove things true, we show them to be not false. Same thing? Not hardly. For a complete discussion on the topic, read the Logic of Scientific Discovery by Karl Popper. However what it comes down to is you do not do a test, and then prove a theory true. That can't be done. What you do is come up with a way to falsify your theory, that is to say you come up with a test that says "If things don't come out this way, we know this theory is wrong." You run the test, things come out that way. You have failed to falsify the theory, and we are now more certain it is true. The more than is done, the more certain we are a theory is correct. Each time we attempt to falsify the theory and fail, we are more sure it must be the truth.
If we do then falsify it, the theory has to be redone. That doesn't mean you toss the whole thing out, it may just mean some refinement is needed. For example you have a theory that predicts when X happens Y will results. In 400 tests, this is the case, however 3 new tests show it isn't. What you discover is that in all those tests, A was also present. You the refine your theory: Y will result from X, except in cases where A is present. Your theory is now a little more narrow in application, and fits with the evidence. Perhaps later you find out what A does, and incorporate that in to a more general theory.
The point of all this is that real science is all about trying to prove your theory wrong. You do everything you can to prove it wrong, then have other people do what they can to prove it wrong. When all of you fail at doing that, when the theory has been refined such that it fits all the evidence and you can't figure out how else to test it, then it is most likely the truth. THAT is what scientific rigor is about. It isn't about coming up with a theory, ignoring data you don't like, showing it to a few people who agree with you, and saying "Ok, we proved this true and nobody else can look at it."
We're also in an incredibly cool climate today; During those periods of high CO2 you mention, the temperature has been about 15 degrees C warmer worldwide. The Earth seems to have two stable temperatures; about 10C, and about 25C. When scientists are talking about feedback loops, they're talking about the transition to a global hot-house.
Take a look here:
http://scotese.com/climate.htm
As you can see, our current climate is unusual. Global temperature was similar during the Precambrian (before any animals), at the end Ordovician and the end Carboniferous. As you can see, the global temperature stayed where in these cold zones for a relatively small time.
So yeah, the trees will be fine if we ramped global temperature 15C, but the point is that it wouldn't be great for human civilization.
Your note indicates you understand research has to be replicated to be science. This is very good step in the right direction. Do you also understand that you cannot just turn the data over to those who agree with you? People who disagree with your conclusions have to have access to the data, otherwise it is just pseudoscience.