UK University Researchers Must Make Data Available

Sara Chan writes "In a landmark ruling, the UK's Information Commissioner's Office has decided that researchers at a university must make all their data available to the public. The decision follows from a three-year battle by mathematician Douglas J. Keenan, who wants the data to do his own analysis on it. The university researchers have had the data for many years, and have published several papers using the data, but had refused to make the data available. The data in this case pertains to global warming, but the decision is believed to apply to any field: scientists at universities, which are all public in the UK, can now not claim data from publicly-funded research as their private property." There's more at the BBC, at Nature Climate Feedback, and at Keenan's site.

Comments

Sudden Outbreak of Common Sense

[nacturation]

The public pays for gathering the data, the public should have access to that data. Kinda hard to find fault with that.

Re:Sudden Outbreak of Common Sense

[NeutronCowboy]

Creationists regularly mangle papers, taking quotes out of context and all.

Get ready for an onslaught of mangled data analysis, with data being taken out of context, the results published to some blog, and people making policy decision based on those blog postings.

the media will focus on the new controversies this will spawn

That's a guarantee. While in theory, I welcome this development, I suspect that in practice it will lead to more chaos than before. Not because the data is shoddy, but because some meteorologist will think that running a data set through an excel curve fitting algorithm is science.

Re:Sudden Outbreak of Common Sense

[jacksonj04]

You know the multi-billion dollar LHC? Guess what they did their first physics on. Not finding new exotic particles, but proving that what we think we know so far still stands up. Duplicating data is exactly how things get proven and disproven. If Group A and Group B use exactly the same source data there's no possibility of Group B proving Group A's research wrong.

Re:Sudden Outbreak of Common Sense

[Anonymous Coward]

some meteorologist will think that running a data set through an excel curve fitting algorithm is science.

Nope -- it's only science if you adjust and filter the data first to make it match your truth. Resist releasing your data though, others may adjust and filter it other ways to make it match their truth. All science in the world of research driven by political agendas and egotistical arrogance.

Disclose, when in doubt disclose more. Anything less in scientific arenas where others can't repeat your experiments is just a symptom of fear, insecurity, and lack of confidence that your conclusions will stand up to the view and study of many brains (some better than yours, some worse).

Same argument for why FOSS is better - many eyes reviewing (in theory) and rapid fixes.

Re:Sudden Outbreak of Common Sense

[thepike]

I totally agree. If people just start looking at each others data instead of verifying it, a lot of mistakes (or fraudulent data) will never be caught.

Also, I have to wonder what the timeline for releasing data is. My research is funded with government money (NIH and NSF) but it can take years to get enough data to make a worthwhile paper. If I have to release my data before then it will hurt my ability to publish papers without getting scooped. You could end up with a whole closet industry of people just data mining the data others have had to disclose. And, here's the main catch, if you don't have to release results you haven't yet reported on, the problem isn't solved at all because I could just choose to "not yet publish" any results that don't agree with what I want to say. Nothing says I ever have to publish results I get, so why wouldn't I just sit on them?

Not that sitting on data just because it doesn't agree is a good thing, but it happens. And plenty of good data goes unpublished (experiments fail, uninteresting results happen, journals don't publish negative results very often etc) so what about that data? Overall this law isn't going to help anything, and will just cause issues.

Re:Sudden Outbreak of Common Sense

[Obfuscant]

If Group A and Group B use exactly the same source data there's no possibility of Group B proving Group A's research wrong.

Wrong. If Group B cannot duplicate Group A's analysis of the data, that proves that Group A did something wrong and probably came to the wrong conclusion.

If Group B cannot duplicate the experiment and get the same data (and knowing that means being able to compare both sets) that calls the experiment as a whole into question.

There is more to science than simply applying equation A to data B and getting number C.

This hubbub all came about because of the difficulty in prying the source data out of the hands of the guy who produced the "hockey stick" figures. It's covered in the book "Broken Consensus" I think it's called. The "hockey stick" is not the "source data", the source data is all of the individual readings from all the instruments, prior to corrections for sampling errors or known issues. One cannot verify the quality of the "hockey stick" result without having the source data and being able to verify the processing steps that were done to it.

The downside to free and open access to all data is that research groups get grants to collect AND process the data to come up with results. Opening the data up for free access means that other groups, who have more interest in scooping than being right, have more ability to do that scooping. That leaves the people who did the work in the cold. There is good reason to delay opening the data until the group being paid to collect it has a chance to use it.

Re:Sudden Outbreak of Common Sense

[michaelwv]

Absolutely. The public should have access to the data. Public grants then also need to pay for curating the data. Libraries aren't free, archives aren't free, package data in an actually useful form takes precious time, which is scientists most precious resource. Having data in a form that is useful to the 25 people in your research group is very different than providing data that can be used by thousands of people. It's analogous to the difference between the quick bash script you have that backs up your movies to your external hard drive, and having something that you're willing to distribute to 1000 people and provide support.

Re:Sudden Outbreak of Common Sense

[finarfinjge]

I'll probably get flamebait or troll for this too, but this has always been the danger of the over-advocacy of climate change. Climate science is not even close to "settled". Nor is evolution, nor is physics. Well established and able to make verifiable predictions yes. Settled. No. The direct result of making the absurd claim that some cutting edge field of science is settled is this. Some complete moron then says "see, global warming wasn't settled, so evolution is bunk too" I've seen similar idiotic comments about plate tectonics as well. A number of years ago (far enough back it hasn't been cached), I wrote here that as scientists, we had better be right about climate change. Now we reap what we have sown. If it annoys you that idiots make claims like "global warming wasn't settled, so how can you be sure about evolution", look to the strident supporters of the cause. They (I'm talking about realclimate etc., here) are as responsible as Beck. By hammering any and all dissent without any concern as to the validity of the claims, they have made this type of comment inevitable. We will be seeing much more of it and we have only ourselves to blame.

Re:Sudden Outbreak of Common Sense

[the gnat]

my experience of this situation comes from protein crystallography and deposition of the hard won data there

Ah, a fellow crystallographer. Welcome, brother!

I was about to post a similar comment. However, I only agree with you up to a point. Once you publish a paper reporting the structure, all of the raw data should be made publicly available (including diffraction images - although deposition of those isn't quite feasible yet). I would apply the same standard to any other field: you shouldn't publish until you are comfortable releasing the underlying data. I don't care if you're still working on some super-secret follow-up paper, as far as I'm concerned your publication is useless if I can't go to the PDB and download the coordinates. And if you're using public resources to solve your structure (like NIH funding, or one of the DOE's synchrotrons), your results are public property.

There was once intense resistance to even mandating coordinate deposition (long before I got started in the field), which just sounds insane now. Some of the people doing the most complaining were in fact some of the best funded. A decade later, the field went through the same bullshit whining with regard to reflection data. Now most journals require both coordinates and reflections, and not only has the field not suffered in the slightest, many more studies are now possible and the majority of structures can be solved without experimental phasing. If we'd left things the way the naysayers wanted it, every group attempting to study, say, ribosome structure would have to either plead with more senior groups for coordinates in order to solve their structures (and, almost certainly, further bloat the author lists and potentially cede some control over their project - which, I imagine, would have suited the senior faculty just fine), or waste half a decade making heavy metal derivatives. It is difficult to convey to non-crystallographers how huge a waste of time and money - most of it coming from tax dollars - this scenario would be.

Now, where it gets messy is situations where you have to release data ASAP, instead of waiting until publication. American structural genomics groups do this (it may be a requirement of the NIH), but PDB deposition is more of an endpoint in itself for them, and no one is going to bother trying to scoop them on most of those proteins. Genomics centers also do this. A grad school classmate of mine worked on a sequencing project where much of the gruntwork was performed by the DOE, and they had extremely strict release rules. She complained that other groups (of bioinformaticists) could start analyzing the data before she'd had a chance to complete her own studies, because the outsiders didn't have to spend a lot of time thoroughly annotating the genome before publishing. (I don't think it held her back in the end - she graduated with several papers in Science.) In many situations like this, to obtain the data you need to agree to an embargo on publications, to prevent that sort of underhanded behavior. I saw an article retraction recently where the scientific content was undisputed, but the investigators had (unintentionally, it appeared) broken an embargo by submitting the paper when they did.

In general, I think the scientific community - especially the part funded by the public - should err on the side of maximum disclosure of data, and I don't have much sympathy for the researchers in this story (and I'm not particularly sympathetic towards "climate skeptics" either). I do worry that rules will be used to harass researchers in supposedly controversial fields (Richard Lenski's adventures with Conservapedia are a particularly nauseating example), but as a scientist, I also think the benefits of making massive amounts of data available to anyone are far too important to let these risks bother us, and the drawbacks of keeping such data private are much worse than having to fight off the occasional knuckle-dragging lunatic.

Re:Sudden Outbreak of Common Sense

[nacturation]

Sure, I'll give you the data. But I wasn't funded to put the data in a format that's easy to understand. I've also got a job, and I don't get paid to support a competitor's data analysis attempts. Good luck.

Your so-called competitors will be sure to mention your viewpoints when your funding runs out and you apply for more. Not only is your research not easy to understand and you don't let others analyze the data to attempt to reproduce your conclusions, but you think that other members in the scientific community are competitors and you feel a need to sabotage their efforts by making it difficult for them to use taxpayer-funded data to advance science. If science is such a business to you, then how about you fund it all yourself from the profits you make?

Re:Sudden Outbreak of Common Sense

[HungryHobo]

not really.
Your problems with these possible situations are based on the deeply flawed system we have in place now.

Give academics the respect and credit they deserve for collecting vast quantities of high quality data rather than merely for the 2 page paper they write about some interesting statistical anomalies they found in said data and this ceases to be a problem.

The way papers are written, reviewed and published today and the way academics are given credit is based on a system hundreds of years old when it costly to print hundreds of pages of boring figures.

Now data is cheap beyond words. Publishing a few hundred words or a gigabyte is little different when your audience is fairly small and the way academics publish should reflect that but it's too hidebound and dogmatic to do that.

A professor who does nothing but produce a high quality and hard to acquire dataset deserves credit even if he comes to no conclusions at all.

The problem is with the system and with the way academics think.
Not with this possible change.

Fix your system.

Re:Sudden Outbreak of Common Sense

[the gnat]

Give academics the respect and credit they deserve for collecting vast quantities of high quality data rather than merely for the 2 page paper they write about some interesting statistical anomalies they found in said data and this ceases to be a problem.

The problem is that interpreting raw scientific data is enormously time-consuming, because there's so much information available that we can't possibly assimilate it all. I have a PhD in biochemistry and advanced training in crystallography, but I couldn't look at a ribosome structure and easily figure out what it meant, because I don't know very much about ribosomes. The people solving the structure, on the other hand, have exactly the background necessary to perform detailed analyses, and they will undoubtedly notice things that completely escape me. And I think you're understating the value of the scientific literature. A 2 page paper on statistical anomalies won't get you a faculty position at a major university, but a well-written 10 page paper on the meaning of a crystal structure certainly can. This is even more the case if they took additional time to perform non-crystallographic experiments to verify new hypotheses.

I don't deny that there are issues with our system, but you're completely missing the point of writing papers. Simply generating massive amounts of data isn't considered science - figuring out what it means is. I say this as someone who is very good at generating data quickly, but not particularly good at interpreting it. Now I write data analysis software instead, and leave the question-asking to more suitable minds.

Re:Sudden Outbreak of Common Sense

[Vornzog]

I work for a government lab that produces DNA sequences. We are obligated to release our data into a public database as soon as it has been verified for any samples that come from the US, and we release most of our foreign data, too, unless the other country involved gets pissy.

Nothing good comes of that speed. We get crackpots thinking they've made major discoveries (not one real one yet), we get scooped for major papers (think Science), sometimes by our own collaborators using only our data and none of theirs, and we generally spend a lot of time, effort and *more money* on media spin control. There is such a thing as releasing the raw data too fast.

We get a *ton* of FoI requests, too - people think we are withholding the good data, or being stubborn by not providing them composite statistics in exactly the format they want to see. The truth is, up until I got involved, the data management technology was so far behind the current bog-standard capabilities of the rest of the world, we couldn't actually answer the questions that were being asked, barring Herculean effort.

Don't get me wrong, I think we *should* be releasing all of this data - delayed by just a bit. That way the people who generate it would have a better shot to get recognition/credit for their work, the crackpots would have less ammo for their rants, the press would be more likely to get the facts right the first time, and the scientific integrity of the whole process would be upheld, as everyone would get the raw data to review. It'd probably save a ton of money.

The "reward" for doing publicly funded research is that you keep getting funded.

Collecting good data is hard work, and the payoff is big publications, which you need if you want to continue getting funded. Once you've got that big publication in your pocket, though, you'd better by coughing up that data set. Otherwise, everything you say is suspect. Kudos to the UK for getting this half-way right, but they'd better set some reasonable constraints on the timing of these required data releases, or face any number of frivolous lawsuits from conspiracy theorists and 'data analysis specialists' who don't want to do any of the hard work themselves...

I don't care one whit what you think you're entitled to: if you're taking my money, you work for me.

I don't care if you are a ditch digger or a particle physicist. Doing all the hard work and getting none of the credit sucks regardless of what we are discussing or who is paying the bills. So put up or shut up. Would you be willing to do all of the grunt work in your job, but take none of the recognition? Most people wouldn't - those are the kinds of jobs that make people go 'Postal'. If you aren't doing it (and even if you are), do you really expect anyone else to?

Re:Sudden Outbreak of Common Sense

[AJWM]

Examining old data has one value and one value alone - verifying that the claim made for the data matches up with the data. [...] Access to raw data for any other reason is pointless.

Hardly. One could analyse the raw data looking for something other than what the original researchers were looking for. There might even be some interesting signal buried in the data that original team, focusing on something else, disregarded as noise. Minute timing errors in, say, solar wind data returned from a spacecraft might turn up some oddity of orbital mechanics, for example. The researcher focusing on the sensor data rather than the timestamps will miss it, but it's all part of the raw data. How many biologists discarded moldy Petri dishes as ruined, without recording that, before Fleming thought to investigate why bacteria didn't grow near the mold?

Re:Many academes want this too...

[DarkKnightRadick]

no, peer review is good. It helps to point out mistakes or inconsistencies. Getting rid of scientific journals is quasi-good (less profit motive in science, but also less chance to get work out there).

Re:Good and bad

[Monkeedude1212]

On the other hand, this will likely produce a whole stream of deliberately inaccurate analyses with ulterior motives behind them.

But with the data public, it'll be easier to shoot them down for picking, choosing, skewing, and what else.

There is no reason why this kind of data should ever be "secret"

Awful summary

[Protoslo]

It turns out that "the data" are measurements of petrified tree rings, which were collected in the course of (presumably) a government grant-funded study. Now Queen's University researchers must compile the data for release because of the (UK) Freedom of Information Act. The scientists quoted in TFA apparently did not use the ring data for anything relating to climate studies, but Keenan has that purpose in mind.

Phil Willis, a Liberal Democrat MP and chairman of the Science and Technology Select Committee, said that scientists now needed to work on the presumption that if research is publicly funded, the data ought to be made publicly available.

That doesn't seem unreasonable to me. Appendices with raw data are often included already in the online editions of journals. Of course, if the ruling applies to all data generated in the course of a study, whether it is used in publications or not, it could be onerous indeed.

Re:yro my ass

[geekoid]

errr... no always.

Putting data into peoples hands whoa aren't experts often leads to bad things. See every non expert who believed Wakefield study because they didn't understand how to interpret data. In that case kids died , and kids are still dying.

In principle I agree with you, but we live in an are where everyone thinks they are a qualified expert in anything. That simply isn't true, and no good will come out of this.

The data wan't show a flaw in the study because it wasn't used, but he will inevitably cherry pick data to 'prove' the study is wrong. And people like Hannah Devlin are always happy to publish claims without proper study. So no good can come from this, and people need to understand that.

It's hard problem to solve.

Re: data retention now required too?

[sl149q]

If people cannot replicate your results it isn't science.

And with Climate Science part of the process is showing how you collected and interpreted the data. If you are not willing to share the raw data so other researchers can attempt to replicate your methods and results then don't bother publishing.

Re:Peers?

[sl149q]

As opposed to the proselytizers who are funded by the NGO's and the new "Green" capitalists and rent-seekers.

One of the more interesting bits of the Climategate emails showed that Mann was happy to share his data EXCEPT to people who he thought would disagree with his methods and results.

And in this case Mann was also the recipient of the tree ring data showing that again if you agreed with the owners ideas he had no problem getting you copies of what you needed.

NOT ONLY DATA, METHODS WANT TO BE FREE!!

[xtracto]

Simply generating massive amounts of data isn't considered science - figuring out what it means is. I say this as someone who is very good at generating data quickly, but not particularly good at interpreting it.

Spot on. I have a PhD in Comp. Sci. (Multi-Agent Systems / Market Based Control). One of the things you learn (maybe in you Universitity degree courses or in your first paper presentation) is that data does not mean *anything*, what matters is the interpretation of such data.

Nevertheless, I am of the opinion that programs used for the generation / manipulation of such data should also be free / scrutinable. Specially those developped during the research as they are also being paid by the tax payers money.

In the field I am working now (Agent based computational economics) a lot of people do these so called agent-based simulations, then they write a nice paper about what their simulations showed and try to publish it. The problem is that they keep their code! and in that respect they are deffinitely removing a good chunk of the "methods" part of their research. It is absolutely impossible to duplicate that work without the code.