Open v. Closed Source-Climate Change Research

theidocles writes "The ongoing debate over the 'hockey stick' climate graph has an interesting side note. McKitrick & McIntyre (M&M), the critics, have published their complete source code and it's written using the well-known R statistics package (covered by the GPL). Mann, Bradley & Hughes, the defenders, described their algorithm but have only released part of their source code, and refuse to divulge the rest, which really makes it look like they have some errors/omissions to hide (they did publish the data they used). There's an issue of open source vs closed source as well as how much publicly-funded researchers should be required to disclose - should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?"

Comments

Arguments for & against open-source

[G4from128k]

Arguments for open-source science:

1. Replication: Science thrives on replicable results. If researchers don't publish everything, others cannot replicate the results and the original findings become suspect.
2. Knowledge Sharing: If the point of publicly funded academic research is to advance human understand of the world, then open access to methods (code) is a key part of that.
3. Reduce/Eliminate Stealth Patents: Releasing knowledge into the public domain will help nip patents in the bud.
4. Preserve Fair Use: University's trends toward turning research into money is threatening the basis of fair use for researchers. How long will it take intellectual property owners to get regulations on "fair use" because academic research has turned into another big, for-profit, corporate enterprise.

Arguments against open-source science:

1. Replication: Science thrives on replicable results. But the key is independent verification. If one scientist simply reuses another scientist's code, there is a chance (high, some would argue) that faults in that original code would corrupt both scientists' results. Closed-source forces independent verification.
2. Commercialization: If the point of publicly funded academic research is to create widely-used products and services, then the system needs some scheme for protecting the value of intellectual assets. Where the cost of bringing the product to market is very high (e.g., pharma), the company/investors needs some assurance that another company can't just copy the results when the product comes out.

I'm sure there are arguments on both sides.

Re:How much is enough?

[R.Caley]

We're paying for it

Of course, often you will only be paying for part of it. It is common for research to come out of a combination of `projects' funded from different sources.

What should happen if, for instance, a drug company funds a project into developing statistical theory and signal analysis and so on to improve analysis of early candidate drug screening data, and then the researchers use the prototype implementation in a publicly funded project they are involved in on climate data and find something significant?

Or what happens for part-state, part-commercially funded projects?

I think one thing which could be done is to give companies a (bigger) tax break on money put into research (internal or when they give grants) if they sign up to give out not only all the data, but things like source of programs and detailed design of prototypes and experimental setups.

Another thing would be to set up some kind of peer review process and then treat published source as a publication for the researcher. If your peers sign off to say that you have produced and documented the code to the level where it is a useful resource for other researchers, then it should count towards departmental and personal evaluations just as a journal article would. The formalised review process is important -- the average bit of lashed-together-to-get-the-data research code is more equivalent to a scribbled note on a whiteboard than to a journal article.

Perhaps all that is needed is an online journal set up and run as a properly organised accademic journal, but specialising in publishing code. Imagine an infrastructure not a million miles away from sourceforge, but with a peer review process to decide what gets counted as a release.