Open v. Closed Source-Climate Change Research

theidocles writes "The ongoing debate over the 'hockey stick' climate graph has an interesting side note. McKitrick & McIntyre (M&M), the critics, have published their complete source code and it's written using the well-known R statistics package (covered by the GPL). Mann, Bradley & Hughes, the defenders, described their algorithm but have only released part of their source code, and refuse to divulge the rest, which really makes it look like they have some errors/omissions to hide (they did publish the data they used). There's an issue of open source vs closed source as well as how much publicly-funded researchers should be required to disclose - should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?"

Comments

How much is enough?

[mfh]

how much publicly-funded researchers should be required to disclose

All of it, baby. We're paying for it -- we should have the right to:

a) Know what you're spending our money on
b) Have the right to make it better ourselves
c) Learn of security flaws early so we can correct them

Especially when there is some doubt about the nature of the results in the closed source model from Mann et al.

It should be called argh!

[Anonymous Coward]

I hate projects with names like R. I used R a while back, and it's a great program, but try searching for "R" plugins on Google. Not fun.

No brainer...

[wileynet]

Science, like government, should be transparent. The public should be able to see and evaluate every part. Any science, or government, that hides it's implementation is inherently suspect to corruption.
Closed science is half a step from religion. You are expected to have faith in the researcher's methodologies, analysis, assumptions, and motives. Sorry, but good science does not rely on faith.

Arguments for & against open-source

[G4from128k]

Arguments for open-source science:

1. Replication: Science thrives on replicable results. If researchers don't publish everything, others cannot replicate the results and the original findings become suspect.
2. Knowledge Sharing: If the point of publicly funded academic research is to advance human understand of the world, then open access to methods (code) is a key part of that.
3. Reduce/Eliminate Stealth Patents: Releasing knowledge into the public domain will help nip patents in the bud.
4. Preserve Fair Use: University's trends toward turning research into money is threatening the basis of fair use for researchers. How long will it take intellectual property owners to get regulations on "fair use" because academic research has turned into another big, for-profit, corporate enterprise.

Arguments against open-source science:

1. Replication: Science thrives on replicable results. But the key is independent verification. If one scientist simply reuses another scientist's code, there is a chance (high, some would argue) that faults in that original code would corrupt both scientists' results. Closed-source forces independent verification.
2. Commercialization: If the point of publicly funded academic research is to create widely-used products and services, then the system needs some scheme for protecting the value of intellectual assets. Where the cost of bringing the product to market is very high (e.g., pharma), the company/investors needs some assurance that another company can't just copy the results when the product comes out.

I'm sure there are arguments on both sides.

Re:Arguments for & against open-source

[goodmanj]

To frame replication of scientific results as an "open source" debate is both a no-brainer and misleading. A no-brainer because if an investigator does not provide enough information to allow their colleagues to replicate their work, they are not doing science -- in that sense, all science is "open source". Misleading because scientific ethics do not require totally open sharing of source code: it is sufficient to verbally describe the algorithms and data used in enough detail that someone else can repeat the experiment. In practice, journal article page limits often require that this description happens on a person-to-person level, rather than in published literature.

Most of the "arguments against open-source science" mentioned here are not about science at all. The secrecy surrounding commericial and national-security "science" is good only in a financial or political sense: they do not help science, per se, at all. And personality conflicts are a factor as well: I suspect that Mann et al's reluctance to release source stems from an extreme personal frustration at McKitrick et al's persistent and (in my view) not always well-supported attacks.

Voodoo, not science

[climb_no_fear]

I'm of the opinion that anything that gets published should be published in its entirety, at least at some point. For example, people who publish protein structures can put the coordinates "on hold" for up to 18 months.

And to say because the research is done with "taxpayer's money" is missing the point: If you can't reproduce every step, it's voodoo, not science. And we make policy decisions based on science, not voodoo (I hope).

real government research goals

[MyRuger]

Obviously you have never written an SBIR or BAA. You when you do research "At the tax payers expense", you need to show your plans to commercialize the results of the research. The government wants you to create a IP twoards a commerial project which will spur the economy, not to contribute to the scientific community as a whole. Take it as you will, but I think that most research would not get funded if your commertilization plan was to release it on sourceforge.

Re:But the Hockey Stick is True!

[BenBenBen]

That Global Warming is a manmade, real phenomona is accepted by 99.9% of scientists in the fields involved. To trot out the "only a theory", "some experts dispute" etc routine is like getting the Flat Earth Society involved every time someone talks about circumnavigation. "Heads in the sand" is going to be on our culture's gravestone when the next lot of intelligent life evolves here and starts wondering why parts of Nevada are 10,000 times the normal radiation level.

with open source, everyone can see you're dumb

[Nick Barnes]

See this debunking of McKitrick's work, showing, among other things, how he:

• denies that average temperature is meaningful,
• confuses degrees with radians,
• invents a whole new temperature scale,
• replaces missing data with zeroes

Covert Perpetuation

[Doc Ruby]

Where's that wealth of information about the secret US wars in Central America in the 1980s? Or in Angola in the 1970s? Or in Chile in the 1970s? Or in Cambodia and Laos in the 1960s? Iran in the 1950s? These secret wars are secret largely for *political* purposes - the military secrecy benefits evaporate within months. But the political purposes - covering liability for abuse, war crimes, and just plain lying about the causes, effects, and benefits of the war - those last forever.