Slashdot Mirror
Open v. Closed Source-Climate Change Research
theidocles writes "The ongoing debate over the 'hockey stick' climate graph has an interesting side note.
McKitrick & McIntyre (M&M), the critics, have published their complete source code and it's written using the well-known R statistics package (covered by the GPL). Mann, Bradley & Hughes, the defenders, described their algorithm but have only released part of their source code, and refuse to divulge the rest, which really makes it look like they have some errors/omissions to hide (they did publish the data they used). There's an issue of open source vs closed source as well as how much publicly-funded researchers should be required to disclose - should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?"
how much publicly-funded researchers should be required to disclose
All of it, baby. We're paying for it -- we should have the right to:
a) Know what you're spending our money on
b) Have the right to make it better ourselves
c) Learn of security flaws early so we can correct them
Especially when there is some doubt about the nature of the results in the closed source model from Mann et al.
The dangers of knowledge trigger emotional distress in human beings.
I hate projects with names like R. I used R a while back, and it's a great program, but try searching for "R" plugins on Google. Not fun.
Science, like government, should be transparent. The public should be able to see and evaluate every part. Any science, or government, that hides it's implementation is inherently suspect to corruption.
Closed science is half a step from religion. You are expected to have faith in the researcher's methodologies, analysis, assumptions, and motives. Sorry, but good science does not rely on faith.
No. By that logic, the taxpayer should be able to see information about how the selection of doctors is made, and which prescriptions should be prescribed, and generally how Medicare money is spent.
Arguments against open-source science:
I'm sure there are arguments on both sides.
Two wrongs don't make a right, but three lefts do.
I'm of the opinion that anything that gets published should be published in its entirety, at least at some point. For example, people who publish protein structures can put the coordinates "on hold" for up to 18 months.
And to say because the research is done with "taxpayer's money" is missing the point: If you can't reproduce every step, it's voodoo, not science. And we make policy decisions based on science, not voodoo (I hope).
Obviously you have never written an SBIR or BAA. You when you do research "At the tax payers expense", you need to show your plans to commercialize the results of the research. The government wants you to create a IP twoards a commerial project which will spur the economy, not to contribute to the scientific community as a whole. Take it as you will, but I think that most research would not get funded if your commertilization plan was to release it on sourceforge.
This is an extremely difficult issue, although it sounds pretty trivial.
For one thing, the taxpayer is rarely participating in discussions like this one. Moreover, the success of scientific institutions is often measured in terms of number of patents, successfully launched businesses by former students/researchers, etc. So not only is there little or no opposition to closed-source software (or scientific articles!), there are also good reasons for researchers to go the closed-source road.
Some researchers have a tendency towards secrecy. Some even seem a little paranoid when it comes to their data and methods. You could compare this to the tendency of the OSS zealot to suspect bugs, glitches, and omissions in any piece of closed-source software.
And as a German side-note: There are laws over here that require you to have the patentability of any piece of software you develop checked by university lawyers. GPLing something is technically illegal for a researcher. I have no idea how this is regulated in other countries.
That Global Warming is a manmade, real phenomona is accepted by 99.9% of scientists in the fields involved. To trot out the "only a theory", "some experts dispute" etc routine is like getting the Flat Earth Society involved every time someone talks about circumnavigation. "Heads in the sand" is going to be on our culture's gravestone when the next lot of intelligent life evolves here and starts wondering why parts of Nevada are 10,000 times the normal radiation level.
The Slashdot Paradox: "100% Overrated"
Not when the secret is current, and espionage is a concern. This is of course the current state of affairs.
Once the secret is no longer of military importance, all information that can be released should be released. In general that's what happens - note the wealth of information available on the WWII atomic weapon program for instance.
Galileo: "The Earth revolves around the Sun!"
Score: -1 100% Flamebait
The problem of course is who decides how secret something is and when it shouldn't be a secret anymore. The government (the current administration moreso than most, but not *much* more) has a definite "make it secret" reflex. It's shocking to me how little outcry various abuses of that have been (like a redacted FOIA on an internal audit, where the redacted sections were exposed, and were revealed to be the parts of the audit where the agency failed). Sadly, although unsuprisingly, both the populace and the government have forgetten who's supposed to be the servant.
Where's that wealth of information about the secret US wars in Central America in the 1980s? Or in Angola in the 1970s? Or in Chile in the 1970s? Or in Cambodia and Laos in the 1960s? Iran in the 1950s? These secret wars are secret largely for *political* purposes - the military secrecy benefits evaporate within months. But the political purposes - covering liability for abuse, war crimes, and just plain lying about the causes, effects, and benefits of the war - those last forever.
--
make install -not war
How many times have you asked someone, "What does your code do to solve this problem?" and got a description of an algorithm which, when you finally get to see the source, does not match the code?
In my case, the answer to that question is, "Lots." I have had it happen in pure science (neutrino physcis), applied science (medical physics) and software development (database programming, data analysis, etc.)
I am painfully aware that my own published descriptions of algorithms have often left out minor details that may be critical in some applications, but that page limits in peer-reviewed journals necessitate. It is not uncommon to get a call from someone doing similar work asking for details about what you've done, how you've done it, and in some cases, asking to look at source code.
In contentious areas of science such requests are not always met with full disclosure, which is a sign that the people involved are no longer doing science. They are doing politics. This happens a lot, and it brings the scientific process to a halt on the question at issue.
In the case at hand, the original authors have done a very poor job of describing what they have done, and an extremely poor job of defending their work. Their refusal to publish their source code for their analysis gives credibility to their critics.
There are certainly legitimate cases where code ought not be published. If a lab has spent many, many years developing a framework for solving a certain type of problem and wants to get the most advantage out of that framework before releasing it, they may reasonably want to limit it's disemination for a while. But those sorts of reason don't apply in this case, and the source should be made available to anyone who wants to reproduce their actual results. That would just be good science.
--Tom
Blasphemy is a human right. Blasphemophobia kills.
One's interests in keeping clients does not entitle you to make a scientific claim that cannot be peer reviewed. If a paper such as Mann is now regarded as fact, and indeed, makes policy, despite the obvious sloppiness regarding its data management process, then, what is the point of science anyway?
Science is supposed to be about peer review, rigor, that every assumption behind every assertion can be challenged. If, all we have is someone with a Phd can claim that they have a fact as our science, then, what is the point of even trusting them?
Without independent verification and an open process, there's nothing to separate scientists from creationists, and the people are going to pick whoever makes the most attractive sales pitch.
This is my sig.
You do know that Mann writes this website, right? You do realize that the source of your argument (http://www.realclimate.org/) is a shill for Mann and his cronies?
Second of all, there was a flaw in the original algorithm that was pointed out by McIntyre and McKitrick before they even got to the bad data being put into the equation.
And, to top it off, Mann's equation always produces hockey-stick graphs, even with randomly distributed data.
Don't point at Mann's own site as a defense of Mann.
Life, the Universe, and Everything... in my image.