Slashdot Mirror
Open v. Closed Source-Climate Change Research
theidocles writes "The ongoing debate over the 'hockey stick' climate graph has an interesting side note.
McKitrick & McIntyre (M&M), the critics, have published their complete source code and it's written using the well-known R statistics package (covered by the GPL). Mann, Bradley & Hughes, the defenders, described their algorithm but have only released part of their source code, and refuse to divulge the rest, which really makes it look like they have some errors/omissions to hide (they did publish the data they used). There's an issue of open source vs closed source as well as how much publicly-funded researchers should be required to disclose - should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?"
should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?
No. I paid for it I want to see it. How else will we know if it works the way they say it works?
I'm a virgo and on Slashdot. Coincidence? Yes.
how much publicly-funded researchers should be required to disclose
All of it, baby. We're paying for it -- we should have the right to:
a) Know what you're spending our money on
b) Have the right to make it better ourselves
c) Learn of security flaws early so we can correct them
Especially when there is some doubt about the nature of the results in the closed source model from Mann et al.
The dangers of knowledge trigger emotional distress in human beings.
I hate projects with names like R. I used R a while back, and it's a great program, but try searching for "R" plugins on Google. Not fun.
The Bush Science Team America has made an amazing discovery:
Fake Science can be easily proven with Fake Data.
The most important thing is the data. You can even use something simple like the Excel (or Gnumeric) spreadsheet "best fit" plotting algorithm to the data, if you've got it.
But from all the stuff I've seen, there are always huge gaps where they are either assuming much lower average temperatures or are leaving the data out altogether and relying on a very short recent timespan to extrapolate into the future.
While I think that they are full of shit, for the most part, I do admit that having multiple tornados tear apart LA and a giant deep-freeze kill off all the Scots would be pretty cool.
is in the interest of the science in this case.
You might want to ask to see the model behind the CIA data which proved conclusively that a 747, deprived of its forward fuselage, can convince over 600 witnesses that said 747 was shot down by a SAM.
The Slashdot Paradox: "100% Overrated"
Comment removed based on user account deletion
The problem with most of these studies is that they refuse to release the raw data.
... but no thanks !!!
A lot of times they select subsets of the data and then normalize or otherwise massage the data.
Thanks
For all we know, there could be a very valid reason why they haven't released all of it. I'm not sure what that reason could be, but given that we don't have anything to go on, we're stuck to just guessing.
Avantslash - View Slashdot cleanly on your mobile phone.
...and I agree with it -- of course anything paid for by public funding should show a return on public interest. It seems way too obvious.
What isn't quite so obvious is employers owning works of an employee. It seems obvious that it should be restricted to stuff that is currently job related and developed on company time, but we all know of scenarios where companies reach too far. So without looking too deeply, I wonder if the other side considers some aspects of their work not relevant to that which was sponsored by public funds. If, for example, the project was intended to deliver data and software tools were developed along the way to achieving that goal, who owns the software tool?
It's sad to see Slashdot becoming a supporter of the Republican Orthodoxy.
Almost as sad as seeing the ad hominem attacks that some greens use against anyone who questions their orthodoxy...
Best Slashdot Co
Science, like government, should be transparent. The public should be able to see and evaluate every part. Any science, or government, that hides it's implementation is inherently suspect to corruption.
Closed science is half a step from religion. You are expected to have faith in the researcher's methodologies, analysis, assumptions, and motives. Sorry, but good science does not rely on faith.
The pharmaceutical industry receives huge subsidies from us - they don't produce "open" drugs - why should this be any different? I know it's apples and oranges - but one should be really careful about the idea of withholding funds from -good- research just because of licensing issues. Lesser of two evils? Would we rather have -no- research?
complicated...
Surely by publishing the data they have to publish how they came to the conclusion that the data leads to and why that data should be trusted as accurate.
:S)
If they have published, then nobody else can patent the idea, and i would hope that somewhere in the paper would be an explanation of the methods used to calculate the data.
Otherwise it would be like saying AMD/Intel is crap because I say so, here is a link to some data that compares a 486 to a n modern CPU.
(Actually as an aside that does kinda sound like th press release for the Turion Mobile thing or what ever it's called
McKitrick & McIntyre (M&M), the critics, have published their complete source code
... where? I haven't been able to find any code on any on of the pages mentioned. I agree it's essential to disclose all data and source code ...
... especially since R can be such a pain (sorry, struggling right now)
Uhrm
and it's written using the well-known R statistics package
Arguments against open-source science:
I'm sure there are arguments on both sides.
Two wrongs don't make a right, but three lefts do.
I'm of the opinion that anything that gets published should be published in its entirety, at least at some point. For example, people who publish protein structures can put the coordinates "on hold" for up to 18 months.
And to say because the research is done with "taxpayer's money" is missing the point: If you can't reproduce every step, it's voodoo, not science. And we make policy decisions based on science, not voodoo (I hope).
Obviously you have never written an SBIR or BAA. You when you do research "At the tax payers expense", you need to show your plans to commercialize the results of the research. The government wants you to create a IP twoards a commerial project which will spur the economy, not to contribute to the scientific community as a whole. Take it as you will, but I think that most research would not get funded if your commertilization plan was to release it on sourceforge.
While I would like all works performed for the government that are not of National Security importance to be more open I don't think it is necessary.
A lot of work peformed for government agencies is contractual with businesses. These same businesses employ tricks of the trade and such to deliver what is required. To have them detail how the work is just suicidal. The same goes for software they develop for use by the government. Unless specifically addressed in the contract I do not believe there is a right to disclose the code, let alone make it available to the public.
That last part is key. Even if they disclose the source to the government there is no obligation on either party to make it public.
This argument that they have something to hide is childish. It is designed to provide no leeway. Simply put, once labeled as such what other option other than disclosure exist? You might as well say "You have to release it, its for the children" and then proceed to use whole "hates kids, wants kids to die" guilt trip that is far to common in politics today.
Summary. Release it if only its an upfront requirement of the project and agreed upon by both parties. In the future a requirement by law that all government projects must be fully disclosed to include the source of any software may be nice but I bet it would have so many exceptions written into it that it would result only in a "feel-good" law.
* Winners compare their achievements to their goals, losers compare theirs to that of others.
http://cran.r-project.org/
However, the name still sucks.
This is an extremely difficult issue, although it sounds pretty trivial.
For one thing, the taxpayer is rarely participating in discussions like this one. Moreover, the success of scientific institutions is often measured in terms of number of patents, successfully launched businesses by former students/researchers, etc. So not only is there little or no opposition to closed-source software (or scientific articles!), there are also good reasons for researchers to go the closed-source road.
Some researchers have a tendency towards secrecy. Some even seem a little paranoid when it comes to their data and methods. You could compare this to the tendency of the OSS zealot to suspect bugs, glitches, and omissions in any piece of closed-source software.
And as a German side-note: There are laws over here that require you to have the patentability of any piece of software you develop checked by university lawyers. GPLing something is technically illegal for a researcher. I have no idea how this is regulated in other countries.
Are you entitled to all of the NSA's or CIA's secrets just because your tax dollars paid for it? No. In science, when you publish results, you publish it in such a way that others could reproduce it. If they can't, they publish their results and discredit yours. There is no need for everyone to make their source code available for everything they do. Also, it may give your competitors an advantage if you are forced to publish code for which you are going to do more with. Science is fairly open but it is competitive for grants and people are entitled to keep source closed that they intend to do further cutting edge work with. Having said that, I and others often make their code available. It is rare to find cases where people selfishly hoard useful code in my field anyways.
Most projects the ideas are free, but the results are not. Seems strange...
Everything funded by the taxpayers should be free, regardless; ideas, results, and any tools built to obtain those results.
This article has the potential for 2 flamewars. For or against global warming and for or against open source. Oh joy!
Eliminate compulsory taxes, which are unconstitutional anyway, and let people choose where their money goes, if it goes anywhere at all.
That way, silly projects like this would never see any funding.
Ruby Neural Evolution of Augmenting Topologies
This is actually an interesting legal debate. How would you register a trademark on the letter R?
R®????
So a team of real scientists (that is, by folks who work in climate science, not reporters or pundits) wrote a Dummies Guide to the latest controversy. Click on the link for a nice question-by-question breakdown, but I'll spoil the conclusion for you:
(MBH98 is the old paper with "closed" source, MM05 is the new "open source") paper)
Read the rest for more explanation.I was under they impression that, with current laws, they have to disclose the entire source code. Any research that goes on at a university, without being directly classified (such as military funded research may be), should ("should" as in "by law") be obtainable for the general public some way or other.
I am not a lawyer, would anyone who is care to correct me if I am wrong?
We know that the earth IS getting warmer. And we know that it goes through natural cycles of climate change. We know that people affect the climate in SOME way, but the debate too often focuses on how much is caused by man, and how much is natural, as if natural warming is somehow better.
The point so many global warming critics ignore is that whether it is a natural phenonema or not, doesn't change the danger. The amount of crops that can be grown worldwide will shrink for every degree the planet goes up, until evolution kicks in.
Just a few degrees globally can literally end up causing the starvation of millions of people.
I think the important conlusion of this guide is that if you take all of the original Mann, Bradley and Hughes data and run it using the same fully open-source algorithms of McKitrick & McIntyre, you get the same results.
Which is reasonable since MM's argument is about source data and not methodology (as per this guide).
should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?
I think it's not fair for a public-funded project to do something like create a product and sell it without source code, or patent their work. But that doesn't mean that every artifact of the research project needs to be made public, necessarily. In this case, the end product that the grants paid for is the scholarly paper, not a computer program. Just as we don't demand their notebooks, time cards, e-mails, and meeting transcripts, it seems okay to not require them to publish the source code of a tool they wrote in the course of doing research. So I don't believe this is a behavior we should be legislating against.
But this only addresses the question of "should we require them to release the source?" Another undertone of the article is, "should they release the source?" I think it's clear that their work is at the center of a controversy, and that other researchers want to try to reproduce their results. It seems clear that making their specific methodology public (source code) would help answer the controversy, so as researchers interested in the truth, they should release it.
Significant research data is generally replicated independantly of the original researchers for verification of the results. Without a description of the method of research used (in this case; the computer model), how can the data be replicated and thus verified? Indeed the very methods itself are commonly scrutinized in the scientific world and, IMHO, any scientist that does not approve of this is not looking for truth but for something else (personal agendas, fame, etc.).
Not detailing the methods used (in this case; giving the entire algorithms, either as source or as a 100% comlete and unambiguous description) basically limits the usefullnes of the resultant data as mere speculation, not proof nor even theory.
If I remember correctly, the computermodel in this case is known to include a rather lacking model of rainfall, which seems like a pretty big omision in a climate model to me.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
I have to usually do C-x C-s C-x k RET.
Have you set it up so it doesn't ask? Can you tell me how? I would like to know.
Today biology heavily depends on specific software to analyse lab generated data. However, even academic, public funded software are not open-source. It's a sad situation, but there are efforts like Bioinformatics.Org trying to change the situation.
---- Where is my mind?
The UK Meteorological Office model was published under a fairly liberal licence I hesitate to call it "Open Source" or "Free Software", but certainly inspectable and runnable if you had the need.
Whilst in theory you could inspect this to find issues with the model. For most organisations without extensive assistance from the UKMO, wide scientific expertise, would not be able to gain much utility from it I suspect. In practice the main groups who used the model were supercomputer vendors, and computer scientists interested in how to optimise numerical computing solutions.
In some ways I'm not sure it helps the climate debate. I know the UKMO model had bugs, I found some. But by the time software gets this complex you are interested in validating the global behaviour, as much as validating the minutae of the code. If the model can match past climate change accurately you assume that the bugs don't matter that much until proved otherwise.
But ultimately you take evidence for the climate debate from a number of sources. If glaciers many 10's of thousands of year old are disappearing, it is a reasonable bet in my mind this is the hottest it has been for 10's of thousands of years.
Then again you can stare at the evidence, and still miss it, witness the first person who skippered a ship through the North West passage without encountering ice, who thought that the idea humans could melt the arctic ice cap was ridiculous.
I think the important reasons to make government software "free software", have little to do with good science, and more to do with good governance.
Although I can't see how publishing the code would make the science less good, and I can see how it might help.
Besides paying for the research, how can another check on the accuracy/repeat the results without the original code?
Except for ending slavery, the Nazis, communism, & securing American independence, war has never solved anything.
The scientific method:
1. Observe and describe a phenomena
2. Formulate an hypothesis
3. Use hypothesis to predict something
4. Independently verify those predictions
Step four requires, of course, that experiments can be corroborated, which implies that they can be duplicated. That is of course impossible if the tools employed are not shared.
The only scientific result that should be given any regard is that produced by real scientists doing real science. Pay no attention to the man behind the curtain.
s (shown left) agree that the late 20th century is anomalous in the context of last millennium, and possibly the last two millennia.
IMO this is the conclusions dead fly contamination. What you say?
The assumption that an anomaly within a 2000 year span has any significance relative to 10^7 years of climate variation and the conclusion, aside from assertions of 'man as cause' that 'something must be done to resolve this dire situation'.
I'm used to thinking on geologic time scale. The climate is intimately tied to geology through biology, chemistry, and physics.
Sure, climate change affects lifestyle. But, on the whole, life has always adapted and considering that man might not even exist in his thoughtful state absent past dramatic climate change, who is to say it's a 'dire situation for the planet'.
Human migration toward better climes is still going on today. Why is change considered bad?
becom3 obsessed his cla5h with
Public funded means owned by the public. I am not talking about abvious things, like miletary secrets, but reasearch like this.
I asume this research has been done to widen our understanding and knowledge, not for profit. To achieve this goal the best thing is to check, check, recheck and then let others recheck as well. This can only be done if you give up all your findings and ways of how you found it.
This is about knowledge and not about being right or wrong (or at least it should be). The knowledge of proving that the theory is right is just as importand as proving it is wrong.
Don't fight for your country, if your country does not fight for you.
How much trust should we put into a study where the computer simulations code does not come under peer review (closed source) versus one where it does come under peer review (open source). Seems to me that the code is as much part of the "study" as the results and data are. Especially considering how much finagling can be gone on in source code.
Also, since the results have to be reproducable by ANYBODY, without the source you can not garuntee that the program is doing what it is being said it can do. After all the complaining I have heard about black box voting, this should probably come under the same heading of "If you want us to trust you SHOW US THE CODE!".
Fly me to the moon Let me sing among those stars Let me see what spring is like On jupiter and mars
Shouldn't another scientist be able to replicate that experiment? Source code is an integral part and they won't let you know how they did that?
That's BS, and all the more so because of the political implications of such research.
Information: "I want to be anthropomorphized"
Yes. Did you know, humans have an average of about 1.8 legs each (and eyes, and arms, etc. etc.). Anything can be done with a few figures, even more so when the way it is done is hidden (Governments use similar maths wizardry).
A better question is "Are they allowed". The answer, no doubt is in the contracts. If you think that they SHOULD NOT BE ALLOWED IN FUTURE, then welcome to the world of POLITICS. Get involved in the process and learn how competing interests are balanced in the democratic political process, which is not perfect, but is the best one we have.
Who made you arbiter of Slashdot?
Would you believe that this happens more often then not. For example, EOLAS which sued MS for half a billion dollars was funded by the California tax payers thru the college that he worked for. Not only that but the college and EOLAS split the patent in half so the college owns half of the patent that the califonia tax payer paid for. NOW THATS SOME BULLSHIT! I as a tax payer pay for research and some company get to walk away with all the rights that I paid for. NOW THATS SOME BULLSHIT.......
My big beef with this is the idea that pharma companies are actually interested in doing "good".
They're not, for the most part. Antidepressant medication is a wonderful look at the BS they're pulling - Zoloft had a 48% improvement in depression in clinical trials compared to the placebo at 42%. 6% improvement for a host of different side effects, many severe and potential risks to life in the cases of preexisting liver/kidney conditions.
Look to the male enhancement market as well.
Considering the recent FDA news stories (for the last 6 months or so) regarding impropriety in the drug approval process and I think the argument for more transparent, open models hold a lot of merit.
In summation, the pharma industry (and more importantly, our national health) would potentially be improved by a more open standard than we currently have in place.
You better watch out, there may be dogs about . .
I think there is no reason to demand that scientist should publish their source code, since scientist usually reuse their code and share their code with people they work with, but should not be obliged to help other scientist that they are competing for funding with to get their own simulation programs.
The demand on scientist are clear though, they should give enough information in their publications so anyone interested (or who want to refute their results) can reproduce what they have done. So any statistical or mathematical methods used should be mentioned. And if they use commercial packages (with closed source usually for all parties), mention which packages they use would be wise so that if there are found bugs in these programs, any influence on their results can be taken into consideration. If enough information is given, then any scientist who can program, can check out the literature how to implement the nummerical algorithms and write their own program. Often they can buy (fairly expensive) commercial packages or even find open source liberies that have already implemented these algorithms, and then reproduce the results.
If these two economist were able to reproduce the results of some major climate scientist, then these climate scientist have given enough information to their fellow scientist and the general public. So lets forget about these two guys, or buy their book if you want to believe they know better about climate changes than the general scientific community.
--- guns don't kill people, people with guns kill people ---
So the climate change critics' site has direct links to everything that supports their position, but when they mention "realclimate.org" they don't make it a link so you have to cut and paste into the URL to get the other side of the story. That pretty much sums up their intensions and intellectual honest right there.
IOW, it's just more FUD from the corporate lobbies. The "hockey stick" is real, it's too late, and we're all doomed to live in a bio-dome... sad but true.
...are about as respected as climatologists as the eponymous rapper is as an ambassador of world peace.
Summary:
people need to understand that a _research code_ isn't _code_ so much as it is the implementation of an algorithm. we numerics researchers aren't programmers so much as mathematicians at keyboards. we release our "source code" (algorithms) into the world in open source fashion (publications). not releasing a code is often just a statement that we are crappy programmers.
Longer version:
this article reminds me of the old adage, "arguing on the internet is like running in the special olympics..." from my vantage point (i do similarly mathy-flavored research) it looks like both groups are somewhat sloppy, and to boot they are both acting somewhat childishly. they have forgotten that what's important is to _get the math right_, not to point out the other guy's wrong.
releasing codes from a university or lab setting is often a real pain in the ass. if you haven't done a good job keeping the code looking nice, it's really not worth the effort to get it out there, for one reason only: your code should implement the methods discussed in your paper. the IDEAS themselves are what are important, and those are available via the literature (unfortunately, that's not always freely available, but that's a separate issue...). i agree with those who say "but my tax dollars paid for it"--the researchers _have_ done work and released it: developing analysis methods. that's what they're paid to do! the currency in their field is, in fact, how much they publish (ie, release); that's how funding gets decided (i'm oversimplifying, but essentially that's true).
Are we arguing whether tax payer funded software should be released, or the openness of the science behind their claims?
In science the experiment must be repeatable by independent researchers. If substantiating the claim requires the release of source and algorithms for scrutiny then that is what has to happen. Otherwise it is just a claim.
Two different things. As a taxpayer you have a right to decide if there IS medicare and who gets it. Representative republic, you vote in your congresscritter and he/she/it does your bidding. Theoretically, of course.
Once you have decided that there shall be Medicare and that there shall be a bureacracy to take care of it, you have no particular rights to anything that happens inside it, unless its happening to you. Doctor/patient confidentiality applies irrespective of who's paying the doctor, both moraly and legaly.
The researcher taking public money is a completely different case. He's doing research for the government, which means for you. Theoretically, of course. Absent pressing matters of national security, there is no reason that the results of publicly funded research should not be available to taxpayers. You paid for it, you should get to look at it.
One caveat, if the researcher used a proprietary method or machine or software to either acquire the data or process the results, you are only entitled to the data and results, not the proprietary device. The government rented the use of it, they didn't buy the rights to it.
"Their argument since the beginning has essentially not been about methodological issues at all, but about 'source data' issues [...] Only if you remove significant portions of the data do you get a different (and worse) answer."
You're over-trivializing a DRAMATICALLY IMPORTANT POINT. The original study is focused on North American data almost exclusively for certain time periods. That data (from a single species of tree) skews the results in such a way as to make the current trend seem unique and drastic. On the other hand, if you treat that data source in such a way as to balance it with the other data that is available, you see a VERY DIFFERENT TREND!
The response has been to claim that weighting the data in this way reduces the number of data points unacceptably (I would agree, but that doesn't make MBH98 right).
That's the whole point here, and the other side continues to say, "you're throwing away data" when any competent researcher would have thrown it out in the first place (note: there's an exception. if you produced a report that was specific to N. America, MBH98 would be your model, and it seems to be a fine model for that... N. America is seeing record warming as compared with the last few centuries, and that's all you can extract from MBH98).
Also keep some perspective in mind here. We're in a period where temperatures could rise MORE than ANYONE is predicting and not make a dent in the graph over the last 10million years. If you graph out the last 10 million years, you see that temperatures over the last 10,000 years have been part of a huge, cyclical spike in temperatures. We're at what is likely the peak of a drastic temperature swing, and it WILL plumet again into a new ice age (unless we decide to and are capable of coming up with a way to prevent it). I'm not drawing any conclusions from that, just pointing out that there are natural forces at work here, capable of making temperature changes that we a) cannot yet conclusively explain and b) the likes of which no human has ever experienced.
It's important to keep a sense of perspective and to remember that we have very impressive climate models... all of which might be wrong.
The blurb author attempts to paint one side as having something to hide, since they only released a part of their source code. Nevermind that both papers' data can be independently validated--no no, one side is bad for only describing the algorithm and not its source code!
But that's exactly the point isn't it? (At least of the blurb.) The original data was used and the nature of the analysis was called into question. So just as the data is made available for double-checking, shouldn't the means of analysis be likewise made available?
When one dataset determines your conclusion, you have to wonder about the validity of the conclusion. According to MM (if I understand them), the bristlecone pine temperature proxy is the only one that produces the hockey stick graph that all the human-induced climate change advocates love. It's not actual temperature data, it's just a proxy for it.
-russ
Don't piss off The Angry Economist
Medicare, etc, are services. The intellectual property in that case is of little market worth and belongs to the patient as the interest in health privacy trumps the need to know.
However, in the case of research, federally funded research should have a complete disclosure. If you have a scientist doing work, and not disclosing the entire body of it, then in reality, the end product must not be regarded as science, but opinion. If Mann does not disclose his entire body of work used to comprise his conclusions, then how else can we assess whether his conclusions are accurate or not?
Science must be open source.
This is my sig.
Where's that wealth of information about the secret US wars in Central America in the 1980s? Or in Angola in the 1970s? Or in Chile in the 1970s? Or in Cambodia and Laos in the 1960s? Iran in the 1950s? These secret wars are secret largely for *political* purposes - the military secrecy benefits evaporate within months. But the political purposes - covering liability for abuse, war crimes, and just plain lying about the causes, effects, and benefits of the war - those last forever.
--
make install -not war
see the realclimate.org link in the right hand column of the home page, 14 links from the top.
And as for FUD -- as someone else has already pointed out, human-induced climate change is a religion, complete with a priesthood that screams "heretic!" when anyone challenges their "findings".
If the proponents of human-induced climate change really had the answer, why don't their models converge? We seem to get a "wait, it's going to get even warmer!" report on an almost weekly basis, which tells you that their models are all over the place.
If you really think that "the hockey stick is real", go use the same modeling technique on the stock market, and you should make billions. I won't hold my breath.
It's Linux, damnit! Pay no attention to renaming attempts by self-aggrandizing blowhards.
You've just stated one of the strong arguments for government purchase of open source: when the government, representing the people, buys open source software, any of the people can see what we bought. And all of the people can benefit from the efforts of our geek minority to understand it.
--
make install -not war
should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?
No. They should not. All results and all tools shall be open for everyone, even when those scientists are sponsored by private institutions, people or companies.
Sigh. Okay. I've just demonstrated that I've tried to understand the perspective of a group of people with which I disagree. But I still can't bring myself to enjoy their sentiment.
for multiple alignments should be published too, instead of merely the finished multiple alignment with all the "non homologous" sites snipped out.
How many times have you asked someone, "What does your code do to solve this problem?" and got a description of an algorithm which, when you finally get to see the source, does not match the code?
In my case, the answer to that question is, "Lots." I have had it happen in pure science (neutrino physcis), applied science (medical physics) and software development (database programming, data analysis, etc.)
I am painfully aware that my own published descriptions of algorithms have often left out minor details that may be critical in some applications, but that page limits in peer-reviewed journals necessitate. It is not uncommon to get a call from someone doing similar work asking for details about what you've done, how you've done it, and in some cases, asking to look at source code.
In contentious areas of science such requests are not always met with full disclosure, which is a sign that the people involved are no longer doing science. They are doing politics. This happens a lot, and it brings the scientific process to a halt on the question at issue.
In the case at hand, the original authors have done a very poor job of describing what they have done, and an extremely poor job of defending their work. Their refusal to publish their source code for their analysis gives credibility to their critics.
There are certainly legitimate cases where code ought not be published. If a lab has spent many, many years developing a framework for solving a certain type of problem and wants to get the most advantage out of that framework before releasing it, they may reasonably want to limit it's disemination for a while. But those sorts of reason don't apply in this case, and the source should be made available to anyone who wants to reproduce their actual results. That would just be good science.
--Tom
Blasphemy is a human right. Blasphemophobia kills.
One's interests in keeping clients does not entitle you to make a scientific claim that cannot be peer reviewed. If a paper such as Mann is now regarded as fact, and indeed, makes policy, despite the obvious sloppiness regarding its data management process, then, what is the point of science anyway?
Science is supposed to be about peer review, rigor, that every assumption behind every assertion can be challenged. If, all we have is someone with a Phd can claim that they have a fact as our science, then, what is the point of even trusting them?
Without independent verification and an open process, there's nothing to separate scientists from creationists, and the people are going to pick whoever makes the most attractive sales pitch.
This is my sig.
It's all in the subject line. If someone can confirm that'd be cool, but I'm pretty sure that any software developed with NIH and NSF funding is required to be open source.
You also funded Microsoft if you purchased anything from them. It does not mean you should be able to see the source for anything at all.
Why not?
There is a bill before congress right now that says basically that - in relation to automobiles. It says basically that people have a right to be able to fix thier own autos and manufactures do NOT have the right to make you go to a dealer for repairs because they hide the source for automotive computer systems.
Now living in a country where so many people can fix software, is it so hard to see that indeed ther should ALSO be a right for a consumer to fix his or her own software if it is not working? Why should you have to go back to the software "dealer" to fix a problem. There's not even anything like a software Lemon Law to protect you!
It's not that hard to see a bill like that passing someday - perhaps twenty years before the heads of government reach that degress of sophisitication is thinking, but it is not unlikley to see in our lifetimes.
"There is more worth loving than we have strength to love." - Brian Jay Stanley
That makes sense.
Why would I, as a taxpayer, support give-away projects to help countries with lower R&D investments and lower taxes compete against my company and my country?
You do know that Mann writes this website, right? You do realize that the source of your argument (http://www.realclimate.org/) is a shill for Mann and his cronies?
Second of all, there was a flaw in the original algorithm that was pointed out by McIntyre and McKitrick before they even got to the bad data being put into the equation.
And, to top it off, Mann's equation always produces hockey-stick graphs, even with randomly distributed data.
Don't point at Mann's own site as a defense of Mann.
Life, the Universe, and Everything... in my image.
1) Science functions only on open review. If you can't duplicate someone's results, they are useless (c.f. Ponds and Fleischman [sp?]). A scientific result is only of value if it describes a consistent replicatable process. This is why I consider the closed source work to be completely meaningless. It may be perfect, it may be bug-ridden garbage, we'll never know!
2) Every tax paying American has paid for my code and work. While I regularly feel they're not getting their money's worth, I definitely don't feel they're paying me to enrich me. They are, in a very real sense, my bosses, and I AM obligated to report to them, if they care. Think of it as a company requiring rights to your work.
3) As an academic working on a fairly limited budget, open source and free software have been a godsend for me and everyone else I know. We run linux because it's more efficient, secure and FREE; we use free or open-source compilers; and we cobble together high-perormance computers and beowulf clusters out of miscellaneous bare metal and lots of googling. The only piece of software I routinely have to pay for is MATLAB.
... grumble, grumble, grumble, mutter, mutter, Millenium... Hand... Shrimp, I tol' 'em, I tol' 'em.
Basically the government might give with one clenched fist, and take back with a team of mules pulling. The National Security Archives are one group of folks constantly struggling with the layers of governmental coverups. It's ongoing and pretty telling. They are having mixed results, some good finds, then a lot of what they are calling "over classification and pseudo classification" still existing. And then the problem becomes getting the information out to joey and janey citizen and voter, the "news" only mostly covers current, people have just been conditioned to accept todays fairy tales as "data and fact", over and over again. Then years later the real story comes out, by then it's too late to influence elections, etc. Look at the finally revealed data on the "Tonkin Gulf attack" that was the primary "lawful" reason for the Viet Nam war. They have (relatively historically recently)finally and quietly admitted it was an invention, but years too late to make it matter for most purposes.
So, in part I agree, some of what the government does needs to be kept secret, but it appears quite a bit is still overzealously kept hidden, primarily to protect the guilty-of-corruption-and-malfeasance aspects of government.
Most (all) of what is published proporting to be "Green" - turns out to be instead "Green Washing". The list is endless. Why? - Money, money, money, - tenure, tenure, tenure. For "experts" who cannot get a 90 weather forecast right 50 percent of the time - predicting not only the far future of the earth's weather but dictating the exact causes for the weather change - well as they say - "Garbage In - Garbage Out". Now if someone put together a "program" that looked at all the possible variables for weather change (assuming we aren't on a "normal" pattern) - all 300 or 400 variables - then "we" might be getting somewhere. Most of you should likely be worrying more about what your Dupont Teflon stain repellant pants, ski clothes, carpets and wiper blades are doing to the atmosphere and your health because of PFOA.
You can look at it from the other side. You want to do academic reseach on farmaceuticals, but the state provides only half the funding you need. Then the farmaceutical company comes by and says: look, I can pay for the other half, but the deal is that you wait a year with publication so that we can file patents. Take it or leave.
It's an other thing if the pharmaceutical company blocks publication because the scientific results say that their product doesn't cure the disease it was meant to cure.
Avantslash: low-bandwidth mobile slashdot.
Peer-review is the essence of the enterprise of science, for without it, everything belongs in the Journal of Irreproducible Results. So, if you want to keep your research algorithms (or their implementations, for the devil is in the details) closed, you can call it what you like, feeding at the public trough, boondoggle, corruption, pseudo-science...whatever, but it sure ain't science.
-I like my women like I like my tea: green-
should they be allowed to generate 'closed-source' solutions at the taxpayers' expense?
This is one problem that I have with PBS. They pay these companies millions of dollars to develop "Barney". Then, instead of the money from the merchandising going back into funding PBS, it goes into the pockets of someone else. And PBS, despite having a "hit" show, one that generates lots of money, still has telethons.
Coder's Stone: The programming language quick ref for iPad
If one can't reproduce the data or figures of another researcher's results, then it isn't science. And possibly its a wrong result too. And in some cases like medicine, harmful if it is wrong. A researcher should always make 100% his/her code and data available after a reasonable time period (typically a year) to publish it first.
Science functions only on open review. If you can't duplicate someone's results, they are useless (c.f. Ponds and Fleischman [sp?]). A scientific result is only of value if it describes a consistent replicatable process. This is why I consider the closed source work to be completely meaningless. It may be perfect, it may be bug-ridden garbage, we'll never know!
If they claim some algorithm can be used to produce their results, then you can test their claims by implementing their algorithm. That's all the repeatability you need. Simply claiming that you need the code to do that is either politics or intellectual laziness. In this case I suspect politics.
However there was a link to McIntyre and McKitrick's website in the topic summary. Why was it relevant for Timothy to include that link, but not include a link to the matching item on RealClimate.org? Is it just non-scientists who are allowed to have weblogs about this stuff?
Regards
Luke
#include witty_one_liner.h
It's worth noting that, while it makes sense that taxpayer-funded research should generate 'open-source' solutions, federal law dictates otherwise.
The Bayh-Dole Act was passed 25 years ago, which dictates:
So in other words the government has dictated since 1980 that government-funded research should not produce open-source solutions, necessarily, as the results of research are to be considered private-sector profit-generating centers for the host universities. (The implications for the 'next BSD4.3 TCP/IP stack', or similar advanced research, are obvious.)
Anyway, regarding the 'hockey stick' controversy, Tim Lambert's weblog is worth a read.
You haven't been in the US very long, have you?
One's interests in keeping clients does not entitle you to make a scientific claim that cannot be peer reviewed. If a paper such as Mann is now regarded as fact, and indeed, makes policy, despite the obvious sloppiness regarding its data management process, then, what is the point of science anyway?
As sad as it is to say, people will believe the Mann paper no matter what is published. Look, the source data are published already and people still believe. Numerous independent reviewers (which is to say _everyone_ else) have debunked Mann and people still believe.
They don't believe Mann because it's verifiable (which it isn't), they believe it because they want to. Or, they claim to believe Mann so that they can justify the self-serving actions they want to take.
The fight we're fighting isn't to convince people that global warming is happening. Really. It isn't. What we're really fighting is to get them to do something about it. It won't be until _after_ we've sold them on taking action that they'll admit that global warming exists.
There's no need to publish Mann's code to peer review its science. Peer review has already happened. The scientific community is already convinced. The only people still claiming to be unconvinced are those who ignore anything that doesn't suit their interest.
"We reject as false the choice between our safety and our ideals." --The American President (20.1.2009)
Yes, but here on /. we're supposed to start a big flamewar everytime something with "climate change" in the title comes up. :)
Avantslash: low-bandwidth mobile slashdot.
The Wall Street Journal has a short history of the hocky stick dispute here here. (free registration)
Ceci n'est pas une signature.
And, to top it off, Mann's equation always produces hockey-stick graphs, even with randomly distributed data.
The above remark appears seriously dubious. See http://www.realclimate.org/index.php?p=98.
We're using one of the most commonly used ocean models in the academic world. Bug reports, some extremely serious, but not catastrophic (crash-inducing) come out monthly (for example, failure to solve horizontal diffusion). In a standard academic paper, I can barely fit all the partial differential equations for these models, let alone the particulars of grid discretization and the choice of high-order solution schemes for spatial and temporal derivatives, some of which vary depending on the nature of the quantity in question.
If you have my code, you can tell exactly what I did.
Scientists do not have large code shops. One recent project focused on modeling the entire west coast from Baja to the Bering Straits and had about 6 programmers at 4 institutions in 3 time zones. All had multiple other responsibilities and only three understood most of the code (I am not one of them and none of us realized we weren't diffusing); none of them had a system administrator or technical support. The release version has O(10e5) lines of FORTRAN 90 code.
... grumble, grumble, grumble, mutter, mutter, Millenium... Hand... Shrimp, I tol' 'em, I tol' 'em.
- the one who says there are no problems
- the one who says we are hopelessly doomed
- the one who says we are doomed unless... (and oh, more money is needed to determine specifics)
Plus you have to please the department chairman and other faculty if you want tenure. By the time you do have tenure, you've invested in a reputation for a particular set of beliefs.Look at all the crap that has persisted until just recently:
Microkernels were all the rage in Computer Science. To get funding for OS research, you had to at least pay lip service to this crap. Cheating on benchmarks was normal. It took over a decade to mostly purge Computer Science of this obviously flawed idea.
Stomach ulcer bacteria were thought to be totally impossible. Nothing could live in the acid of the stomach. A doctor had to infect himself and cure himself before being taken seriously.
Cancers create a blood supply, but all the researchers denied it. A surgeon knew though; he could tell by the heat of the tumor that it had an unusually rich blood supply.
Climate change is no different than these other examples. The academic politics mean that one must toe the party line to get ahead. To even be a professor, you must support the politically correct conclusion.
I hate to point this out, but your fighter pilot once worked for the government. Oh, but wait, his story supports your point of view, so different standards apply, right?
If you were to do a little research on the subject, you'd find out that eye-witnesses are usually the least accurate source of information. Try looking at the research of Elizabeth Loftus, and you'll see why some of us are unimpressed by the stories of the witnesses, over the findings of a metallurgist who was actually looking at was left of the aircraft in question. If you could come up with convincing piece of physical evidence, a video recording of the "attack" for instance, you might be more convincing.
Many posts above use the argument that public funding should yield open results. I'm all for that, but the stronger argument for opening the original source is that the future costs will be much, much greater. The Kyoto protocol will costs billions of $,£ or whatever, and the hockey stick has been "talismanic" in promoting the protocol. See Spot the hockey stick at McIntyre's web site.
We do know where medicare is spent. We can see what doctors are prescribing, how much it costs, etc. We can see everything except for the patient's name connected with that treatment due to confientiality, but the feds (and indirectly, you), have that right. Oh, and we also know who is on medicare adn typically what doc. they are connected to.
I prefer the "u" in honour as it seems to be missing these days.
The whole point of science is not so that we can trust the opinions of scientists, it is so that scientists can give us repeatable steps to demonstrate a new point.
This whole notion of "it makes the scientists happy so we should just trust them" goes against every single thing that we in the west have fought for since the renaissance.
Your whole argument illustrates this problem precisely. You argue that, "well, even though the key piece of statistical evidence in global warming is questionable, we should still believe in the conclusion."
This is so wrong.
Maybe if scientists published all of their data in a uniform format, to a uniform site, with exact steps to reproduce, all of their source data, and how they draw conclusions from them, then, you might have a field that is useful. But right now, you have got hyper expensive journals all over the place as a repository for articles that only sketch out a discovery and not actually do it, and that simply is not good enough to be taken credibly.
The scientific process is excellent. But today's scientific product sucks.
This is my sig.
I switched to A.
I prefer the "u" in honour as it seems to be missing these days.
I have repeatedly seen glorious pictures of beautiful ocean models, only to discover, after a few months of working with the scientist in question, that they have artfully underemphasized the failings in their models, while, at the same time, being perfectly accurate in their description of the algorithms. It's only when you use their tools that you understand their errors.
If you can't reproduce the model based on the algorithms, then they didn't disclose enough details of the algorithms. If the model is too large and complex to reproduce in that way, then I would question whether it's science at all... regardless of whether they released the source code or not.
If you really need to ship the source code around, then you really have no better understanding... and probably less... of the models and algorithms you're working on than Metzger understood the biology and biochemistry of his fruit flies. The software isn't a model, any more, it's an experimental subject.
All had multiple other responsibilities and only three understood most of the code (I am not one of them and none of us realized we weren't diffusing)
If the people working on the code didn't understand it well enough, then it wasn't science, it was... I don't know, philosophy, or art.
I doubt Richard Feynman would have been impressed.
Arse.
I shoulda previewed...
you're working on than Metzger understood the biology and biochemistry of his fruit flies
I meant to say...
you're working on than Metzger understood the biology and biochemistry of his fruit flies when he started
I think it's safe to say, though, there's probably as much manufactured "science" as there is manufactured "news".
This is a big problem in all of Computer Science. Journals and conferences should get a backbone and refuse to publish experimental results without source code. There is simply no excuse. The ability to re-produce results is vital. Especialy in fields like AI.... That's why I try to stick to theory ;)
bash-2.04$
bash-2.04$yes "Don't you hate dialup connections?"| write USERNAME
So your argument is that if you remove just enough of carefully selected data, you can come to the "real" results. While if you use too much data, the wrong results must come out.
Lars T.
To the guy who modded me down from perfect to terrible Karma - Apple haters still suck
Lars T.
To the guy who modded me down from perfect to terrible Karma - Apple haters still suck
Maybe we are getting warmer over a timescale of 100 years, but this is nothing within a much longer cooling trend that will have us freezing our asses off 2000 years from now.
If, on the other hand, you are more concerned with the facts than with PhD flame-fests, you can try it for yourself and see. I did this a few years back (but if you don't believe me, try it for yourself) and came to the realization that you can't just extend Fourier analysis to imaginary periods (i.e. exponentials), which is in effect what they are doing. Why? Because in series for any finite set of random data, there will be some term with a positive exponent and it will quickly grow to dominate the expansion when you project into the future. In other words, such a series does not converge as time goes to infinity.
I believe that this is the sort of question begging "a priori" factor that they mention in the links everyone keeps citing--if you know, in your heart of hearts, that global warming is real, well of course you have no objection to a model in which exponential run away is built in. And this may be fine if you are trying to quantify something that has already been established, but it is worthless for answering the question "is this exponential growth or not."
But even though they may have limited utility in such cases, I personally, have no faith whatsoever in such models, regardless of whether they are applied to the weather, the stock market, or the number of bunnies on an imaginary island. Why? Because it's too easy to make them say what you want them to, and the people who choose them are only human. I bailed out of the stock market in early 2000 and I've never lost a nights sleep over global warming.
--MarkusQ
One can. And one does. However, the type of model I'm describing is very similar to the climate models described above. Whether or not it is SCIENCE, it is widely held to be so by all major universities and government funding agencies. My opinion is a bit more skeptical, but I know where some of the skeletons lie.
If you're saying the problem is too complex and should not be attempted, then I might agree. Otherwise, I suggest you go here and simplify things for us. And I'm sure there are still some bugs.
The attached model has been under continuous development for about as long as Windows... probably longer, though with a much smaller budget. It is one of about 5 such packages in the world today. Casually whipping up another such model is not in the cards.
... grumble, grumble, grumble, mutter, mutter, Millenium... Hand... Shrimp, I tol' 'em, I tol' 'em.
For example, the last time the oceans warmed up enough that methane hydrates bubbled out:
3 4
d ec 02.htm
0 .htm
QUOTE:
"... A long lifetime for CO2 adjustment is also consistent with an isotopic event in the deep sea sedimentary record from 55 million years ago, the Paleocene/Eocene Thermal Maximum event. The record tells the story of the sudden release of an isotopically light source of carbon, triggering a fast warming in the deep sea of about 5 degrees C. Both the carbon isotope signal and the temperature (inferred from oxygen isotopes) then relaxed back toward their initial values in about 100,000 years. If the released carbon were initially in the form of methane, it would have been oxidized to CO2 within a few decades, but as CO2 it apparently stuck around, warming the deep ocean, for a long time before it went away...."
END QUOTE
from: http://www.realclimate.org/index.php?p=134#more-1
Also see:
http://www.mala.bc.ca/~earles/methane-wipeouts-
(nice graphics page, makes you wonder what a big subduction earthquake offshore of Vancouver Island will produce -- likely a LOT of methane gas bubbling up).
His references are:
Pecher, I., 2002, Gas hydrates on the brink, Nature, Vol. 420, p. 622-623. (December 2002)
Wood, W., Gettrust, J., Chapman, N., Spence, G. and Hyndman, R., 2002, Decreased stability of methane hydrates in marine sediments owing to surface roughness, Nature, Vol. 420, p. 656-660. (December 2002)
For background information on methane hydrates see: http://www.mala.bc.ca/~earles/m-hydrate-nov99.htm and http://www.mala.bc.ca/~earles/mh-instability-apr0
If you're saying the problem is too complex and should not be attempted, then I might agree.
I'm not suggesting anything quite so definite.
I'm simply saying that if you need the source code to evaluate a model then you're either not qualified to evaluate the model or the model is so complex that source code access is not going to tell you if it's valid or not.
The attached model has been under continuous development for about as long as Windows.
Funny you should say that. In my opinion Windows should not be used for quite a wide variety of applications simply because it's to complex to determine whether it meets the requirements.
I'm a scientist (with a computer science background). I've seen the code that most of my colleagues write. You don't want to see their code. Trust me.
That doesn't change the fact that Mann's results are essentially in line with everyone else's, and that the published McIntyre & McKitrick stuff appears to be completely uninformed by very much relevant skill. Open source code isn't always good code.
I don't know why Mann's complete code isn't published. Possibly the computers have all been surplused and the backup tapes misplaced, so he can't get bit-for-bit what he published. There's no sign that Mann behaved in any way other than up to the standards of the day, and their work is still in play in the discussion of the millenial reconstructions.
I think the only good thing to come out of MacIntyre & McKitrick's "work" is that it motivates support for open source. Unfortunately, the usual lies about global warming being an argument between two equally sound scientific camps gets reinforced in the bargain. On the whole, we're probably worse off, but the demand for open source from the Wall Street Journal crowd is certainly a silver lining.
mt
Climatology is a basic science program, not a development grant. I think climatology is funded through a peer review mechanism similar to NIH. NIH has been pushing for open access. and it's likely other agencies will be forced to follow suit. One rule they settled on was to make NIH-funded research papers freely available after 12 months.
The technology transfer programs you describe largely plague the Department of Defense, though I can see why other non-basic-science departments might use that mechanism too.
"Real scientists" huh? You mean like the entire community of "real scientists" that came to "scientific concensus" without a single scientific entity, government body or anyone on the IPCC's behalf ever bothering to reproduce Mann's findings prior to them being accepted as gospel and the root basis for my government (Canada) spending 10's of billions of tax dollars on unachievable and useless greenhouse emissions targets?
Pffft.
C'mon you have mod points so use them!
Mod Parent +1 Interesting and then -1 Overrated!
Main Entry: crackpot
Pronunciation: 'krak-"pät
Function: noun
: one given to eccentric or lunatic notions
Tim Lambert, Blogger without a portfolio
"Bloggers are just nerds in pajamas"
Jonathan Klein
"Tim Lambert - an arch opponent mostly criticizing unrelated work in which I'm not involved. I haven't seen any substantive criticisms."
Stephen McIntyre.
Okay first off, my reply was to the posting of cvdwl "or is it some crackpot in the back end of nowhere splashing up a web page because he's peevish and doesn't get out enough?! "
I used shorthand to reflect his entire meaning. So there are actually 4 items in contention here.
1. crackpot
2. in the back end of nowhere
3. splashing up a web page because he's peevish
4. doesn't get out enough
You live in Australia, I think we can grant point 2.
peevish
Pronunciation: 'pE-vish
Function: adjective
Etymology: Middle English pevish spiteful
1 : querulous in temperament or mood : FRETFUL
2 : perversely obstinate
3 : marked by ill temper
Simple Google search on your name I think proves point 3. Certainly it's subjective; I'll leave it to the reader to decide.
From your website:
"What I Teach:
Computer Graphics
Computational Geometry
I use Java for teaching. "
I think between J. Klein's comment and your self admitted profession we can also stipulate point 4.
Okay like holocaust deniers you don't believe that "Global Cooling" was never an issue in the 70's
Newsweek 1975
wiki
Most people around during that era remember it, but you deny it.
As to the Hockey stick, the Global Warming religion is backpedaling like mad away from the Hockey stick, yet you continue to defend it. Continued in part II
For the sake of brevity I'll limit myself to the 4 claims of the Parent. denies that average temperature is meaningful, confuses degrees with radians, invents a whole new temperature scale, replaces missing data with zeroes Average temperatures: From reading your site you either have a remarkable lack of reading comprehension for someone who has attended university or else you could not attack the research in any meaningful way so you chose slight of hand. For your edification, the comments by McKitrick discussed the change in sample size, and how it effects mean average. To borrow your example "if I have one kg of water at 20 degrees and another at 30 degrees, then their average temperature is 25 degrees." Yes this is true, but if I then increase my sample to the other five barrels, also of one kg next to your hypothetical two samples, and they are of 0C, 15C, 2C, 18C and 32C, then your average is 16.71 degrees C. Which was the entire point. Tangentially he was discussing Urban Heat Island effect as it affects results in the GHCN. Due to budget cuts measuring stations were decommissioned, the majority of these were in rural areas, leaning the data towards urban areas, where the heat island effect is pronounced. Since Global Mean temperature is irrelevant in relation to population density, removing rural measurement stations skews the data, but most scientists understand this. In a similar effect, there is a lack of temperature data for the South Pacific, and South Atlantic this is due to the lack of monitoring stations, and ships, in this area to make complete studies. For your knowledge, this is a large area that extends almost exactly due west of your current location, extending to the west coast of South America, then from the East Coast of SA further unto the west coast of Africa, and then from the East Coast of Africa to the Western shore of your fine backwater country. This is larger than 50% of the southern hemisphere, of which we have little data, this also unfairly skews the Global Mean temperature. Since you professed to have such a grand understanding of "average" I assume you realize to have an accurate measurement of the global Mean temperature you should have evenly spaced global samples, You can't measure 6 Places within 100 Miles of Hong Kong and profess to have a sufficient sampling. Sampling based upon population intentionally or not, is also flawed. Oh then you decide to further reinforce his point by stating the total amount of stations lost was actually higher than he mentioned, increasing the effect, not reducing it. Degrees/radians: Why are there so many references that have no relation whatsoever to what you are discussing? More importantly you point out a minor math error, that has since been corrected, and the paper still stands. Shall I find a minor math error in any Global warming paper does that discount the entire paper? Particularly if it is not corrected? For instance how CGCM1 and HadCM2 incorrectly estimated 20th century global warming increase by 300%, and actually was less predictive than random numbers. You also mention that this invalidates all of McKitrick math, but you don't say by what magnitude, If I miscalculate a 20% tip on a $25.00 tip as $5.000000000001 instead of $5.00 does it matter? Continued in part III
Anyway, this whole discussion is beside the point: In http://cgi.cse.unsw.edu.au/~lambert/cgi-bin/blog/s cience/McKitrick Tim Lambert's blog I found a reference to ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/IT RDB/NOAMER/pca-noamer.fMike Mann's tree-ring source program (fortran), which he apparently has made public. Happy refuting!
Open source is the standard in science. People who do not publish their work have something to hide. Research funded by the public must be made public. Period.