CIA Software Developer Goes Open Source, Instead

jamie found this piece, at Wired's Danger Room from a couple of days back, about an encouraging sign for the growth of open source in the military / intelligence sphere. "For three years, Matthew Burton has been trying to get a simple, useful software tool into the hands of analysts at the Central Intelligence Agency. For three years, haggling over the code’s intellectual property rights has kept the software from going anywhere near Langley. So now, Burton’s releasing it — free to the public, and under an open source license."

Comments

This fucking guy

[oldhack]

Doesn't he understand how the revolving door system work? Why is he fucking with our common well? Damn. Like we don't all have boat payments and stuff.

An interesting counterpoint...

[Anonymous Coward]

on why ACH (the subject of this story) might not have been readily adopted in some quarters.

Summary:

Ironically, the widespread adoption of ACH as the official method for hypothesis evaluation is the result of a failure to consider alternative hypotheses (ie. alternative possible answers to the question, 'What would be the best way to make hypothesis evaluation more rigorous and reliable?') ACH has been falsely assumed to be (a) valid and (b) the only game in town. That is just the kind of 'jumping to conclusions' that ACH would supposedly help us avoid.

So when we hear about software for ACH failing to be adopted by the US intelligence community, we shouldn't assume that it is another case of tragic bungling by massive bureaucracy. In this case, it might in fact be a lucky escape.

Re:An interesting counterpoint...

[ewanm89]

The answer as in an expert system software is not to entirely rely on it, but use it as a tool in your arsenal to help you do the job. Yes a computer can't figure every conceivable option in most circumstances, but neither can a human, the key is they my both come up with solutions unique to one another.

Wired... empf

[El_Muerte_TDS]

There is something about Wired I cannot digest since the whole wikileaks farce.

Re:Wired... empf

[Jah-Wren Ryel]

He's talking about Wired's connection to Adrian Lamo who claims to have outed the guy apparently responsible for leaking that video of the civilians being gunned down by a helicopter and perhaps even the latest round of documents. Without getting into the details there is something fishy about the relationship between Lamo and the reporter at wired that wrote (broke?) the story.

Re:Wired... empf

[PopeRatzo]

there is something fishy

When has Wired magazine been anything besides glossy fishwrap? Their website is your standard Conde Nast press release publishing machine. There is so much fishy going on at Wired magazine between the editorial, advertising sales and the PR industry that whenever I read something of theirs I come away feeling like I'm covered in grease. There used to be a couple of good bloggers over there, including the great Bruce Sterling, but even he has started mailing it in, probably because even submitting stories to Wired leaves him feeling like he's covered in grease, too.

The last straw came a long time before the filthy business between Adrian Lamo and the editorial staff's sucking up to power, in true Conde Nast style and selling out wikileaks.

As hard as they try to appear hip and edgy, they're really nothing but part of a huge corporate billboard machine. There are dozens of excellent sites on the web that cover technology and culture much better. There's no need for anyone to visit or read Wired.

Re:Wired... empf

[Raenex]

Well ok, for the sake of others following along I found an article on Salon airing out these suspicions: http://www.salon.com/news/opinion/glenn_greenwald/2010/06/18/wikileaks

Most of this just seems to be anger directed at Lamo and Wired via proxy.

Are Expert Systems Still Around?

[Gazzonyx]

You know, I remember reading about expert systems when I was a kid... are they any better or more intelligent than they were 15 years ago? Expert systems seems to be like artificial intelligence; mostly unheard of outside of academia with very few breakthroughs technologically.

Re:Are Expert Systems Still Around?

[martin-boundary]

Both AI and expert systems are widely used without people realizing what they are, though.

For example, everybody who has an email spam filter uses one. If it's based on rules like name of the sender, source IP etc, then it's an expert system in disguise. If it's based on bayesian tech, it's AI in disguise.

The labels AI and expert system are slightly toxic, due to the overpromising about them that was done in th epast, but the fundamental ideas are sound and useful.

Re:"Open Source" tells us almost nothing

[Anonymous Coward]

If you had bothered to actually RTFA, you'd have seen it has been released under the Apache license. While not a BSD license, it's about as liberal and "do whatcha want" as most OSS licenses get.

Re:Do we want that?

[WWWWolf]

Do we really want growing open source use in the military / intelligence sphere?

The article mentions several good points, biggest of which is that it stops people from reinventing the wheel all the time.

Where is the border between helpful and harmful, and where is the moral event horizon for the contributors?

All of the definitions of the open source and free software currently say "no discrimination of fields of endeavour" or something similar. Software shouldn't be "private use only" or "private or non-profit use only" or "only for use in field X".

What would you say if you found an awesome graphics application, but its license said "only for professional design industry use"? A license like that would annoy art students (who aren't in the industry yet), independent artists (who don't give a damn about the "industry"), or plain old normal people who happen to have a need to patch up some graphics some times (and who think "industry" = "they'll charge a lot of money from us if I want anything done").

From the description, it sounds like this software package would be very useful for researchers, analysts, and maybe even lawyers. Is arbitrarily limiting this software to "only for military intelligence use" really fruitful?

"Software for Analysts" sounds harmless, but could very well be their best shot at re-creating 1984.

There are more than one software packages in existence. They have widely varied forms of operation. Software vendors are capable of producing very different products that have nothing to do with each other.

Let's try this conspiracy theory in private sector: "Microsoft released Windows, which was their opening salvo for an unspeakable horror unleashed upon mankind in form of Bob." Yeah, that conspiracy worked really well and now Windows is suspicious. (Well, Windows is suspicious, but not for this reason.)

Is it really encouraging to have Echelon being empored by open source to eavesdrop on even more emails and phone calls? Or how about drones, avionics, etc? Would you feel empowered by having a killbot using your code?

Here's the thing: You could say the same thing about science. You can use science to explore the universe and improve the quality of life. But at the same time, you can use science to blow the shit out of your enemies. People discovered rockets - and now they can be used to both propel people to the moon, and to propel warheads across the world.

Like science, software solves problems. Sometimes these problems can be applied to problems that either morally sound or morally questionable.

Who says Echelon's code couldn't benefit morally acceptable uses? The details are scarce, but assuming the system exists, it must process tons of data really fast. Telephone call analysis part sounds very interesting - even the best publicly available speech analysis systems are very weak and there's certainly a legitimate, pressing need for actually working automatic speech transcription. Drones and avionics? Tons to pick apart, but even I could list a few things that come to mind - navigation systems (route finding, location awareness/reaction stuff) would be awesome. Smart weapons do a lot of image processing, too; identifying people and reacting to their movements sounds like a tough image processing challenge - and if the science behind it was more accessible to people, it could be used for all sorts of cool things.

You may say that this is backwards, but the direction doesn't really matter. If you build any publicly accessible piece of software, it can be copied and reverse-engineered by people who are up to no good, if it helps them to accomplish their goals. The military keeps an eye on the scientists and their new discoveries and wonder how this helps them to blow more people up. They get the

The need for open source sensemaking tools

[Paul Fernhout]

I posted two comments related to this issue of open source sensemaking tools to understand how socio-politico-techno-economic stuff works at the following URL in response to a larger issue raised by Marshall Brain on the USA's ongoing economic decline:
http://blogs.howstuffworks.com/2010/08/06/makes-you-think-in-america-we-realize-that-our-children-will-do-worse-than-their-parents/

In short, I feel open source tools for collaborative structured arguments, multiple perspective analysis, agent-based simulation, and so on, used together for making sense of what is going on in the world, are important to our democracy, security, and prosperity. Imagine if, instead of blog posts and comments on topics, we had searchable structured arguments about simulations and their results all with assumptions defined from different perspectives, where one could see at a glance how different subsets of the community felt about the progess or completeness of different arguments or action plans (somewhat like a debate flow diagram), where even a year of two later one could go back to an existing debate and expand on it with new ideas. As good as slashdot is, such a comprehensive open source sensemaking system would be to slashdot as slashdot is to a static webpage. It might help prevent so much rehashing the same old arguments because one could easily find and build on previous ones. Hopefully in a better way than this classic: :-)
"Argument Clinic Sketch by Monty Python"
http://www.youtube.com/watch?v=kQFKtI6gn9Y

As I mention in my comments to Marshall Brains' blog entry, Elizabeth Warren did a terrific job of socio-economic sensemaking, in terms of "The Two Income Trap" and her presentation on the struggles of US middle-class families in the video Marshall Brain linked to. But why should even Harvard Law professors essentially wing it as far as sensemaking with only email, spreadsheets, and word processors, probably working mostly alone, and in a way that she can not easily share all the details of her explorations? Especially when the USA has invested, probably, literally billions of dollars to create software to help groups of people collectively understand complex social and economic issues? And given the US is likely to spend billions more in this area? And given that, if we have any faith in "truth", one would hope that helping everyone in the world come to a better understanding of various truths and a better understanding of each other would, in general, lead to less conflict rather than more?

I also commented on that idea about a year ago:
"[p2p-research] FOSS modeling tools (was Re: Earth's carrying capacity and Catton)"
http://listcultures.org/pipermail/p2presearch_listcultures.org/2009-August/004130.html

I tried a little to put together a non-profit foundation to do that, so far to not much success.

And here is why I feel the (non-secret) results of any public funding should be open source rather than proprietary:
http://www.pdfernhout.net/open-letter-to-grantmakers-and-donors-on-copyright-policy.html
http://www.pdfernhout.net/on-funding-digital-public-works.html

I feel there is room here for an entirely new approach towards structured collaboration across the internet. It has its roots in Doug Englebart's Augment ideas from the 1960s, and in scale may well be the next Red Hat, Wikipedia, or even Google (whether for-profit or non-profit). Or, it is possible it may be some bunch of related companies and non-profits, all using a common infrastructure