Slashdot Mirror
Open Source Intelligence
Artifice_Eternity writes: "Time magazine is running a story highlighting the US government's neglect of open source intelligence, or OSINT. OSINT includes stuff on the Internet and in various newspapers and periodicals, as well as "gray literature" (limited-availability publications like dissertations, local phone directories, etc.). It also includes foreign-language experts, and commercial data (satellite maps, news archives, scientific research). The mass of data to be crunched indicates how intelligence is an information processing problem in today's world."
This is quite funny on two levels:
- It's a lot of "take" and not much "give" on an international level, i.e. not at all open source (unless you take Microsoft's use of BSD code as "open source")
- It's extremely US centric
The second point is forgivable in itself, seeing as he's an ex-spook, and it's an article aimed at improving the US's intelligence. But what's with the open source phrase? How can the rest of the world make use of it? (hint for the easily amused - read "America" and "U.S." as "Microsoft", and "Open Source" as "Embrace and Extend")Non-Governmental Organization Data Warehouse ($10M) to provide free storage and network access to the various international organizations whose "local knowledge" is vital to U.S. understanding.
Regional Open Source Information Networks for Africa, Asia, Europe, and Latin America ($40M) , each with an open source collection and processing center in partnership with local governments who will provide regional language skills and access to gray literature and local experts.
What makes him think that these places - some of the poor and rebellious even internally - will co-operate with the US in matters of security? He's not even suggesting bipartisan sharing, which doesn't even approach what true open-source would be.
The closest he gets to saying that this idea will be truly "open source" is an immediate increase in open source information sharing across the departments and with the private sector; and finally, the provision of a foundation for a web-based OSINT exchange with allies, other nations and international groups, in other words it's only open source if you're in the clique. A bit like any major software company we could name...
And what's this?
Digital Marshall Plan ($20M) to provide direct assistance and subsidies to extend the Internet to every corner of the world (including rural areas in America) via wireless delivery means.
This is another example of how US-centric his ideas are - the most remote corner of the world he can think of is "rural areas in America"...
This is clearly an example of some hyper-patriot using buzzwords and buzzconcepts to expand his country's control over scant international resources (intelligence analyses) without really understanding the international environment, or indeed without really understanding the terms he's using. Open source? Not likely. Open (to him) intelligence sources, closed (to everyone else) information.
jer
We may be human, but we're still animals
- Steve Vai
Unfortunately, our spies and our satellites have lost touch with reality, for they collect less than 10% of the relevant information that we must digest to understand the complex multi-cultural world that is now capable of producing very wealthy and suicidal terrorists.
There's a good reason for the above mentioned figure... While I agree that there's a lot of useful information on the net, there's also a lot of crap.
Any intelligence agency looking to filter out the 99.9% of nonsense that's out there to glean the remaining .01% of useful information faces an incredible challenge. That's not to say that it's impossible, just very, very difficult, time-consuming, and expensive. For every real threat being posted on the net, there are tens of thousands of harmless, steam-blowing rants posted. And how do you decipher between the two? Do we now get into investigating every idle threat someone posts on a bb, or in a chat channel? The sheer scope of this project would daunt even the most dedicated Government sanctioned snoop.
Just as an example, sift through the comments of a /. article at -1, and you're guaranteed to find at least one or two flame-ridden rants about god knows what. Or look at your own past history. How many 'harmless' comments have you made about the stupidity of this or that idea, or how you'd like to kill that person for doing this? You know its harmless. Anyone who knows you probably thinks the same. But how does someone completely outside of your community know?
The article does make a few valid points, however:
Shocking as it may seem, our intelligence community does not routinely strive to identify the top people in the world (not just Americans) on the various topics of concern -- from terrorism to the environment to human trafficking to corruption to disease and public health -- with the result that our analysis tends to be shallow and incestuous, relying on the same consultants again and again.
I think just about anyone who lives outside the US looking in (as I do) would agree with this statement; one has only to observe the lack of knowledge American citizens display with regards to the rest of the world to see that this attitude is quite widespread, and probably does affect intelligence gathering. Raise your hand anyone who's seen the (Canadian) 22 minutes special "Talking to Americans". It's rather depressing actually, to think that so many people, including prominant politicians, could believe that Canada works on a 20 hour clock, or that we're going to change the country's name to Chicago (I'm not exaggerating either... quite a few people were taken in by this).
The Recommended Open Source Initiatives proposed in the article are interesting, though idealistic. One example: Digital History Project ($5M) to digitize and translate key Islamic, Chinese, and other foreign language historical, political, economic, cultural, social, and technical materials.. Having been involved in translation projects (French to English and vice versa) myself, I think he has seriously underestimated how much this would cost..Translation is an incredibly difficult and time-consuming activity; it's not a simple matter of babelfishing an article. Localized phrases and slang do not translate well from one language to another.
Before you can even begin to sift through the plethora of information, you'll need people that are very net and tech savvy. Combining tech skills with those of an intelligence agent is just the beginning. I won't even go into the thorny privacy issues that could be touched on here... that's just a political bomb waiting to go off.
Moral indignation is jealousy with a halo - H. G. Wells
In the article:
Shocking as it may seem, our intelligence community does not routinely strive to identify the top people in the world (not just Americans) on the various topics
of concern - from terrorism to the environment
to human trafficking to corruption to disease
and public health - with the result that our
analysis tends to be shallow and incestuous,
relying on the same consultants again and
again.
I am not sure how many of you have looked into getting a clearance. There are some serious ethical choices that you need to make in order to get one. Some of these include:
1. Restricted travel.
2. Not being able to associate freely with non US citizens.
3. (probably the most important for the academic types). Depending on what clearence you get and from which agency anything that you go to publish will have to be peer reviewed by people in the intelligence community before it can be published.
THe reason that this is important is that the intelligence agencies can not just talk to any Tom, DIck or Harry about somethings. Many of the academic leaders can not justify the restrictions on personal freedom, so they choose not to work in the intelligence community.
It would be great for them to pull resources off of everybody but, they are limited to what they can talk about and it would be easier for those opposed to our interests to learn what we are up to.
I agree that it would be great to be able to do this but there are several things that need to be resolved before hand.
Either give it away or get top dollar, but never sell yourself cheap.
After all, the computer world has lots of oddities that might seem nonsensical to someone unfamiliar with the details or history of a particular technology.
I am seriously concerned, however, that it looks like the CIA will resume covert investigation within the US... something that has been banned for decades. GWBush's father, when he was CIA Director, prior to being President, resisted this ruling and always wanted to eliminate the restriction. It looks like his son may succeed.
As we become a more computerized and networked society, it seems likely that the once-obvious boundaries of our country will become more blurred. For example, the NSA has been evesdropping on US phone communication without warrants for years, even though they are not allowed to gather information about the US from within the US... just like the CIA. They have gotten around this limitation by using satellites that capture signals escaping into space; mostly microwave from repeater towers.
I have no conclusions or suggestions... just a little food for thought. The most dangerous threat for any democracy is the complacency of its citizens. I am really glad to see that slashdot and other similar tools are thriving. The most dangerous threat to any despot is free communication between the people.