Slashdot Mirror
Open Source Intelligence
Artifice_Eternity writes: "Time magazine is running a story highlighting the US government's neglect of open source intelligence, or OSINT. OSINT includes stuff on the Internet and in various newspapers and periodicals, as well as "gray literature" (limited-availability publications like dissertations, local phone directories, etc.). It also includes foreign-language experts, and commercial data (satellite maps, news archives, scientific research). The mass of data to be crunched indicates how intelligence is an information processing problem in today's world."
And then, they'll have the source code to it.
Whoopity-doo
I know the Secret Service reads kuro5hin. They hauled one guy in for questioning after a post he made there.
Best Slashdot Co
They've been doing that for a long time. Computers and the net has just made it easier.
isn't the US supposed to make all that intelligence availible to the public after 30 years? If so, is there a site where I can look at all that "gray" information? hell, not like the us follows its own laws anyways ;-)
Sig you!
If it dosen'tome from a reputable source, why bother. We all know reputable sources are only in it for the money. Otherwise they'd be godless heathens only out for the common good. Damn communists.
Shift happens. Fire it up.
...it's called google.com.
This is quite funny on two levels:
- It's a lot of "take" and not much "give" on an international level, i.e. not at all open source (unless you take Microsoft's use of BSD code as "open source")
- It's extremely US centric
The second point is forgivable in itself, seeing as he's an ex-spook, and it's an article aimed at improving the US's intelligence. But what's with the open source phrase? How can the rest of the world make use of it? (hint for the easily amused - read "America" and "U.S." as "Microsoft", and "Open Source" as "Embrace and Extend")Non-Governmental Organization Data Warehouse ($10M) to provide free storage and network access to the various international organizations whose "local knowledge" is vital to U.S. understanding.
Regional Open Source Information Networks for Africa, Asia, Europe, and Latin America ($40M) , each with an open source collection and processing center in partnership with local governments who will provide regional language skills and access to gray literature and local experts.
What makes him think that these places - some of the poor and rebellious even internally - will co-operate with the US in matters of security? He's not even suggesting bipartisan sharing, which doesn't even approach what true open-source would be.
The closest he gets to saying that this idea will be truly "open source" is an immediate increase in open source information sharing across the departments and with the private sector; and finally, the provision of a foundation for a web-based OSINT exchange with allies, other nations and international groups, in other words it's only open source if you're in the clique. A bit like any major software company we could name...
And what's this?
Digital Marshall Plan ($20M) to provide direct assistance and subsidies to extend the Internet to every corner of the world (including rural areas in America) via wireless delivery means.
This is another example of how US-centric his ideas are - the most remote corner of the world he can think of is "rural areas in America"...
This is clearly an example of some hyper-patriot using buzzwords and buzzconcepts to expand his country's control over scant international resources (intelligence analyses) without really understanding the international environment, or indeed without really understanding the terms he's using. Open source? Not likely. Open (to him) intelligence sources, closed (to everyone else) information.
jer
We may be human, but we're still animals
- Steve Vai
Geez, I wonder what would happen if they spent their time reading through /. comments. They'd be pretty scared. I feel sorry for the dude named Anonymous Coward living in South Dakota...
scott
This may be a bit anal of me, but did anyone else notice the verbiage in the first paragraph of the article... I have supplied the bolding to highlight the peculiar wording in the excerpt below...
"[...] paying more than $30 billion a year for a national intelligence and counterintelligence community to protect it from both traditional state-based threats and unconventional non-state actors, the events of 9-11 demonstrated our inability to detect and prevent bold asymmetric attacks [...]"
Non-state ACTORS , huh? Hmmm... I can definitely think of some actors that could be of a real and present danger to the US - most of whom are imports from our friends to the north, Canada!
And for those of you who appreciate long and confusing acronym's, try this one on for size: The Open Source Intelligence International Non-State Actors Watch List, or as its friends call it - the OSINT INSAWL....
Beer is proof that God loves us and wants us to be happy. -- Benjamin Franklin
I think this is carrying the "open source" moniker a bit too far.
What we're talking about is simply publicly available information.
This guy is advocating gathering it and sifting it for useful nuggets of intelligence, a goal with which I agree."We're sorry, but the website you're trying to reach has been disconnected."
The governments use of the word Open Source is different than what we mean...
To the intelligence community, something like Time Magazine is an open source of information. Open, because everyone can have it.
A phone tap, classified information from another agency, a spy, etc are closed sources of information.
This does not mean that they are reading Slashdot, or reviewing the Linux source code. (I'm sure 'they' ARE, but thats not what this term means...)
Remember May 7, 1999? Chinese embassy in Belgrade accidentally bombed because it was down on old maps used by US military planners as a Yugoslav government agency.
You do realize that this has nothing to do with open source software...
Granted his stuff is fiction, though quite a bit of it has been described as "unsettlingly accurate" by government types. In all of his books it's made note that nearly every Intelligence group has tvs turned to CNN and the like. I believe in one even CNN was called "the best civilian intelligence agency".
The US Government should change copyright policy to require electronic deposition. If every copyrighted work was available in data form via the Library Of Congress, OSINT would be a lot simplier and cheaper. Copyright interests would not like it much, but compared to the damage done to civil liberties so far in the aftermath of 9/11, it's a small sacrifice.
"How perfectly Goddamn delightful it all is, to be sure" Charles Crumb
Security is obscurity ... is an old proverb that describes today's terrorists attitude very good, they try to hide their data in the really big mass of data floating trough the net every day. The U.S. have IMO a good chance of fetching up such information, when they do it the right way ... but what is the right way? :-)
- I'm not stupid, but somethimes I do things that make me look stupid!
Life sucks.
Well, of course they don't have people left reading the open literature. That's because they shot them all in Three Days of the Condor!
Unfortunately, our spies and our satellites have lost touch with reality, for they collect less than 10% of the relevant information that we must digest to understand the complex multi-cultural world that is now capable of producing very wealthy and suicidal terrorists.
There's a good reason for the above mentioned figure... While I agree that there's a lot of useful information on the net, there's also a lot of crap.
Any intelligence agency looking to filter out the 99.9% of nonsense that's out there to glean the remaining .01% of useful information faces an incredible challenge. That's not to say that it's impossible, just very, very difficult, time-consuming, and expensive. For every real threat being posted on the net, there are tens of thousands of harmless, steam-blowing rants posted. And how do you decipher between the two? Do we now get into investigating every idle threat someone posts on a bb, or in a chat channel? The sheer scope of this project would daunt even the most dedicated Government sanctioned snoop.
Just as an example, sift through the comments of a /. article at -1, and you're guaranteed to find at least one or two flame-ridden rants about god knows what. Or look at your own past history. How many 'harmless' comments have you made about the stupidity of this or that idea, or how you'd like to kill that person for doing this? You know its harmless. Anyone who knows you probably thinks the same. But how does someone completely outside of your community know?
The article does make a few valid points, however:
Shocking as it may seem, our intelligence community does not routinely strive to identify the top people in the world (not just Americans) on the various topics of concern -- from terrorism to the environment to human trafficking to corruption to disease and public health -- with the result that our analysis tends to be shallow and incestuous, relying on the same consultants again and again.
I think just about anyone who lives outside the US looking in (as I do) would agree with this statement; one has only to observe the lack of knowledge American citizens display with regards to the rest of the world to see that this attitude is quite widespread, and probably does affect intelligence gathering. Raise your hand anyone who's seen the (Canadian) 22 minutes special "Talking to Americans". It's rather depressing actually, to think that so many people, including prominant politicians, could believe that Canada works on a 20 hour clock, or that we're going to change the country's name to Chicago (I'm not exaggerating either... quite a few people were taken in by this).
The Recommended Open Source Initiatives proposed in the article are interesting, though idealistic. One example: Digital History Project ($5M) to digitize and translate key Islamic, Chinese, and other foreign language historical, political, economic, cultural, social, and technical materials.. Having been involved in translation projects (French to English and vice versa) myself, I think he has seriously underestimated how much this would cost..Translation is an incredibly difficult and time-consuming activity; it's not a simple matter of babelfishing an article. Localized phrases and slang do not translate well from one language to another.
Before you can even begin to sift through the plethora of information, you'll need people that are very net and tech savvy. Combining tech skills with those of an intelligence agent is just the beginning. I won't even go into the thorny privacy issues that could be touched on here... that's just a political bomb waiting to go off.
Moral indignation is jealousy with a halo - H. G. Wells
To a point I can understand that the CIA and such may not think that open public information isn't usefull but for some reason I just don't completely believe this article. As we continuously find out all the stuff our government was doing 20 or 30 years ago that no one though they were doing or was even possible the more I think we should doubt people who put such strong numbers on our govertments secret practices. Most people accept that the CIA is probably at least 10 years ahead of the times technology wise (mail for cryptographic reasons I'd assume). If these are the same people that are gathering intelegence I doubt they would be so hard nosed as to ignore anything but "relevent sources of information"
"actors" are people that act upon, and interact with, things/people/whatever. You also have various "entities" which can be, but aren't necessarily, actors.
Best Slashdot Co
Um, you guys realize that the term "Open Source" here has absolutly nothing to do with Open Source Software, right?
I mean, it's kind of intresting, I guess. But it dosn't really have anything to do with the OSS "movement" or anything. It certanly isn't some fallout from the "revolution".
autopr0n is like, down and stuff.
I'm not saying that Open Source Intelligence is a bad thing; just that the gullibility index of interpreters will be a major fact into how useful it becomes.
I've wondered who gets the task of monitoring this stuff. Can you imagine what it must be like to have your job being to read Slashot? (officially, I mean, not counting all the people who make it their job de facto ...)
Sig: What Happened To The Censorware Project (censorware.org)
Here's a somewhat in-depth think-tank article, "Considering the Net as an Intelligence Tool (Open Source Intelligence) (pdf format) I found that focuses on Internet-based OSI and espionage, dating back to 1996.
Error:
The reason the US doesn't do more is it's too risky. Speaking as an expert (I saw 3 Days of the Condor _twice_), it's clear that we're simply putting bookish analysts (who happen to look like Robert Redford) at risk when the evil oil cartels discover the location of their open source reading rooms.
Also, given that the bookish chaps have to resort to phone phreaking and even kidnapping to thwart Max von Sydow, OS int would put homeland security at risk.
Nope, it's just to dangerous to risk.
A.
And the intelligence agencies ignored them.
That may be because intelligence agencies have been in the business of collecting intelligence for a few hundred years. And the congressional committee has never been in the business of collecting intelligence. So maybe, and I may be grasping at straws here, but, maybe, the cia knows more about collecting intelligence than a reporter for time magazine. (audience gasps)
Before you discard my opinion, what do you think about congressional committees when they discuss the Harmful Effects of Video Games? Or the horrors of Pirated Music? Just because a few congresspersons decide the spooks don't know what they're doing doesn't mean that the congresspersons were right.
I should also note that I met somebody once whose job was to work for the CIA and search the internet. I'm sure they are using osi to the degree they feel necessary.
Jack Valenti and the MPAA are to technology as the Boston strangler is to the woman home alone
I remember 5 years ago sitting in the Navy/Marine Corps Intelligence training center as a young PFC(Private First Class) and having drilled into our head that we must look at open sources to develop a full intelligence picture. Even our closing practical app exercise included simulated CNN and reuters news bits for us to analyze. Exercises I was involved with when in the fleet included those. On 9/11, our intelligence officers first orders included one to keep the TV on and tuned to CNN and MSNBC 24/7 for the forseeable future.
As for not translating korean stuff, well I remember these were unclassfied, if you can ever track down some of the north korean radio transcripts, they are worth a laguh and a look into the North Korean mindset.
While open source intelligence may not get the attention it deserves at the highest levels, it isn't as badly neglected as this article seems to think.
I'm not sure how detailed I can legally get on this point, so I'll be general. Some of this reluctance to use false sources may be due to false alarms such things have caused in the past. Some of our enemies will manipulate the open sources in an attempt to cause us to react to a perceived threat. It worked for us in WWII with the whole landing at Calais deal, and worked against us recently in the middle east(thats the bit I can't be more specific on).
(the U.S. does not have military maps for 90% of the world.)
this is bunk
there are four levels of military maps the lowest is available openly 1:100000
the next layer 1:1000 is supposed to be opened but recent events will probably change that
layers go 1:100
and 1:10 in terms of coverage accuracy
The US has the world maps covered, as do the French, Russians, UK etc. most gov'ts sell and trade data (for a price) to most anyone with enough geld.
dgd
looking good from here
--
the strongest word is still the word "free"
The Drudge Report - Hey, he links to the important and interesting stuff in the mainstream media and breaks the stuff they won't report. What's not to like?
Stratfor.com - Great, clean analysis that is hard to find elsewhere. Drawback - has one free article each day on their site; full access requires a subscription of $80-$120. Still cheaper than ignorance or Jane's.
Debka.com - Provides interesting intel on the Middle-East from an Israeli perspective.
Anyone else have any favorite newshound/intel links?
In the article:
Shocking as it may seem, our intelligence community does not routinely strive to identify the top people in the world (not just Americans) on the various topics
of concern - from terrorism to the environment
to human trafficking to corruption to disease
and public health - with the result that our
analysis tends to be shallow and incestuous,
relying on the same consultants again and
again.
I am not sure how many of you have looked into getting a clearance. There are some serious ethical choices that you need to make in order to get one. Some of these include:
1. Restricted travel.
2. Not being able to associate freely with non US citizens.
3. (probably the most important for the academic types). Depending on what clearence you get and from which agency anything that you go to publish will have to be peer reviewed by people in the intelligence community before it can be published.
THe reason that this is important is that the intelligence agencies can not just talk to any Tom, DIck or Harry about somethings. Many of the academic leaders can not justify the restrictions on personal freedom, so they choose not to work in the intelligence community.
It would be great for them to pull resources off of everybody but, they are limited to what they can talk about and it would be easier for those opposed to our interests to learn what we are up to.
I agree that it would be great to be able to do this but there are several things that need to be resolved before hand.
Either give it away or get top dollar, but never sell yourself cheap.
What this guy proposes sounds all well and good, but once all his specific proposals are implemented, the U.S. government will have a very robust methodology in place to gather information on EVERYONE, not just terrorists and evil-doers.
If you read the article with a bit of a paranoid mindset, you'll realize that the tools described, if they are ever developed, will enable any government to have a comprehensive picture of all things happening in the world, good or bad. Given the fact that most governments have very little regard for personal privacy and freedom, especially after September 11, what is going to prevent an Orwellian use of information against citizens?
The U.S. government, in particular, has enough money and enough self-righteousness to become a terribly effective Big Brother, and making a clear distinction between dangerous zealots and regular citizens with unpopular views will become ever more blurred.
Morel
sounds like it was a bogus committee setup to get more funding for the military. is anyone surprised the CIA didn't listen to a bunch of political babble mouths and took the money to do what they wanted? talk about mis-information.
The Atlantic Monthly magazine covered this a month ago in a story that's on the web at http://www.theatlantic.com/issues/2002/02/fallows. htm
And why should anyone be surprised that they're using common sense _as well as_ your spare CPU cycles to find out what's going on outside those smoked-glass windows?
...comes the title of the article:
"Open Source Intelligence"
The speed of time is one second per second.
Sifting through publicly available information to find important trends, threats, etc is a wonderful no-brainer type of idea. He says it is not being done. Why and are there examples of where publicly available information could have been helpful but was not used? (Maybe it is not that helpful as it seems it would be.)
Check out it out here: http://osi.theofficersclub.com/
Robert D. Steele has been to many hacking conferences over the years. He has been a force in the movement to reform the US Intelligence Community and presents many fresh ideas.
Unlike most of the rest of the Intelligence Community, he is open to us and our views.
The term "Open Source" has had a meaning in the Intelligence field long before it came into vogue as a software development movement - RDS makes an analogy - that open intelligence sources and methods are more trustworthy (than closed sources) for the same reasons that open source programs are.
I first read the article on Sunday afternoon. It caught my attention because, like many on /., I made the right associations and tried to match "intelligence" with "open source" in the espionage sense. I was disappointed after reading 2/3 of the article, and didn't finish it. This was written by someone mostly grandstanding. The author focused too much on gathering data from all sources without giving enough thought to interpretation of those data.
I had the privilege (misfortune?) to work with a few intelligence types. When you talk to the people in the field, not to the public figures, public affairs wags, or the pundits, you will almost universally get the same two answers, rated in order of importance:
The open intelligence article advocates only points 2 and 3, and barely touches on 1. Just like in coding, where the problem is not writing code but writing code that does something useful, intelligence is all about interpreting the data so that policy and actions may be appropriately channeled.
Dr. Ray S. Cline (former deputy director, CIA; look him up) once said that the world needed fewer spies and more critical thinking (I'm paraphrasing here a bit). Everyday disasters and attacks that could have been prevented still happen because there are too many toys and budgets and bureacratic fiefdoms to protect and there aren't enough ears who understand the bad guys' language, not enough cooperation between three-letter agencies, and not enough brains focused on making sense of the data gathered through various channels.
Thus, while part of the problem is gathering data, making sense of it is what will prevent another catastrophe like the terrorist attacks last September.
Ehttp://eugeneciurana.com | http://ciurana.eu
I've long been an advocate of something very similiar to what he's proposing. However, the single most limiting factor in todays intelligence community is one of employees. Satellites aren't able to record conversations in swahili in a back room of shanty town a. You need to have someone physically there (and capable of speaking the language/culture, plus be able to blend in as one of the locals). In order to make use of most of these 'open source' intelligence avenues, you're going to have to have manpower at their locations to 1) identify them 2) collect them.
today's intel community suffers from a lack of qualified individuals who are able to successfully staff locations around the world. Identifying openly available sources of information is a great idea, but we've got to get the people in there to do it.
1. The CIA, including its precursors, has only existed since WWII.
2. It's the JOB of our elected congressional officials to oversee & regulate the functions of ALL the government, including the intelligence agencies. You don't have to be a veteran spook to see a waste when the CIA spends tens of thousands of dollars for information that's published in some Pakistani daily paper.
You met someone once whose job it was to search the net for the CIA? Congratulations.
The only tool you've got against psychosis is experience.
Body of Secrets by James(?) Bamford is very enlightening regarding the intelligence operations of the nation over the last few decades. Er, sorry, no Open Source stuff, though.
Check it out:
http://www.mercyhurst.edu/Academics/riap.htm
It is the only one offered outside of Isreal for an undergraduate degree.
OSINT is strangely similar to competitive intelligence: Check out http://www.scip.org/
LL
----
Times flies even when you're not having fun.
I'm sure I'll get marked as Flamebait - this going against the Slashdot communities consensus and all - but this really shows how much the objections to Carnivore are a straw man.
If the U.S. intelligence and law enforcement communities can't even keep up with publically available information posted on the web, how the hell are they going to find time to snoop through people's encrypted porn? (Or whatever the hell else they actually bother to encrypt.)
Face it, it takes a hell of a lot to get an FBI agent to bother a judge about your private files.
Or, as Dennis Miller put it: "The biggest conspiracy is that there is no conspiracy. Nobody is out to get you. Nobody gives a shit whether you live or die. There, feel better now?"
The secret service asked the guy some questions after he described how he would infect the Vice President with airbourne diseases. It was more likely a tip from someone to the Secret Service than the Secret Service dilegently reading K5 (although, after that, I bet they read it now!). Unfortunately, the Open source method works better against windbags on K5 than it does against read terrorists...
I am not a number! I am a man! And don't you
After reading several of your comments to this article, you need to write a (suitably sanitized) article for kuro5hin.
Best Slashdot Co
The idea is wonderful, but a little bit naive, To achieve an objective, it is essential to consider the cost of the method.
To collect the open information, process them, make decision on what is important, what is not, involves enormous cost (in term of money and time), This is what economists call transaction cost. If those effort is costless, all the firms will collect all the consumers info, do the direct marketing. You will continue to put your effort into certain endeavor if it yields more results than you put in. otherwise, you will put your effort into other method with better prospective to get more out of your effort.
What at issue is not that CIA, FBI have less information, but how to do with all the collected information, and act on them. The recent report of 7 out 19 hijackers being noticed before 9.11 is the case for the point.
After all, the computer world has lots of oddities that might seem nonsensical to someone unfamiliar with the details or history of a particular technology.
I am seriously concerned, however, that it looks like the CIA will resume covert investigation within the US... something that has been banned for decades. GWBush's father, when he was CIA Director, prior to being President, resisted this ruling and always wanted to eliminate the restriction. It looks like his son may succeed.
As we become a more computerized and networked society, it seems likely that the once-obvious boundaries of our country will become more blurred. For example, the NSA has been evesdropping on US phone communication without warrants for years, even though they are not allowed to gather information about the US from within the US... just like the CIA. They have gotten around this limitation by using satellites that capture signals escaping into space; mostly microwave from repeater towers.
I have no conclusions or suggestions... just a little food for thought. The most dangerous threat for any democracy is the complacency of its citizens. I am really glad to see that slashdot and other similar tools are thriving. The most dangerous threat to any despot is free communication between the people.
slashdot is sucking mad ballz today.
You know, outside of our own narrow world, not all the lexical phraseaology we are accustomed to carrys the same meaning. In short, he doesn't mean Open Source in the GNU sense you fracking idgits. OSINT in this case refers to non-standard methods of intelligence gathering. Traditional electronic intelligence and human intelligence methods would remain a foundation, but not the exclusive domain of intelligence gathering. Are any of you reading the article before posting your inane rants?
You can all unwad your panties and stop tripping already.
RANT ON
Jaw Jaw rather than War War.
People should enter the political arena and all views should be rationalised and criticed - both internally and externally to the groups concerned. The world is a big negociation, what worked in the north of Ireland [big political fudge that most people voted for] may work elsewhere (even in the U.N., come on guys pay up).
Force people to examine and justify their and their opponents beliefs as part of a negociating position. Keeping U.S. 'intelligence' secret is against the politicial process, let the U.S. Intelligence fights it out with other peoples (even non US) rationalised view of the world.
RANT OFF
Be Free: Free Software Tuition
Damn, that's why my editor gets so frustrated with me...
Ehttp://eugeneciurana.com | http://ciurana.eu
unless you take Microsoft's use of BSD code as "open source"
Unlike some Linux developers
The CIA, FBI and NSA should just use Google.
The REAL jabber has the user id: 13196
What you do today will cost you a day of your life
This comment has been removed since it was clearly in violation of Title 18, United States Code, Section 871 (Threats Against The President) and / or Section 875 (Interstate Communications: Extortions / Threats). You can Read More Here. We're sorry to have to do this, and while we don't necessarily agree with this, it is still the law. When the Secret Service gets involved, we don't have many options. We appreciate your understanding in the matter. Please call (202) 406-5000 if you have any questions.
BEEP!
This exact comment has already been posted. Try to be more original...
MEEP!
Open source intel is a vast field, any written piece stateing anything about a government or its activities is valuable intellegence to a forign power. This is a double edged sword enough foregn govt's can look at CNN, The New York Times, hell, even /. and get a good idea of how things are going in this country. The problem is sorting out what is redundant, and what pieces of intellegence are accually worth the time and effort, an accually rather small percentage. a very time consuming process, i don't know how many of you are paying attention to military news, but the us army just involentarilly extended all mi posisitons in an effort to combat this problem (again another piece of open source news better not known by people who don't have any business there, but it needs to be said) so the gov't is workin the problem, but the amount of open souce intel out there is dauting, and combined with all the accual human intellegence, imigary intellegence and signal intellegece thats constatly coming in, unfortunitly not everything can get the full attention it deserves.
kablooey
They do a little more than that, but that's their primary focus. Their mission statement is two paragraphs, one about protecting officials, their families, etc, and the second one follows:
Just a tidbit.
That what was all this school was for... to teach us how to solve our own problems. -- janeowit
The information available from public sources is not complete. Because of this, things that aren't problems will look like problems and things that are problems won't in way too many cases. There is little way to discern what needle in the haystack is real and which axes aren't. Hindsight will always tell you, but never implies that any filtering could have shown you that some nugget was in fact a nugget beforehand. If you adjust your filter to deliver that particular nugget to you next time, it will just deliver that nugget and it still may not in this new case truly be a nugget.
This is a problem that could truly consume infinite resources and hurt more often then it helps. If you destroy 1000 lives due to false alarms and catch one, you've not done the world a favor. Safety via this type of prevention is not achievable. Its like trying to build a strategic missile defense,,, there will always be a hole for the enemy to exploit much more cheaply than you can plug it. Therefore, the enemy always wins.
How was the Soviet Union defeated? They were drawn into an arms race that their economy couldn't afford.
That is exactly what is now happening to us. We've got this crazy idea that the world can be made safe, and our enemy is using it to draw us into an arms race that our economy can't afford. Where they spend 100s of thousands of dollars to create an attack, we are spending billions to prevent another.
The solution is to realize that defense can never be the sole weapon in a war. If all you have is defense, you'll lose. We need to relearn the idea that offensive acts must be met with strong punitive judgement. Only then will we be able to get back to a society where we don't have to lock our doors. You may think that locking your door is cost effective, but in reality, the cost of locks in this country is far greater than the total cost of crime (and, yes, I'm talking literally).
CNN has been caught employing CIA agents and many accuse the CIA of employing CNN reporters.
Insanity is the last line of defence for the master diplomat. But you have to lay the groundwork early.
Lord knows I've posted some stupid things... Thank God none of them were moderated up to 5 for everyone to see...
Sucks to be you
Possibly one reason his proposal hasn't been accepted is because he doesn't seem to be pitching it very well. His rhetorical style is weak, and doesn't effectively convince me of what hes trying to say. He does make a stand on his authority as an expert, but this comes without any other really supportive logic. The first questions I would ask, if I were one of the Cabinet members he was trying to reach would be "How did you arrive at these numbers for different departments? Are you qualified to know the things you have stated as facts in this proposal?"
Check my Go-related blog for beginners: DGD
Sorry, I'm not an Anonymous Coward, I just can't get logged in at the moment. We've been making a number of resources available to the Internet community, and anyone here that's interested should take a look at the old 7Pillars website at http://www.7pillars.com/ or a wealth of new material at http://www.metatempo.com/publications.html
Among this material is a piece (look for the Ontology) that explains what's gone wrong with U.S. intelligence, and provides a next-generation look at an intelligence process. We also provide our brief intelligence training manual.
Michael Wilson
I suspect that most governments in the world generally neglect to use ANY sort of intelligence.
And if intelligence was GPL'd, Microsoft would campaign against intelligence because it gets in the way of innovation. </joke>
1. Open Source
2. Government
3. Intelligence
Occasionally two of these are combined, but all three? That would be on the day when pigs fly.
Isn't that a new oxymoron?
WireDog goes for the BoneFlower ... does he bite?
From the article:
Funny, that makes them sound just like the media. Oops, was that my outside voice?
He spoke at H.O.P.E. many a year ago stating that hackers are "law-abiding citizens who have immense potential to contribute to society". Yes, he is a spook. Or was a spook. He resigned from the CIA out of frustration with their information gathering skills.
He believes that publically available information is often more useful and accurate than information the CIA/NSA/DIA blow huge wads of our cash to collect.
"I had spent eighteen years as a professional intelligence officer," he says, "and discovered that a whole lot of classified data wasn't really there. We just had a whole bunch of facts about Soviet missile silos. Nothing on the Third world, for instance. At the Marin Corps Intelligence Center we were spending $2 million a year on a system for accessing classified data from the CIA, NSA, and DIA--and I found that for $25,000/year I could get beter data from open sources."
I knew it was him as soon as I saw the Slashdot headline. Very unique character. :)
This word "Insightful" - I do not think it means what you think it means. Try "uninformed" instead.
Geeze, could the font be any smaller? Is this their way of enticing us to buy the dead tree version?
If all this should have a reason, we would be the last to know.
I most enjoy /. when the best of the posts provide more benefit than the posted article(s), and as is the case on this thread.
I spent the greater part of a day in deep discussion with a fellow who traversed WWII as a polish soldier brought into Russian intelligence, who then moved on to British Intelligence, and ended his career having much to do with the founding of the Canadian Intelligence Service. A fair measure of our talk centred around the need for covert intelligence gathering in the face of laws either protecting the rights of citizens or curtailing the access to information. The facts of the world as it is brought home the inescaple need for covert intelligence gathering but also the prerequisites for said actions. The Russians, my military intelligence type relatives tell me, were (in)famous for garnerning intelligence at social gatherings by simply working the cocktail circuit and baldly asking pointed questions of targets in the offhand manner of party chatter. I'm told it was a very effective ploy. Working sources of information requires access and understanding, more especially of the social mindset of those under observation, and to this end the philosophical gap between Open Source Software and Intelligence gathering can be bridged by the seemingly trival observation that intelligence is most effective in an open enviornment where the information is freely available. Gadetry comes into play more so where direct access is not available. The recent reports of 23 bugs found on the American made jet for the Chinese President is a case in point.
Perhaps most telling is the obvious fact that intelligence gathering requires we better understand one another.
cheersheuristic algorithm seeks stochastic relationship
do you mean the class that divides itself into various factions, where one faction will inevitably claim they represent the 'common man' while they are lifelong recipients of the caviar and limousines? The ones that instead of practicing what they preach on helping the common man, actually destroy hard working people in the name of 'freedom'? If you refer to this as the class, then you are right. However, I do indeed wonder what the situation looks like to you when you consider how this inefficient red tape mess of a system is more of the responsibility of the voters (the same voters that sheepishly let themselves be led into believing that a proven corrupt, lying, manipulating, dishonest, abusive and dangerous elitist politician somehow actually cares about the little guy). Its like when you see someone drive all over town like an idiot, then after wrecking and hurting his car and himself actually sues and WINS on the basis that the car should have somehow been engineered to detect and stop his stupidity.
using OSINT to gather info on the CIA - http://www.trustmatta.com/services/docs/Matta_Coun terintelligence.pdf
-- ken williams
Controlling huge amount of information is not actually that hard. Consider the Internet: Vast, hard to seek anything from. Add absolutely mandatory key words from the standard word list for every document. Add expiration date, category systems.
There was an article in Slashdot about mapping Internet a while ago. One of the demo pages had a 3d system that was developed for some intelligence agency. One (MIT student or something like that) guy had made a 3d mapping tool that created spheres on the information how close words appear in document(s) to each other. Nice.
You have just to present the data in more intuitive way and in some searchable form and you will have no problem with all of the world's intelligence data. Trust me. How does Echelon (system that handles a LOT of data) work? It tracks for certain keywords, not for the whole conversations and such. It is also a highly disintegrated system, consisting of dozens of parts. Let them all keep their lot clean.
If someone gave me some funding, we could do magic to the Internet. (Referring mostly to the www with the Internet.) Someone out there might have in a way done it already.
Business needs intelligence information all the time. I suspect they have many automated tools to help them keep tabs on their market, on their competitors, technologies, etc.
Reminds me somewhat of one metric of the economy:
"Provided by the management for your protection."
There is an interesting element of debate in the book (IIRC) regarding whether or not all the spy and cryptographic stuff that was done during the war was worth the cost when much of the information alledgedly could have been deduced from open sources.
There is a branch of the government called FBIS (Foreign Broadcast Information Service) who's explicit charter is to process open source information. See http://199.221.15.211/ for details. This organization has existed for many years and does a great job with a very limited budget. I used to work for them and was involved in the difficult job of trying to determine what to collect knowing the budget just wasn't there to collect/analyze/translate/catalog/summarize every bit of open source there was. Interestingly enough since FBIS was formed back in the 1950's (I recall) their usage of the term 'open source' interestingly was widely used well before the internet was formed and well before the open source movement. There is a wealth of information and these guys have the market on attempting to collect it all and sort for the tidbits of importance.
This is old news, most of which was solved in the Cold War. The problem is not so much collecting, and translating information, as getting U.S. policy makers, Departments and Agencies to use the public sources, and use them correctly.
Historically, the US Government has (and may still do) purchase many periodicals, and maps abroad and shipped them to Washington DC, in quantity.
Since early in the Cold War, the U.S. Government via its Foreign Broadcast Information Service, a.k.a. FBIS, has collected, translated and published mountains of TV and radio broadcasts and newspaper articles. Many Universities have the unclassified version of FBIS on CD-Rom if you want to search it.
Journal translations are a more expensive problem. During the Cold War, U.S. translators and scientist worked through complex translations of all kinds. The critical break throughs in stealth designs came from Soviet research, translated and provided to the Lockheed Skunk Works. Skunk Works designed a diamond shaped plane that became the F-117 Stealth Fighter Bomber, the only aircraft to that date designed by electrical engineers.
Today the translation problem is its decentralization. Translations of uncommon works are selected and translation paid for by smaller itelligence offices and units. To my limited knowledge, those translations are bartered and lent between small offices, not centralized and indexed and free for all Intelligence Agencies. This sounds to me like a cost saving exercise to keep the cost of many security cleared translators down.
As for commercial sattelite coverage, U.S. Space Command identified this problem. The U.S. government has a mixture of legal shutter control, such as for images of Isreal, and monopolistic purchases of commercial images of war zones, such as exclusive buying all Ikonos images of Afganistan. This has the benefit of providing a pool of images for release to the media, while securing U.S. Sattelite technical capabilites.
As for maps, although we buy more foreign maps all the time, U.S. maps for much of the world are as good as they get. This is not to declare that they are perfect, but for example, the CIA street map of Moscow was the ONLY Moscow street map that was worth a damn.
This topic is not just old news, this is Cold War news.
-Nathaniel
Steele is the founder of Open Source Solutions, Inc., Write an "big boogyman article", offer it to Time Magazine, where you know it will get published and drive TONS of traffic to your site where you sell OSINT services.
"To many eyes, all bugs/intelligence problems are shallow."
I.e., the knowledge and expertise are out there. It's a question of making use of them, or of relying only on "proprietary" closed methods.
I do this for a living.
/., you read dozens of news sources daily, pull the nuggets out and present them for discussion. Then you do deep research on topics that of interest or help to understand things that could eventually harm your employees.
Its good work if you can get it, Its a lot like
"--by exploiting these open sources we can create open source intelligence, or OSINT, suitable for informing our public as well as our state and local authorities and our international partners, as to the threats to our nation."
It used to be called 'content analysis' and it was used as far back as WWII. You could tell -something- was going to happen if trains were consistently late, if people were getting turned back from an area, if grain was in short supply here but not there.
Example: Imagine you want to know when a vessel is leaving port. Who do you ask to find out?
The answer is the local whores. They know, and they'll tell.
This is easy stuff, but it helps when you can analyze more content. Trouble is there's so much more content now that it's hard to figure out which sources are useful; therefore you have to sort of vote (correlate). Very much like detecting cracking attempts.